Work support system, information processing terminal, and work support method

Abstract

To easily display information about a member related to an object in a manufacturing plant on a field of view of a worker in a display unit.SOLUTION: A work support system 2 includes an acquisition unit 21 and a display processing unit 22. The acquisition unit 21 acquires reference position information indicating a position of a production apparatus 3 (object) arranged on a floor, and relative position information indicating a relative position of a member related to the production apparatus 3 with respect to the position of the production apparatus 3. Based on the reference position information and the relative position information acquired by the acquisition unit 21, the display processing unit 22 displays member position information indicating a position of the member in superimposition on a field of view of a worker in a display unit 11 of an information processing terminal 1 carried by the worker.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a work assistance system, an information processing terminal, and a work assistance method for assisting a worker in work using objects arranged in a manufacturing factory. [Background technology]

[0002] Patent Document 1 discloses a solder replenishment position instruction system that can use an augmented reality device to instruct a worker on the position where solder should be replenished. In this system, when the worker visually recognizes a recognition target provided on a printing press, a display unit displays virtual information including the position where the worker should replenish solder, superimposed on a real mask that the worker visually recognizes via the augmented reality device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-174420 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a work support system and the like that can easily display information about components related to objects in a manufacturing factory in the field of view of a worker on a display unit. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, there is provided a work assistance system including an acquisition unit and a display processing unit. The acquisition unit acquires reference position information indicating a position of an object placed on a floor of a manufacturing factory, and relative position information indicating a relative position of a component related to the object with respect to the position of the object. The display processing unit displays component position information indicating the position of the component in a field of view of the worker on a display unit of an information processing terminal carried by the worker, based on the reference position information and the relative position information acquired by the acquisition unit.

[0006] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of the system, the method, the integrated circuit, the computer program, and the recording medium. The recording medium may also be a non-transitory recording medium. [Effects of the Invention]

[0007] The work assistance system and the like according to the present disclosure has the advantage that it is easy to display information about components related to objects in a manufacturing factory in the field of view of the worker on the display unit.

[0008] Further advantages and effects of one aspect of the present disclosure will become apparent from the specification and drawings. Such advantages or effects are provided by some embodiments and configurations described in the specification and drawings, but not necessarily all configurations. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the overall configuration including a work support system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a floor map. [Figure 3] FIG. 3 is a diagram showing an example of a display of member position information. [Figure 4] FIG. 4 is a diagram showing an example of display of contour information. [Figure 5] FIG. 5 is a diagram showing another example of display of contour information. [Figure 6] FIG. 6 shows an example of how the type-1 guide notification is displayed. [Figure 7] FIG. 7 is a flowchart showing an example of the operation of the work support system according to the embodiment. [Figure 8] FIG. 8 is a sequence diagram showing an example of the overall operation including the work support system according to the embodiment. [Figure 9] FIG. 9 shows an example of how the type-3 guide notification is displayed. [Figure 10] FIG. 10 is a diagram showing another example of the display of component position information. [Figure 11] FIG. 11 is a diagram showing yet another example of the display of component position information. [Figure 12] FIG. 12 is a diagram illustrating an example of a work instruction definition table. [Figure 13] FIG. 13 is a sequence diagram showing an example of the overall operation including the work support system according to the work instruction definition table. DETAILED DESCRIPTION OF THE INVENTION

[0010] A work assistance system according to a first aspect of the present disclosure includes an acquisition unit and a display processing unit. The acquisition unit acquires reference position information indicating the position of an object placed on a floor of a manufacturing factory, and relative position information indicating the relative position of a component related to the object with respect to the position of the object. The display processing unit displays component position information indicating the position of the component superimposed on the field of view of the worker on a display unit of an information processing terminal carried by the worker, based on the reference position information and the relative position information acquired by the acquisition unit.

[0011] This has the advantage that by referring to the reference position information and relative position information, the position of the component relative to the worker's current position can be recognized, making it easier to display information about the component related to the object in the worker's field of view on the display unit.

[0012] In a work assistance system according to a second aspect of the present disclosure, in the first aspect, the acquisition unit further acquires shape information indicating a shape of the member, and the display processing unit displays contour information indicating a contour of the member superimposed on the field of view of the worker on the display unit based on the shape information acquired by the acquisition unit.

[0013] This has the advantage that the outline of the component is superimposed on the worker's field of view on the display unit, making it easier for the worker to grasp what component is being worked on.

[0014] In a work assistance system according to a third aspect of the present disclosure, in the first or second aspect, the acquisition unit includes an imaging unit that captures an image of a marker placed on the floor, and the acquisition unit acquires the reference position information based on an imaging result of the imaging unit capturing the image of the marker.

[0015] This has the advantage that the worker does not have to search for the object to be worked on, because reference position information can be obtained by bringing the marker into the worker's field of view on the display unit, even if the object is not within the worker's field of view on the display unit.

[0016] In a work assistance system according to a fourth aspect of the present disclosure, in any one of the first to third aspects, the object is a production device. The acquisition unit further acquires error information indicating an error related to the production device. When the acquisition unit acquires the error information, the display processing unit displays a type-1 guide notification superimposed on the display unit in the worker's field of view, guiding the worker to the production device where the error has occurred.

[0017] This has the advantage that by seeing the Type 1 guide notification, the worker can more easily reach the production equipment where the error is occurring, even if the production equipment is not visible from the worker's position.

[0018] In a work assistance system according to a fifth aspect of the present disclosure, in the fourth aspect, the acquisition unit further acquires behavior information indicating the behavior of the worker with respect to the production equipment, and the display processing unit displays either the component position information or the first guidance information on the display unit, superimposed on the field of view of the worker, based on the behavior information acquired by the acquisition unit.

[0019] This has the advantage that appropriate information can be superimposed on the worker's field of vision on the display unit depending on, for example, whether the worker is approaching the production equipment or whether the cover of the production equipment is open (i.e., whether the inside of the production equipment is visible or not), which makes it easier to improve workability.

[0020] In addition, in the work support system according to the sixth aspect of the present disclosure, in the fifth aspect, when the acquisition unit acquires information indicating that a cover of the production equipment has been opened as the behavior information, the display processing unit switches the information to be displayed on the display unit superimposed on the worker's field of view from the first guidance information to the component position information.

[0021] This has the advantage that, for example, when the cover of a production device is opened, the information displayed on the display unit superimposed on the worker's field of vision can be switched to appropriate information, which makes it easier to improve workability.

[0022] In addition, in the work assistance system according to a seventh aspect of the present disclosure, in any one of the fourth to sixth aspects, the acquisition unit further acquires work element position information indicating the position of the work element corresponding to the error, and the display processing unit superimposes and displays a second guide notification for guiding the user to the work element in the field of view of the worker on the display unit before displaying the first guide notification.

[0023] This has the advantage that when the worker sees the type-2 guide notification, it becomes easier for the worker to reach the location where the work tools are located.

[0024] In addition, in the task assistance system according to an eighth aspect of the present disclosure, in any one of the first to seventh aspects, the acquisition unit includes a communication unit that receives the relative position information from the object.

[0025] This has the advantage that by obtaining relative position information from the object, even if the object is, for example, a production device and the position of the component changes depending on the production status of the production device, the position of the component displayed on the display unit in the worker's field of view can be updated in real time, making it easier for the worker to grasp the position of the component.

[0026] In addition, in a work support system according to a ninth aspect of the present disclosure, in any one of the first to eighth aspects, the component position information is divided into multiple hierarchies each having a different degree of detail indicating the component position, and is managed in correspondence with the content of the work instruction.

[0027] This has the advantage that it becomes easier for the worker to grasp the position of the component compared to when the position of the component is shown to the worker by a single piece of information.

[0028] In addition, in the work support system according to the tenth aspect of the present disclosure, in the ninth aspect, the display processing unit displays information indicated by one or more of the multiple hierarchies in the worker's field of view on the display unit, superimposed on the information in accordance with the content of the work instruction.

[0029] This has the advantage that it becomes easier for the worker to grasp the position of the component compared to when the position of the component is shown to the worker by a single piece of information.

[0030] According to an eleventh aspect of the present disclosure, there is provided a work assistance system including an acquisition unit and a display processing unit. The acquisition unit acquires shape information indicating the shape of a component related to an object placed on a floor of a manufacturing factory. The display processing unit, based on the shape information acquired by the acquisition unit, displays contour information indicating the contour of the component superimposed on the worker's field of view on a display unit of an information processing terminal carried by the worker.

[0031] This has the advantage that the outline of the component is superimposed on the worker's field of view on the display unit, making it easier for the worker to grasp what component is being worked on.

[0032] In addition, in the work assistance system according to a twelfth aspect of the present disclosure, in any one of the first to eleventh aspects, the object includes at least one of a production device and a shelf on which the component is stored.

[0033] This has the advantage that it is easy to display information about at least one of the components related to the production equipment and the components stored on the shelves in the field of view of the worker on the display unit.

[0034] An information processing terminal according to a thirteenth aspect of the present disclosure includes the task support system according to any one of the first to twelfth aspects, the display unit, and a sensor, wherein the sensor detects the position and orientation of the worker.

[0035] This has the advantage that by referring to the reference position information and relative position information, the position of the component relative to the worker's current position can be recognized, making it easier to display information about the component related to the object in the worker's field of view on the display unit.

[0036] Furthermore, in the information processing terminal according to a fourteenth aspect of the present disclosure, in the thirteenth aspect, the display unit is a transmissive display.

[0037] This has the advantage that the worker can see the real space through the display unit, making it easier to grasp the situation in the real space.

[0038] In addition, a work assistance method according to a fifteenth aspect of the present disclosure acquires reference position information indicating the position of an object placed on a floor of a manufacturing factory, and relative position information indicating the relative position of a component related to the object with respect to the position of the object. Furthermore, in the work assistance method, component position information indicating the position of the component is superimposed and displayed in the field of view of the worker on a display unit of an information processing terminal carried by the worker, based on the acquired reference position information and relative position information.

[0039] This has the advantage that by referring to the reference position information and relative position information, the position of the component relative to the worker's current position can be recognized, making it easier to display information about the component related to the object in the worker's field of view on the display unit.

[0040] In addition, a work assistance method according to a sixteenth aspect of the present disclosure acquires shape information indicating the shape of a component related to an object placed on a floor of a manufacturing factory, and based on the acquired shape information, displays contour information indicating the contour of the component superimposed on the field of view of the worker on a display unit of an information processing terminal carried by the worker.

[0041] This has the advantage that the outline of the component is superimposed on the worker's field of view on the display unit, making it easier for the worker to grasp what component is being worked on.

[0042] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0043] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, or step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept are described as optional components.

[0044] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Furthermore, in each figure, the same components are assigned the same reference numerals. Furthermore, in the following embodiments, expressions such as "same" are used. For example, "same" does not only mean completely the same, but also means substantially the same, that is, including an error of, for example, a few percent. Furthermore, "same" means the same within the scope in which the effects of the present disclosure can be achieved.

[0045] In this disclosure, the vertical direction is referred to as the Z-axis direction or up-down direction. In this disclosure, a direction on a plane perpendicular to the vertical direction is referred to as the Y-axis direction, left-right direction, or lateral direction. In this disclosure, a direction perpendicular to the Y-axis direction on this perpendicular plane is referred to as the X-axis direction or depth direction.

[0046] [1. Overall structure] First, the overall configuration including a work support system 2 according to an embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a block diagram showing the overall configuration including a work support system 2 according to an embodiment. Fig. 2 is a diagram showing an example of a floor map. The floor map shown in Fig. 2 is a 3D map that uses CG (Computer Graphics) to represent a floor F1 of a manufacturing factory where one or more production devices 3 are installed.

[0047] The work support system 2 is a system for supporting work by a worker U1 on an object (here, a production device 3). The "object" here refers to a work item used by the worker U1 in the work. In the following, unless otherwise specified, the object will be described as the production device 3. The object includes, for example, at least one of the production device 3 and a shelf 6 (see FIG. 9) on which components related to the object are stored. In particular, in the embodiment, the work support system 2 is a system for supporting work by the worker U1 to resolve an error that has occurred in the production device 3. The work support system 2 is applied, for example, to a floor F1 of a manufacturing factory where one or more production devices 3 are installed.

[0048] In this embodiment, the work support system 2 is realized by an information processing terminal 1 carried by a worker U1. In this embodiment, one or more production devices 3 are managed by a host system 4.

[0049] The information processing terminal 1 is a terminal carried by the worker U1. Here, "carried" includes being worn on a part of the body of the worker U1 (for example, on the head). In the embodiment, the information processing terminal 1 is a glasses-type wearable terminal, so-called AR (Augmented Reality) glasses. In the embodiment, the information processing terminal 1 is worn on both eyes of the worker U1, but may also be worn on one eye of the worker U1. The information processing terminal 1 includes a display unit 11, a sensor 12, and a work support system 2.

[0050] The display unit 11 is a transmissive display that transmits ambient light. Therefore, the worker U1 can see the real space through the display unit 11. In addition, various information is displayed superimposed on the field of view of the worker U1 on the display unit 11. Therefore, the worker U1 can see not only the real space but also holograms (or pseudo-holograms) that do not exist in the real space, such as images or characters. Hereinafter, the hologram and pseudo-hologram will be referred to simply as "hologram." Note that the display unit 11 may be configured to display various information superimposed on the field of view of the worker U1, and may, for example, project various information directly onto the eyeballs of the worker U1.

[0051] The sensor 12 detects the position and orientation of the worker U1. In this embodiment, the sensor 12 is configured by a 6DoF (Degrees of Freedom) sensor that detects, for example, acceleration in three axial directions (X-axis direction, Y-axis direction, and Z-axis direction) and angular velocity in rotational directions (roll direction, pitch direction, and yaw direction).

[0052] The production apparatus 3 is a device used to produce products. The production apparatus 3 is, for example, a component mounting apparatus, a printing apparatus, or an inspection apparatus. In the embodiment, the production apparatus 3 is a component mounting apparatus. The production apparatus 3 includes a parts feeder 30 (see FIG. 6), a mounting head 31 (see FIG. 3), a component recognition camera 32 (see FIG. 3), a nozzle changer 33 (see FIG. 3), a pair of conveyor rails 34 (see FIG. 3), an X-axis beam 35 (see FIG. 5), and a Y-axis beam 36 (see FIG. 3). The production apparatus 3 also includes components other than these components, but the description thereof will be omitted here.

[0053] The parts feeder 30 is a tape feeder that has the function of pitch-feeding a carrier tape holding components to be mounted. When driven, the parts feeder 30 supplies components to a component suction position by a suction nozzle of the mounting head 31.

[0054] The mounting head 31 is equipped with a plurality of holding shafts. A suction nozzle that can pick up and hold a component and can be raised and lowered individually is attached to the lower end of each holding shaft.

[0055] When the mounting head 31 holding the component moves above the component recognition camera 32, the component recognition camera 32 captures an image of the component held by the mounting head 31 and recognizes it.

[0056] For example, an unused suction nozzle is arranged in the nozzle changer 33. The nozzle changer 33 is used, for example, when replacing a used suction nozzle (including a defective nozzle) attached to the mounting head 31 with an unused suction nozzle.

[0057] The pair of transport rails 34 constitute a board transport path for transporting the board 5. The board 5 is transported in order along the pair of transport rails 34. Then, the mounting head 31 performs the task of mounting components on the transported board 5.

[0058] The X-axis beam 35 has a linear drive mechanism and is equipped with the mounting head 31. When driven, the X-axis beam 35 moves the mounting head 31 along the X-axis direction.

[0059] The Y-axis beam 36 has a linear drive mechanism, and is connected to the X-axis beam 35. When the Y-axis beam 36 is driven, it moves the X-axis beam 35 along the Y-axis direction. In other words, when both the X-axis beam 35 and the Y-axis beam 36 are driven, the mounting head 31 can be moved in either the X-axis direction or the Y-axis direction.

[0060] The host system 4 functions as a management computer. The host system 4 individually controls one or more production devices 3 by sending control signals to each of the one or more production devices 3. The host system 4 also manages the status of each of the one or more production devices 3 by receiving information from each of the one or more production devices 3. The "status" here may include, for example, the status of the production device 3 itself, such as whether the production device 3 is operating normally or has an error, as well as the number of products produced by the production device 3. The host system 4 also exchanges information with the information processing terminal 1. The host system 4 also has the function of relaying information sent from the production device 3 to the information processing terminal 1.

[0061] Furthermore, the upper system 4 stores in a storage device a component information database 41 that aggregates information on components (described later) related to the production equipment 3, and a work instruction definition table 42 (described later).

[0062] [2. Work Support System] Next, the work support system 2 will be described in detail. The work support system 2 is a system including a processor, a communication interface, a memory, etc. The memory is a ROM (Read Only Memory) and a RAM (Random Access Memory), etc., and can store programs executed by the processor. The configuration of the work support system 2 is realized by the processor executing the programs stored in the memory. In the embodiment, the processor and memory that realize the work support system 2 are shared by the processor and memory of the information processing terminal 1.

[0063] 1, the work support system 2 includes an acquisition unit 21 and a display processing unit 22. The acquisition unit 21 also includes an imaging unit 211 and a communication unit 212.

[0064] The imaging unit 211 mainly captures an image in front of the worker U1. The imaging unit 211 also captures an image of a marker (reference marker M1 or auxiliary marker M2) placed on the floor F1, as will be described later. The imaging range of the imaging unit 211 is the same as the field of view of the worker U1 or is wider than the field of view of the worker U1. The imaging unit 211 is realized by an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor. In the embodiment, the imaging unit 211 is shared with the image sensor provided in the information processing terminal 1.

[0065] The communication unit 212 is a wireless communication interface for wireless communication with the upper system 4. In this embodiment, the communication unit 212 is shared with the wireless communication interface provided in the information processing terminal 1.

[0066] The acquisition unit 21 acquires reference position information and relative position information. The reference position information is information indicating the position of the production equipment 3 arranged on floor F1. More specifically, the reference position information is information indicating the relative position of the production equipment 3 with respect to the current position of the worker U1. The relative position information is information indicating the relative position of a component related to the production equipment 3 with respect to the position of the production equipment 3. The production equipment 3 in the reference position information is the production equipment 3 that is the target of work by the worker U1. In the embodiment, the production equipment 3 in the reference position information is the production equipment 3 in which an error has occurred.

[0067] Here, components related to the production apparatus 3 include, for example, a part of the production apparatus 3, or a unit that is attached and incorporated into a part of the production apparatus 3. Components related to the production apparatus 3 also include products produced by the production apparatus 3, or parts that constitute part of the products. For example, if the production apparatus 3 is a component mounting apparatus, the units may include a mounting head 31, a parts feeder 30, a mounting head 31, a component recognition camera 32, a nozzle changer 33, a pair of conveyor rails 34, an X-axis beam 35, or a Y-axis beam 36, etc. Furthermore, the product is, for example, a substrate 5, etc. Furthermore, the parts are, for example, chips to be mounted on the substrate 5, etc.

[0068] In the embodiment, the acquisition unit 21 acquires reference position information based on the imaging result obtained by the imaging unit 211 capturing an image of the marker (reference marker M1 or auxiliary marker M2).

[0069] The reference marker M1 is configured by a sticker or the like on which an identification code, such as a QR code (registered trademark), that can be read by the imaging unit 211 is printed. The identification code includes information indicating the coordinates of the reference marker M1 on the 3D map. The reference marker M1 is placed, for example, at a position that is visible to the worker U1 no matter where he is on the floor F1, or at a position that is relatively easy for the worker U1 to see.

[0070] The acquisition unit 21 can read information indicating the coordinates included in the reference marker M1 by capturing an image of the reference marker M1 using the imaging unit 211. The acquisition unit 21 then acquires reference position information indicating the relative position of the production device 3 with respect to the current position of the worker U1 by referring to the detection result of the sensor 12, the 3D map information stored in the memory, and the information read from the reference marker M1. That is, the 3D map information includes the coordinates of each of the one or more production devices 3. Therefore, the acquisition unit 21 can recognize the current coordinates of the worker U1 on the 3D map, the coordinates of the reference marker M1, and the coordinates of each of the one or more production devices 3, and can therefore acquire the reference position information. As such, in this embodiment, as long as the worker U1 can see the reference marker M1, it is possible to acquire the reference position information even if the production device 3 is not within the worker U1's field of view.

[0071] Here, in the embodiment, in addition to the reference marker M1, one or more auxiliary markers M2 are installed on the floor F1. Like the reference marker M1, each of the one or more auxiliary markers M2 is configured by a sticker or the like on which an identification code, such as a QR code (registered trademark), that can be read by the imaging unit 211 is printed. Each of the one or more auxiliary markers M2 includes information indicating its coordinates relative to the reference marker M1 on the 3D map. Therefore, by capturing and reading an image of one or more auxiliary markers M2 in addition to the reference marker M1 using the imaging unit 211, the acquisition unit 21 can correct the position of the read reference marker M1 based on the information included in the auxiliary marker M2, and this is expected to improve the accuracy of acquiring reference position information.

[0072] Furthermore, in the embodiment, the acquiring unit 21 receives relative position information from the production device 3 via the communication unit 212. As will be described later, the acquiring unit 21 acquires error information, which includes the coordinates of the component in which the error occurred. Therefore, by receiving the error information via the communication unit 212, the acquiring unit 21 can acquire the coordinates of the component in which the error occurred, i.e., the relative position information indicating the position of the component relative to the production device 3.

[0073] In the embodiment, the acquisition unit 21 further acquires error information indicating an error related to the production device 3. The error is detected, for example, by the production device 3 performing self-diagnosis using a sensor or the like mounted on the production device 3. The error information includes the type of error, an identifier of the production device 3 (for example, the name of the production device 3), and the coordinates of the component in which the error occurred. Here, the coordinates of the component in which the error occurred are expressed, for example, in an XYZ Cartesian coordinate system with the center position of the production device 3 as the origin.

[0074] When the production device 3 detects an error, it transmits error information to the upper system 4. When the upper system 4 receives the error information, it transmits the received error information to the information processing terminal 1. As a result, the acquisition unit 21 of the information processing terminal 1 acquires the error information via the communication unit 212.

[0075] In the embodiment, the acquisition unit 21 further acquires shape information indicating the shape of the component. The shape information includes, for example, information indicating the contour shape of the component or information indicating the dimensions of the component. The shape information may be acquired from the production device 3 if the production device 3 in which the error occurred has stored the shape information, or from the host system 4 if the host system 4 has stored the shape information. The shape information may also be stored in the memory of the information processing terminal 1, for example.

[0076] In the embodiment, the acquisition unit 21 further acquires work component position information indicating the position of the work component corresponding to the error. The work component is a component required for work to resolve the error, such as grease, an oiler, or a screwdriver. The acquisition unit 21 determines whether to acquire the work component position information depending on the type of error included in the error information. In other words, the acquisition unit 21 may not acquire the work component position information depending on the type of error.

[0077] In the embodiment, the acquisition unit 21 transmits request information to the host system 4, and acquires work component position information in response to the response from the host system 4. The request information includes information indicating the type of work component corresponding to the error. The host system 4 manages the type of work component and the position of the work component in association with each other, and upon receiving the request information, transmits the work component position information to the information processing terminal 1.

[0078] In the embodiment, the acquisition unit 21 further acquires behavioral information indicating the behavior of the worker U1 with respect to the production equipment 3. The behavior of the worker U1 is, for example, the worker U1 approaching the position of the production equipment 3, or opening a cover attached to the production equipment 3, etc.

[0079] For example, when the coordinates of the production equipment 3 on the 3D map and the current coordinates of the worker U1 approximately match, the acquisition unit 21 acquires behavioral information indicating that the worker U1 has approached the production equipment 3. Furthermore, for example, when the acquisition unit 21 detects the worker U1 using a human presence sensor installed on the production equipment 3, the acquisition unit 21 may acquire behavioral information indicating that the worker U1 has approached the production equipment 3 by receiving the detection result from the production equipment 3. Furthermore, for example, when the acquisition unit 21 recognizes, using an appropriate image recognition algorithm, that the production equipment 3 or a marker provided on the production equipment 3 is included in the image captured by the imaging unit 211, the acquisition unit 21 may acquire behavioral information indicating that the worker U1 has approached the production equipment 3. Furthermore, for example, the acquisition unit 21 may acquire behavioral information indicating that the worker U1 has approached the production equipment 3 when the worker U1 performs an operation of selecting a predetermined icon displayed superimposed on the field of view of the worker U1 on the display unit 11.

[0080] Furthermore, for example, when a sensor provided on the production device 3 detects that the cover has been opened, the acquisition unit 21 receives the detection result from the production device 3, thereby acquiring behavioral information indicating that the cover of the production device 3 has been opened. Furthermore, for example, when the acquisition unit 21 recognizes, for example, by an appropriate image recognition algorithm, that the image captured by the imaging unit 211 includes the production device 3 with its cover open, the acquisition unit 21 may acquire behavioral information indicating that the cover of the production device 3 has been opened.

[0081] The display processing unit 22 displays component position information indicating the position of the component superimposed on the field of view of the worker U1 on the display unit 11 of the information processing terminal 1, based on the reference position information and relative position information acquired by the acquisition unit 21. In the embodiment, the component position information is displayed as a hologram on the display unit 11 in the field of view of the worker U1.

[0082] In order for the display processing unit 22 to display the component position information in the field of view of the worker U1 on the display unit 11, it is necessary to recognize the position of the component relative to the current position of the worker U1. Here, the acquisition unit 21 has acquired the reference position information, so the display processing unit 22 can recognize the position of the production equipment 3 relative to the current position of the worker U1. Then, the acquisition unit 21 has further acquired the relative position information, so the display processing unit 22 can recognize the relative position of the component with respect to the position of the production equipment 3. Therefore, the display processing unit 22 can recognize the position of the component relative to the current position of the worker U1 based on both the reference position information and the relative position information.

[0083] Fig. 3 is a diagram showing an example of the display of component position information. Fig. 3 shows the field of view of worker U1 on display unit 11. As shown in Fig. 3, information related to the work that worker U1 is currently performing is displayed superimposed on the field of view of worker U1 on display unit 11. Specifically, a first hologram A1, a second hologram A2, a message B1, a first icon B2, a second icon B3, and a third icon B4 are displayed superimposed on the field of view of worker U1 on display unit 11.

[0084] The first hologram A1 is a hologram represented by an arrow indicating the position of a component (here, the mounting head 31), and corresponds to component position information. The second hologram A2 is a hologram represented by a magnifying glass urging the operator to examine the component. The message B1 is a hologram represented by a string of characters explaining the task that the operator U1 is currently performing. The first icon B2 is a hologram selected by the operator U1 when the operator U1 wishes to display information about the previous task superimposed on the operator U1's field of view on the display unit 11. The second icon B3 is a hologram selected by the operator U1 when the operator U1 wishes to display information about the next task superimposed on the operator U1's field of view on the display unit 11. The third icon B4 is a hologram selected by the operator U1 when the operator U1 wishes to display information other than task information superimposed on the operator U1's field of view on the display unit 11.

[0085] In the example shown in Figure 3, by looking at the first hologram A1 and the second hologram A2, worker U1 can understand what the component to be worked on (here, the mounting head 31) is and what kind of work should be done.

[0086] Furthermore, the worker U1 can select a hologram by, for example, placing his / her finger on a hologram to be selected in the worker U1's field of view on the display unit 11. Specifically, the work support system 2 recognizes the position of the worker U1's finger in the image captured by the imaging unit 211 using an appropriate image recognition algorithm, and determines that the hologram has been selected if the finger's position overlaps the position of the hologram to be selected. If the work support system 2 has a function for recognizing the worker U1's line of sight, the work support system 2 may determine that the hologram has been selected if the time the worker U1's line of sight remains on the hologram exceeds a threshold time. If the work support system 2 has a voice recognition function, the work support system 2 may determine that the hologram has been selected based on the results of recognizing the voice of the worker U1.

[0087] FIG. 4 is a diagram showing an example of the display of contour information. The contour information is information indicating the contour of a component. FIG. 4 shows the field of view of worker U1 on display unit 11. In the example shown in FIG. 4, message B1, first icon B2, second icon B3, and third icon B4 are omitted from the illustration. As shown in FIG. 4, a third hologram A3 and a fourth hologram A4 are displayed superimposed in the field of view of worker U1 on display unit 11. The third hologram A3 is a hologram represented by an arrow urging the removal of the component to be worked on (here, substrate 5). The fourth hologram A4 is a hologram indicating the position of the component to be worked on and representing the contour of the component to be worked on. In other words, the fourth hologram A4 is both component position information and contour information.

[0088] The display processing unit 22 displays contour information superimposed on the field of view of the worker U1 on the display unit 11 based on the shape information acquired by the acquisition unit 21. Specifically, by referencing the component position information and shape information, the display processing unit 22 displays a hologram representing the contour of the component with the center of the component as the origin superimposed on the field of view of the worker U1 on the display unit 11. Here, the contour of the component may be in a form that allows the worker U1 to recognize the type of component. In other words, the contour of the component does not have to be the same as the external shape of the component.

[0089] 4, worker U1 can see the third hologram A3 and the fourth hologram A4 to determine what the component to be worked on (here, substrate 5) is and what work should be done. In particular, in the example shown in Fig. 4, the outline of the component is superimposed in worker U1's field of view on display unit 11, making it easier for worker U1 to determine what the component to be worked on is.

[0090] FIG. 5 is a diagram showing another example of displaying contour information. FIG. 5 shows the field of view of worker U1 on display unit 11. Unlike the example shown in FIG. 4, the component to be worked on is not board 5 but X-axis beam 35. Also, unlike the example shown in FIG. 4, the example shown in FIG. 5 shows fifth hologram A5, sixth hologram A6, and seventh hologram A7 superimposed on the field of view of worker U1 on display unit 11, instead of third hologram A3 and fourth hologram A4. Fifth hologram A5 is a hologram indicating the position of the component to be worked on (here, X-axis beam 35) and also representing the contour of the component to be worked on. In other words, fifth hologram A5 is component position information and contour information. Sixth hologram A6 is a hologram represented by an arrow urging the component to be worked on to move. Seventh hologram A7 is a hologram represented by a hand urging the component to be worked on to be pushed with the hand.

[0091] In the example shown in Fig. 5, worker U1 can determine what the component to be worked on (here, X-axis beam 35) is and what work should be done by looking at fifth hologram A5, sixth hologram A6, and seventh hologram A7. In particular, in the example shown in Fig. 5, similar to the example shown in Fig. 4, the outline of the component is displayed superimposed in the field of view of worker U1 on display unit 11, making it easier for worker U1 to determine what the component to be worked on is.

[0092] FIG. 6 is a diagram showing an example of the type-1 guide notification displayed. The type-1 guide notification is information for guiding the user to the production device 3 where an error has occurred. FIG. 6 shows the field of view of the worker U1 on the display unit 11. As shown in FIG. 6, the type-1 guide notification is displayed superimposed on the field of view of the worker U1 on the display unit 11. Specifically, the type-1 guide notification displays an eighth hologram A8 and a ninth hologram A9 superimposed on the field of view of the worker U1 on the display unit 11.

[0093] The eighth hologram A8 is a hologram that shows an arrow to guide the worker U1 to the production device 3 where the error occurred. The ninth hologram A9 is a hologram that shows an arrow indicating the position of the production device 3 where the error occurred and a character string indicating the distance from the current position of the worker U1 to the production device 3. Note that the distance in the ninth hologram A9 does not have to be displayed.

[0094] When the acquisition unit 21 acquires the error information, the display processing unit 22 displays a type-1 guide notification superimposed on the field of view of the worker U1 on the display unit 11. Specifically, the display processing unit 22 identifies the coordinates of the production equipment 3 where the error occurred on the 3D map by referencing the coordinates of the component where the error occurred, which are included in the error information. Then, the display processing unit 22 displays the type-1 guide notification superimposed on the field of view of the worker U1 on the display unit 11, based on the route from the worker U1's current coordinates on the 3D map to the identified coordinates of the production equipment 3.

[0095] 6, by looking at the eighth hologram A8 and the ninth hologram A9, worker U1 can easily reach the production device 3 where an error is occurring, even if the production device 3 is not visible from worker U1's position. Furthermore, if the ninth hologram A9 includes a character string indicating the distance, worker U1 can easily grasp how far the production device 3 is from his or her current position.

[0096] Furthermore, although not shown, when the acquisition unit 21 acquires work component position information, the display processing unit 22 superimposes a second guide message for guiding the worker U1 to the work component before displaying the first guide message in the worker U1's field of view on the display unit 11. The second guide message differs from the first guide message in that it is information for guiding the worker U1 to the work component instead of the production device 3, but the hologram format is the same as that of the first guide message.

[0097] When the acquisition unit 21 acquires the work component position information, the display processing unit 22 displays a type-2 guide notification superimposed on the field of view of the worker U1 on the display unit 11. Specifically, the display processing unit 22 identifies the coordinates of the work component on the 3D map by referencing the coordinates of the work component included in the work component position information. Then, the display processing unit 22 displays the type-2 guide notification superimposed on the field of view of the worker U1 on the display unit 11 based on the route from the worker U1's current coordinates on the 3D map to the identified coordinates of the work component.

[0098] In this example, worker U1 can more easily reach the location of the work component by viewing the hologram as the type-2 guide notification. Furthermore, if the hologram contains a character string indicating the distance, worker U1 can more easily determine how far the work component is from his or her current position.

[0099] [3. Operation] The operation of the work support system 2 according to the embodiment will be described below with reference to FIG. 7. FIG. 7 is a flowchart showing an example of the operation of the work support system 2 according to the embodiment. The example shown in FIG. 7 illustrates the operation from the point when the acquisition unit 21 of the work support system 2 acquires error information and starts navigation to guide the worker U1 to the component where the error occurred. The example shown in FIG. 7 also illustrates the operation when a work component corresponding to the error is not required. That is, in the example shown in FIG. 7, the type-2 guide message is not superimposed on the display unit 11 of the information processing terminal 1 in the field of view of the worker U1.

[0100] First, the acquisition unit 21 acquires reference position information (step S101). In the embodiment, as already described, the acquisition unit 21 acquires the reference position information based on the imaging result of capturing an image of a marker (reference marker M1 or auxiliary marker M2) by the imaging unit 211. Next, the acquisition unit 21 acquires relative position information (step S102). In the embodiment, as already described, the acquisition unit 21 acquires the relative position information by receiving relative value information from the production device 3 (here, the production device 3 in which the error occurred) via the higher-level system 4 using the communication unit 212. Next, the acquisition unit 21 acquires behavior information (step S103). In the embodiment, the acquisition unit 21 acquires, as the behavior information, information indicating whether the worker U1 has approached the position of the production device 3 and information indicating whether a cover attached to the production device 3 has been opened. Steps S101 to S103 do not have to be performed in this order.

[0101] Next, the display processing unit 22 generates component position information and the first guidance information based on the error information, reference position information, and relative position information acquired by the acquisition unit 21. Then, the display processing unit 22 displays either the component position information or the first guidance information on the display unit 11, superimposed on the field of view of the worker U1, based on the behavior information acquired by the acquisition unit 21.

[0102] Specifically, the display processing unit 22 refers to the behavior information, and if the target production equipment 3 (the production equipment 3 where the error occurred) has been reached (Step S104: Yes) and the cover of the production equipment 3 is open (Step S105: Yes), the display processing unit 22 displays the component position information superimposed on the field of view of the worker U1 on the display unit 11 (Step S106). Note that the display processing unit 22 may also determine that the worker U1 has reached the production equipment 3 if the cover of the production equipment 3 is open. In other words, the display processing unit 22 does not need to make the determination in Step S104. On the other hand, if the behavior information indicates that the target production equipment 3 has not been reached (Step S104: No) or the cover of the production equipment 3 is closed (Step S105: No), the display processing unit 22 displays the type-1 guidance message superimposed on the field of view of the worker U1 on the display unit 11 (Step S107). In other words, when the acquisition unit 21 acquires information indicating that the cover of the production equipment 3 has been opened as behavioral information, the display processing unit 22 switches the information that is superimposed on the field of view of the worker U1 on the display unit 11 from the first guidance information to component position information.

[0103] The above steps S101 to S107 are executed periodically or each time the current position of the worker U1 is updated. Therefore, the information displayed on the display unit 11 so as to be superimposed on the field of view of the worker U1 is updated in real time.

[0104] Next, the overall operation including the work support system 2 according to the embodiment will be described with reference to Fig. 8. Fig. 8 is a sequence diagram showing an example of the overall operation including the work support system 2 according to the embodiment. The example shown in Fig. 8 shows the operation when a work member corresponding to an error is required.

[0105] First, when any of the one or more production devices 3 installed on floor F1 detects an error (step S201), the production device 3 transmits error information to the host system 4 (step S202). Upon receiving the error information, the host system 4 transmits the error information to the information processing terminal 1 (i.e., the work support system 2) (step S203).

[0106] When the acquisition unit 21 of the work support system 2 acquires the error information, the work support system 2 starts navigation to guide the worker U1 to the component where the error occurred (step S204). Specifically, the work support system 2 reads out navigation information corresponding to the type of error included in the error information from the navigation information stored in the memory. The navigation information includes information indicating a series of tasks to be performed to address the error, information indicating the work components required to address the error, and the like.

[0107] The work support system 2 transmits the request information to the host system 4 via the communication unit 212 (step S205). Upon receiving the request information, the host system 4 references the type of work component included in the request information and returns work component position information indicating the position of the work component to the work support system 2 (step S206).

[0108] The display processing unit 22 then displays the second guide notification superimposed on the field of view of the worker U1 on the display unit 11 (step S207). After that, when the acquisition unit 21 acquires information indicating that the worker U1 is carrying a work tool, the display processing unit 22 next displays the first guide notification superimposed on the field of view of the worker U1 on the display unit 11 (step S208). Whether or not the worker U1 is carrying a work tool can be determined, for example, by whether or not an appropriate image recognition algorithm recognizes that the work tool is included in the image captured by the imaging unit 211.

[0109] Then, when the acquisition unit 21 acquires behavior information indicating that the worker U1 has approached the production equipment 3 in which an error has occurred or behavior information indicating that the cover of the production equipment 3 has been opened, the display processing unit 22 then displays the component position information superimposed on the field of view of the worker U1 on the display unit 11 (S209). Thereafter, the display processing unit 22 appropriately switches the information to be displayed superimposed on the field of view of the worker U1 on the display unit 11 according to the progress of the work by the worker U1, etc.

[0110] 4. Advantages The advantages of the work support system 2 according to the embodiment will be explained below, along with a comparison with a work support system of a comparative example. The work support system of the comparative example differs from the work support system 2 according to the embodiment in that a marker is provided for each component related to the production device 3, and by capturing and recognizing the marker with an imaging unit, a hologram representing the component is displayed on the display unit superimposed on the field of view of the worker U1.

[0111] In the work support system of the comparative example, a hologram representing each component is displayed in the field of view of the worker U1 on the display unit only after the imaging unit captures and recognizes the markers attached to each component. Therefore, the worker U1 must search for the markers attached to the component to be worked on in order to bring them into his field of view. Here, if the production device 3 is a device equipped with many units and components, such as a component mounting device, there are many markers, which poses a problem that the worker U1 has to spend time finding the markers attached to the component to be worked on. Furthermore, for example, if the component to be worked on is located in a position that cannot be brought into the field of view of the worker U1 without opening the cover, or if the component to be worked on is located in a position that cannot be brought into the field of view of the worker U1 without peering into the production device 3, the worker U1 has to spend time finding the markers attached to the component to be worked on. Thus, the work support system of the comparative example has the problem that it takes time to find the markers attached to the component to be worked on, which tends to reduce workability.

[0112] In contrast, the work support system 2 according to the embodiment can recognize the position of a component relative to the current position of the worker U1 by referring to the reference position information and the relative position information, which has the advantage of easily displaying information about the component related to the production equipment 3 in the worker U1's field of view on the display unit 11. More specifically, the work support system 2 according to the embodiment has the advantage of easily displaying information about the component related to the production equipment 3 (object), such as the location of the component, and information about the movement route to the production equipment 3 (object), in the worker U1's field of view in an intuitive and easily understandable manner. Therefore, the work support system 2 according to the embodiment can display a hologram representing the component in the worker U1's field of view on the display unit 11 without having to fit a marker in the worker U1's field of view. Therefore, the work support system 2 according to the embodiment eliminates the need for the worker U1 to find a marker attached to the component, which facilitates improved workability.

[0113] (Other embodiments) The task support system 2 of the present disclosure has been described above based on the embodiment, but the present disclosure is not limited to the above embodiment. As long as it does not deviate from the spirit of the present disclosure, various modifications conceivable by a person skilled in the art to the present embodiment and forms constructed by combining components of different embodiments are also included within the scope of the present disclosure.

[0114] In an embodiment, when a component used in the production device 3 needs to be replaced, the work support system 2 may guide the user to a shelf 6 that stores the replacement component, as shown in FIG. 9. In this case, the object is the shelf 6 that stores the component related to the object (the replacement component). The replacement component is, for example, an unused suction nozzle placed in the nozzle changer 33. The task of replacing the suction nozzle may occur, for example, when switching the model of the production device 3 or when the suction nozzle being used is damaged.

[0115] FIG. 9 is a diagram showing an example of the type-3 guide notification being displayed. FIG. 9 shows the field of view of the worker U1 on the display unit 11. As shown in FIG. 9, the type-3 guide notification is displayed superimposed on the field of view of the worker U1 on the display unit 11. Specifically, a first guide hologram C1 is displayed superimposed on the field of view of the worker U1 on the display unit 11 as the type-3 guide notification. The first guide hologram C1 is a hologram that represents an arrow to guide the worker U1 to the shelf 6 that contains the replacement part. The first guide hologram C1 may further include a hologram that represents an arrow indicating the position of the shelf 6 and a character string indicating the distance from the current position of the worker U1 to the shelf 6.

[0116] The display processing unit 22 displays a type-3 guide notification superimposed on the field of view of the worker U1 on the display unit 11, based on the reference position information acquired by the acquisition unit 21. Specifically, the display processing unit 22 identifies the coordinates of the shelf 6 on the 3D map where the replacement part is stored, by referencing the coordinates of the shelf 6 included in the reference position information. The display processing unit 22 then displays the type-3 guide notification superimposed on the field of view of the worker U1 on the display unit 11, based on the route from the worker U1's current coordinates on the 3D map to the identified coordinates of the shelf 6.

[0117] 9, by looking at the guide hologram C1, the worker U1 can easily reach the shelf 6 storing the replacement parts even if the shelf 6 is not visible from the worker U1's position. Furthermore, if the guide hologram C1 includes a character string indicating the distance, the worker U1 can easily grasp how far the shelf 6 is from his / her current position.

[0118] Furthermore, when the worker U1 reaches the target shelf 6, the work support system 2 may guide the worker U1 to the location of the component to be replaced on the shelf 6, as shown in FIG. 10. FIG. 10 is a diagram showing another example of the display of component position information. FIG. 10 shows the field of view of the worker U1 on the display unit 11. As shown in FIG. 10, the component position information is displayed superimposed on the field of view of the worker U1 on the display unit 11. Specifically, a second guide hologram C2 is displayed superimposed on the field of view of the worker U1 on the display unit 11 as the component position information.

[0119] The second guide hologram C2 is a hologram that displays a character string that indicates the position of the replacement component on the shelf 6 where the replacement component is stored. In the example shown in Fig. 10, the second guide hologram C2 includes an "area number" that indicates the area where the replacement component is located, a "shelf number" that indicates the shelf 6 where the replacement component is stored, and a "tier" that indicates the position of the replacement component on the shelf 6.

[0120] If the user U1 has reached the target shelf 6 based on the behavior information acquired by the acquisition unit 21, the display processing unit 22 displays component position information (here, the second guide hologram C2) superimposed on the field of view of the worker U1 on the display unit 11. Here, the display processing unit 22 determines the component position information to be displayed by referring to a work instruction definition table 42 (described later) stored in the higher-level system 4.

[0121] In the example shown in FIG. 10, the worker U1 can easily ascertain the location of the replacement member on the shelf 6 by looking at the second guide hologram C2.

[0122] Furthermore, when the worker U1 arrives at the production equipment 3, the work support system 2 may guide the worker U1 to the replacement location of the replacement component in the production equipment 3, as shown in FIG. 11. FIG. 11 is a diagram showing yet another example of the display of component position information. FIG. 11 shows the field of view of the worker U1 on the display unit 11. As shown in FIG. 11, the component position information is displayed superimposed on the field of view of the worker U1 on the display unit 11. Specifically, a hologram similar to that of FIG. 3 and a detailed hologram C3 are displayed superimposed on each other as the component position information in the field of view of the worker U1 on the display unit 11.

[0123] The detailed hologram C3 is a hologram that displays a character string indicating the replacement location of the replacement part, etc. In the example shown in Fig. 11, the detailed hologram C3 includes a "line number" indicating the number of the line on which the target production equipment 3 is located, an "equipment number" indicating the target production equipment 3, a "table number" indicating the target table, and a "nozzle position" indicating the position of the suction nozzle to be replaced.

[0124] If the user U1 has reached the target production device 3 based on the behavior information acquired by the acquisition unit 21, the display processing unit 22 displays component position information (here, detailed hologram C3) superimposed on the field of view of the worker U1 on the display unit 11. Here, the display processing unit 22 determines the component position information to be displayed by referring to a work instruction definition table 42 (described later) stored in the higher-level system 4.

[0125] In the example shown in FIG. 11, by looking at the detailed hologram C3, the worker U1 can easily understand which part in the production device 3 should be replaced with the replacement part.

[0126] FIG. 12 is a diagram showing an example of the work instruction definition table 42. The work instruction definition table 42 is data that defines the relationship between the specific work steps that the worker U1 is to perform the work instruction and the location information of the components required for each work step. The work instruction definition table 42 has multiple pieces of hierarchical information (here, first to fifth hierarchical information) that hierarchically indicate the positions of the components for each work step, that is, for each task performed by the worker U1. The display processing unit 22 refers to this work instruction definition table 42 to determine the component position information to be displayed superimposed on the field of view of the worker U1 on the display unit 11.

[0127] For example, if the work step is "retrieving replacement material," that is, work to retrieve a replacement component from the target shelf 6, the display processing unit 22 determines component position information by referring to "No. 02" work instruction definition table 42. In this case, before the worker U1 reaches the target shelf 6, the display processing unit 22 refers to the first to third hierarchical information in the work instruction definition table 42, and once the worker U1 reaches the target shelf 6, the display processing unit 22 refers to the first to fifth hierarchical information in the work instruction definition table 42.

[0128] Furthermore, for example, if the work step is "replacement," that is, work to replace a replacement part at the target production equipment 3, the display processing unit 22 determines the part position information by referring to "No. 05" of the work instruction definition table 42. In this case, before the worker U1 arrives at the target production equipment 3, the display processing unit 22 refers to the first to third hierarchical information in the work instruction definition table 42, and once the worker U1 arrives at the target production equipment 3, the display processing unit 22 refers to the first to fifth hierarchical information in the work instruction definition table 42.

[0129] That is, in this example, the component position information is divided into multiple layers, each with a different level of detail indicating the component position. In this example, the display processing unit 22 displays information indicated by one or more layers of the multiple layers in the field of view of the worker U1 on the display unit 11, superimposed according to the work of the worker U1.

[0130] Fig. 13 is a sequence diagram showing an example of the overall operation including the work support system 2 in accordance with the work instruction definition table. Note that explanations of parts in Fig. 13 that are common to Fig. 8 will be omitted. In the example shown in Fig. 13, the work support system 2 does not read out navigation information. Furthermore, the request information sent by the work support system 2 to the higher-level system 4 does not include the type of work material, but does include information instructing the sending of the work instruction definition table 42.

[0131] The work support system 2 transmits the request information to the upper system 4 via the communication unit 212 (step S205). Upon receiving the request information, the upper system 4 transmits the work instruction definition table 42 to the work support system 2 in accordance with the request information (step S205A).

[0132] The work support system 2 generates a type-3 guide notification in accordance with the received work instruction definition table 42 (S206A). The display processing unit 22 then displays the type-3 guide notification superimposed on the worker U1's field of view on the display unit 11 (step S207A). Thereafter, when the acquisition unit 21 acquires information indicating that the worker U1 has a replacement part in its possession, the display processing unit 22 next displays a type-1 guide notification superimposed on the worker U1's field of view on the display unit 11 (step S208). Whether the worker U1 has a replacement part can be determined, for example, by whether an appropriate image recognition algorithm is used to recognize that the work part is included in the image captured by the imaging unit 211.

[0133] Then, when the acquisition unit 21 acquires behavior information indicating that the worker U1 has approached the production equipment 3 in which an error has occurred or behavior information indicating that the cover of the production equipment 3 has been opened, the display processing unit 22 then displays the component position information superimposed on the field of view of the worker U1 on the display unit 11 (S209). Thereafter, the display processing unit 22 appropriately switches the information to be displayed superimposed on the field of view of the worker U1 on the display unit 11 according to the progress of the work by the worker U1, etc.

[0134] In the embodiment, the acquisition unit 21 acquires the reference position information based on the imaging result of capturing an image of a marker by the imaging unit 211, but this is not limited to this. For example, the acquisition unit 21 may acquire the reference position information based on matching the positioning result of a positioning system such as a GPS (Global Positioning System) provided in the information processing terminal 1 with a 3D map. Furthermore, for example, the acquisition unit 21 may acquire the reference position information based on matching the image of surrounding objects captured by the imaging unit 211 with a 3D map.

[0135] In the embodiment, the error information from the production device 3 is transmitted to the information processing terminal 1 via the higher-level system 4, but this is not limiting. For example, if communication is possible between the production device 3 and the information processing terminal 1, the error information may be transmitted directly from the production device 3 to the information processing terminal 1.

[0136] In the embodiment, the information processing terminal 1 is an AR glass, but is not limited to this. For example, the information processing terminal 1 may be a goggle-type head-mounted display or a headset such as an MR (Mixed Reality) headset. Furthermore, the display unit 11 of the information processing terminal 1 is not limited to a transparent display, but may be a non-transparent display. In this case, an image of the real space captured by the imaging unit 211 is displayed on the display unit 11. Furthermore, the information processing terminal 1 is not limited to a head-mounted display, but may be a portable terminal such as a smartphone or a tablet terminal. In this case, the worker U1 may hold the information processing terminal 1 in front of him / her to display the image in front of him / her on the display unit 11.

[0137] In the embodiment, the production device 3 may be remotely operable using the information processing terminal 1. For example, when the worker U1 removes the suction nozzle of the mounting head 31, the worker U1 may perform an operation to move a hologram indicating the suction nozzle that is displayed on the display unit 11. In this case, an instruction to move the mounting head 31 to a position where the suction nozzle can be removed is transmitted from the information processing terminal 1 to the production device 3 via the higher-level system 4, or directly to the production device 3. Upon receiving this instruction, the production device 3 moves the mounting head 31 to a position where the suction nozzle can be removed.

[0138] In the embodiment, the acquisition unit 21 does not need to acquire shape information. In this case, the display processing unit 22 only needs to display the member position information in the field of view of the worker U1 on the display unit 11, and does not need to display the contour information.

[0139] In this embodiment, the acquisition unit 21 does not necessarily have to acquire behavior information. In this case, the display processing unit 22 only needs to display the component position information in the field of view of the worker U1 on the display unit 11, and does not necessarily have to display the type-1 guide notification.

[0140] In this embodiment, the acquisition unit 21 does not necessarily have to acquire work component position information. In this case, the display processing unit 22 only needs to be able to display the component position information in the field of view of the worker U1 on the display unit 11, and does not necessarily have to display a type-2 guide message.

[0141] In the embodiment, the acquisition unit 21 may acquire only the shape information without acquiring the reference position information and the relative position information. Then, the display processing unit 22 may display contour information indicating the contour of the component superimposed on the display unit 11 in the field of view of the worker U1 based on the acquired shape information.

[0142] In the above embodiment, the display processing unit 22 and the like may be configured with dedicated hardware, or may be realized by executing a software program suitable for the display processing unit 22. The display processing unit 22 may be realized by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0143] The following cases are also included in this disclosure:

[0144] (1) Specifically, the display processing unit 22 may be a computer system composed of a microprocessor, ROM, RAM, a hard disk unit, a display unit, a keyboard, a mouse, etc. A computer program is stored in the RAM or hard disk unit. The display processing unit 22 achieves its functions by the microprocessor operating in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands to the computer to achieve a predetermined function.

[0145] (2) The display processing unit 22 may be configured as one system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically, is a computer system configured to include a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating in accordance with the computer program.

[0146] (3) The display processing unit 22 may be configured as a removable IC card or a standalone module. The IC card or module is a computer system configured with a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned ultra-multifunctional LSI. The IC card or module achieves its functions when the microprocessor operates according to a computer program. The IC card or module may be tamper-resistant.

[0147] (4) The present disclosure may be embodied as the methods described above, a computer program for implementing these methods on a computer, or a digital signal comprising the computer program.

[0148] The present disclosure may also be a computer program or a digital signal recorded on a computer-readable recording medium, such as a flexible disk, a hard disk, a CD (Compact Disc)-ROM, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), or a semiconductor memory, or a digital signal recorded on such a recording medium.

[0149] Furthermore, the present disclosure may involve transmitting a computer program or a digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, or data broadcasting, etc.

[0150] Furthermore, the program or digital signal may be recorded on a recording medium and transferred, or the program or digital signal may be transferred via a network or the like, so that the program or digital signal may be implemented by another independent computer system. [Industrial Applicability]

[0151] The present disclosure is applicable to, for example, a system for assisting a worker in performing work using production equipment. [Explanation of symbols]

[0152] 1. Information processing terminal 2. Work support system 3 Production equipment (objects) 4. Upper system 5. Substrate 6 Shelf (Object) 11 Display section 12 sensors 21 Acquisition Department 22 Display processing section 30 Parts feeder 31 Mounting head 32 Parts Recognition Camera 33 Nozzle changer 34 Transport rail 35 X-axis beam 36 Y-axis beam 211 Imaging unit 212 Communications Department A1 First hologram A2 Second hologram A3 3rd hologram A4 4th hologram A5 5th hologram A6 6th hologram A7 7th hologram A8 8th hologram A9 9th Hologram B1 Message B2 1st Icon B3 2nd icon B4 3rd icon C1 First guide hologram C2 Second guidance hologram C3 Detailed Hologram F1 Floor M1 fiducial marker M2 auxiliary marker U1 Worker

Claims

1. an acquisition unit that acquires reference position information indicating a position of an object placed on a floor of a manufacturing plant, and relative position information indicating a relative position of a member related to the object with respect to the position of the object; a display processing unit that displays member position information indicating a position of the member in a field of view of the worker on a display unit of an information processing terminal carried by the worker, based on the reference position information and the relative position information acquired by the acquisition unit. Work support system.

2. The acquisition unit further acquires shape information indicating a shape of the component, The display processing unit displays contour information indicating a contour of the member in a field of view of the worker on the display unit, superimposed on the field of view of the worker on the basis of the shape information acquired by the acquisition unit. The work support system according to claim 1 .

3. the acquisition unit includes an imaging unit that images a marker placed on the floor, The acquisition unit acquires the reference position information based on an imaging result of imaging the marker by the imaging unit. The work support system according to claim 1 or 2.

4. The object is a production device, The acquisition unit further acquires error information indicating an error related to the production device, When the acquisition unit acquires the error information, the display processing unit displays a type-1 guide message on the display unit in a field of view of the worker, the type-1 guide message directing the worker to the production device in which the error has occurred, superimposed thereon. The work support system according to claim 1 or 2.

5. The acquisition unit further acquires behavior information indicating an action of the worker with respect to the production device, the display processing unit displays, on the basis of the behavior information acquired by the acquisition unit, one of the member position information and the first guide information in a superimposed manner in a field of view of the worker on the display unit. The work support system according to claim 4.

6. when the acquisition unit acquires information indicating that a cover of the production equipment has been opened as the action information, the display processing unit switches information to be superimposed on the field of view of the worker on the display unit from the first guide information to the member position information. The work support system according to claim 5 .

7. The acquisition unit further acquires working member position information indicating a position of a working member corresponding to the error, the display processing unit displays a second guide message for guiding the user to the work member in a superimposed manner before displaying the first guide message in the field of view of the worker on the display unit. The work support system according to claim 4.

8. the acquisition unit includes a communication unit that receives the relative position information from the object; The work support system according to claim 1 or 2.

9. The component location information is divided into a plurality of hierarchical levels each indicating the location of the component with a different degree of detail, and is managed in association with the contents of the work instruction. The work support system according to claim 1 or 2.

10. the display processing unit displays information indicated by one or more hierarchical levels among the plurality of hierarchical levels in a field of view of the worker on the display unit in accordance with a content of the work instruction. The work support system according to claim 9.

11. an acquisition unit that acquires shape information indicating a shape of a component related to an object arranged on a floor of a manufacturing plant; a display processing unit that displays contour information indicating a contour of the component in a field of view of the worker on a display unit of an information processing terminal carried by the worker based on the shape information acquired by the acquisition unit, Work support system.

12. The object includes at least one of a production device and a shelf on which the part is stored. The work support system according to claim 1 , 2 , or 11 .

13. The work support system according to claim 1 or 2, The display unit; A sensor for detecting the position and orientation of the worker. Information processing terminal.

14. The display unit is a transmissive display.

14. An information processing terminal according to claim 13.

15. obtaining reference position information indicating a position of an object disposed on a floor of a manufacturing plant and relative position information indicating a position of a member related to the object relative to the position of the object; and displaying member position information indicating the position of the member in a superimposed manner in the field of view of the worker on a display unit of an information processing terminal carried by the worker based on the acquired reference position information and the acquired relative position information. Work support method.

16. obtaining shape information indicative of shapes of components for objects located on a manufacturing plant floor; based on the acquired shape information, displaying contour information indicating a contour of the member in a field of view of the worker on a display unit of an information processing terminal carried by the worker; Work support method.

Images