Information notification system

Abstract

An information notification system S for notifying operators 5 working with production equipment in a factory comprises: a detection apparatus 100 that detects operator information of the operators 5 working in the factory; audio devices 121–124, 135, 140 that are capable of outputting audio to a specific operator 5 working in the factory; and a server computer 150. The server computer 150 selects an operator as a conversation partner from among operators to whom information can be relayed in the factory, on the basis of operator information detected by the detection apparatus 100, and issues a notification regarding information on the work by audio using the audio devices 121–124, 135, and 140.

Description

Information Notification System

[0001] The technology disclosed in this specification relates to a system for notifying an operator of information.

[0002] On a factory production line, if a machine generates an error or warning, the machine outputs an alarm and displays the information on a large monitor.

[0003] When an operator becomes aware of an error or warning, he or she goes to the machine to check the situation and then starts work. Patent Document 1 listed below discloses a method for displaying information about errors and warnings on a mobile terminal.

[0004] WO2005 / 009101 publication

[0005] The technology disclosed in this specification aims to convey information to operators in a factory having production equipment such as a mounting line by using voice without interrupting work.

[0006] (1) An information notification system for notifying information to operators working on production equipment in a factory, comprising: a detection device for detecting operator information of operators working in the factory; an audio device capable of outputting audio to a specific operator working in the factory; and a server computer. The server computer selects an operator to communicate with from among operators in the factory with whom information can be communicated, based on the operator information detected by the detection device, and notifies information about the work by audio using the audio device.

[0007] This configuration allows information to be conveyed to operators without interrupting their work. Furthermore, the audio device can output audio to specific operators working in the factory, while the audio is barely heard by other operators. This prevents other operators from misunderstanding the audio and interrupting their work.

[0008] (2) In the information notification system described in (1), the operator information may include two or more of the operator's current location, working posture, work content, and gestures. The server computer may select an operator to interact with based on two or more pieces of operator information. With this configuration, by taking two or more pieces of operator information into consideration, it is possible to accurately determine the most suitable person for the work.

[0009] (3) In the information notification system described in (1) or (2), the server computer may send a confirmation voice to the selected operator and change the processing depending on the operator's response. This configuration allows for flexible responses and dialogue with the operator.

[0010] (4) In the information notification system according to any one of (1) to (3), the server computer may switch to another audio device if it determines that the audio is not received or is unclear. With this configuration, if the audio is not received or is unclear, switching to another device can resolve poor information transmission.

[0011] (5) In the information notification system according to any one of (1) to (4), the server computer may automatically adjust the volume when it determines that the volume is inappropriate. This configuration can eliminate poor voice transmission due to insufficient or excessive volume.

[0012] (6) In the information notification system according to any one of (1) to (5), the operator information may include movement information of the operator. The server computer may switch the audio device in accordance with the movement of the operator. With this configuration, audio can be transmitted to a moving operator.

[0013] (7) In the information notification system described in any one of (1) to (6), the audio device may be any of a directional speaker installed in a structure or production facility, a built-in speaker of a mobile terminal, or a wireless earphone.

[0014] (8) In the information notification system described in (7), the audio device may be the directional speaker. The server computer may automatically adjust the volume or sound collection position of the directional speaker depending on the distance from the audio device to the operator. This configuration allows audio to be transmitted to the operator regardless of the distance to the operator.

[0015] (9) In the information notification system according to any one of (1) to (8), the audio device may be selectable from a directional speaker installed in a structure or production facility, a built-in speaker of a mobile terminal, or a wireless earphone. The server computer may select the built-in speaker of the mobile terminal or the wireless earphone in preference to the directional speaker.

[0016] According to the technology disclosed in this specification, it is possible to convey information to operators in a factory having production equipment such as a mounting line without interrupting work, and to avoid distracting the operators.

[0017] Diagram of the line configuration of a mounting line Diagram showing an overview of the information notification system Diagram showing the system configuration of the information notification system Diagram showing a specific example of an audio device Diagram showing a specific example of an audio device Flowchart of information notification processing Diagram summarizing specific examples of positive, negative, and confirmatory responses Plan view of a mounting line Plan view of a mounting line Diagram showing audio device switching Diagram showing audio device switching Diagram showing adjustment of the direction of a directional speaker Diagram showing adjustment of the volume of a directional speaker Diagram showing the system configuration of an information notification system

[0018] First Embodiment 1. Description of Mounting Line Fig. 1 is a line configuration diagram of a mounting line 10. The mounting line 10 is a line that mounts components on a board. The mounting line 10 is an example of production equipment.

[0019] The mounting line 10 is installed in a factory 1 and includes a printing machine 11, an SPI 12, mounters 13A to 13C, an inspection machine 14, a reflow device 15, and an inspection machine 16. These machines 11 to 16 are connected in series via a conveying device.

[0020] The mounting line 10 sends the boards to each machine 11 to 16 in order, while performing the prescribed operations (printing ⇒ post-printing inspection ⇒ component mounting ⇒ post-mounting inspection ⇒ reflow ⇒ post-reflow inspection).

[0021] When an error or warning occurs in any of the machines 11 to 16 on the mounting line 10, it is desirable to transmit this information to the operator without interrupting the work.

[0022] This specification discloses an information notification system that uses voice to communicate information to operators in a factory having production equipment such as a mounting line without interrupting work.

[0023] 2. Description of the Information Notification System Fig. 2 is a diagram showing an overview of the information notification system S, Fig. 3 is a block diagram of the information notification system S, and Figs. 4 and 5 are specific examples of audio devices.

[0024] The information notification system S is a system that notifies information to an operator 5 in a factory 1 that is equipped with production equipment such as a mounting line 10. The information notification system S includes a detection device 100, audio devices 121 to 124, 135, 140, and a server computer 150.

[0025] The detection device 100 is a device that detects information about an operator 5 working in a factory 1. The detection device 100 may be, for example, a camera 110 and an image analysis unit 115. The camera 110 is installed on a structure such as a ceiling 2 or a wall 3 of the factory 1, and captures images of the inside of the factory, including the operator 5. There may be one camera 110 or multiple cameras 110.

[0026] The image analysis unit 115 analyzes the image from the camera 110 and acquires operator information. The operator information is information about the operators 5 working in the factory, and includes the identification of the operators 5, the current location of each operator 5, the working status, the working posture, etc. The operator information may also include gestures of the operators 5. The operator information may also include movement information of the operators 5 and the reactions of the operators 5 to voices. The operators 5 may be identified, for example, by reading an ID number from an image.

[0027] The audio devices 121 to 124, 135, and 140 are devices that transmit information by voice to a specific area within the factory or to a specific operator 5 working within the factory. For example, directional speakers can be used when installed on structures such as the ceiling 2 or wall 3. Directional speakers have a small directivity angle, and can transmit audio to a specific area within the factory or to a specific operator 5 working in the factory 1.

[0028] 4, four directional speakers 121, 122, 123, and 124 are installed on the ceiling 2 of a factory 1. Of the four directional speakers, 121 and 123 are located above the first mounting line 10A, and 122 and 124 are located above the second mounting line 10B.

[0029] Furthermore, the audio devices 121 to 124, 135, and 140 may be a built-in speaker 135 of a portable information terminal 130 carried by the operator 5, or a wireless earphone 140, as shown in FIG.

[0030] The audio devices 121 to 124, 135, and 140 may be of one type, or may be selectable from a plurality of different types of audio devices 121 to 124, 135, and 140.

[0031] Additionally, a priority may be assigned to the selected audio devices 121 to 124, 135, and 140. For example, a portable device (such as the built-in speaker 135 or the wireless earphone 140) may be prioritized over a stationary device (such as the directional speakers 121 to 124).

[0032] 3, the server computer 150 is connected to each of the machines 11 to 16 and the detection device 100 via a LAN (local area network) or the like. In addition, the audio devices 121 to 124, 135, and 140 are also connected via the LAN or wirelessly, and are capable of communication.

[0033] The server computer 150 includes a database 160 that stores floor information of the factory 1, a control device 170, and a communication unit 180. The floor information includes layout information of the mounting line 10 installed in the factory (position information of each machine 11 to 16).

[0034] In response to an event such as an error or warning that occurs on the mounting line 10, the control device 170 acquires operator information of the operator 5 working in the factory 1 from the detection device 100, and performs the following operations.

[0035] (1) Selection of operators who can perform the work (2) Selection of audio devices (3) Audio instructions to operators

[0036] 3. Explanation of Information Notification Flow The flow of work instructions to the operator 5 will be explained with reference to Figure 6. When an error or warning occurs on the mounting line 10, the machines 11 to 16 send that information to the server computer 150. Here, we will explain the case where a machine trouble occurs on the first mounting line 10A.

[0037] When the server computer 150 receives error or warning information, it acquires operator information from the detection device 100. The operator information includes the current position, working posture, work content, gestures, etc. of the operator 5 working in the factory.

[0038] The server computer 150 then selects an operator 5 in the factory to whom information can be transmitted. "Information transmission is possible" means that information can be transmitted by the audio devices 121 to 124, 135, and 140.

[0039] The server computer 150 may manage information necessary for determining whether or not to transmit information. For example, the server computer 150 may be configured to link the information with the operator's ID and manage whether or not the operator possesses the audio devices 135 and 140, and if so, the type of the audio devices.

[0040] If the operator is carrying audio devices 135, 140, it can be determined that information can be transmitted, and if the operator is not carrying audio devices 135, 140, it can be determined whether information can be transmitted based on the relative positions of the operator's current location and the directional speakers 121 to 124.

[0041] Then, the server computer 150 selects an operator 5 to be the conversation partner from among the operators with whom the information can be transmitted (S10). The conversation partner operator is selected to carry out work on the error or warning.

[0042] The selection of the operator to talk to may be determined based on one piece of information such as the current location, or may be determined based on two or more pieces of information from the current location, working posture, work content, and gestures. Note that gestures are, for example, expressions of intention using body movements or postures.

[0043] In this embodiment, an operator to talk to is selected based on the current location and the work content from among the operators who can communicate with the voice devices 121 to 124, 135, and 140. Specifically, operator 5, who is located near the machine 11 to 16 where the error or warning occurred and who is performing work with low urgency, is selected as the conversation partner.

[0044] The following describes a case where the operator 5A located near the first mounting line 10A is selected.

[0045] After selecting the operator 5A, the server computer 150 selects the types of audio devices 121 to 124, 135, and 140 (S20). Specifically, the audio device is selected from among the ceiling-mounted directional speakers 121 to 124, the built-in speaker 135 of the personal digital assistant 130, and the wireless earphone 140.

[0046] The audio devices 121 to 124, 135, and 140 may be selected according to priority, or may be automatically selected based on operator information to suit the operator's situation. In this example, the wireless earphone 140, which has been set to a high priority, is selected.

[0047] If the wireless earphone 140 is selected in S20, S30 is skipped and the server computer 150 uses the selected audio device (wireless earphone 140) to notify the operator 5A of a confirmation voice (S40). The confirmation voice is a phrase such as "Are you available to work?" or "Can you hear me?"

[0048] When the server computer 150 outputs the confirmation voice, the detection device 100 checks the response of the operator 5 to the confirmation voice and determines which of patterns (1) to (4) it corresponds to. Figure 7 shows the judgment criteria (specific example).

[0049] (1) Positive response (2) Negative response (3) Confirmatory response (4) No response

[0050] If the response is positive, the server computer 150 uses the audio device (wireless earphone 140) to notify the operator 5A by voice of the error details of the machines 11 to 16 and work instructions (S60).

[0051] Thereafter, the server computer 150 checks the reaction of the operator 5A to the work instruction using the detection device 100, and determines which of patterns (1) to (4) the reaction corresponds to, as in S50 (S70).

[0052] If the operator responds positively to the work instruction, the operator 5A moves to the machine 11 to 16 where the error or warning occurred and performs the work in accordance with the instructions given by voice in S60.

[0053] If the response to the work instruction is negative, the process proceeds to S80, where the server computer 150 selects another operator 5. After S80, the process returns to S20. The same applies if the response to the confirmation voice in S50 is negative.

[0054] If the response to the work instruction is a confirmation response, the process proceeds to S90, where the server computer 150 changes the audio device. The same applies to the case where the response to the confirmation voice in S50 is a confirmation response. Here, the audio devices 121 to 124, 135, and 140 are changed from the wireless earphone 140 to the ceiling-mounted directional speakers 121 to 124.

[0055] The confirmatory response is a response that checks the state of the audio (in other words, a response that notifies the user that the audio is not working properly), and occurs when the audio is not received or is unclear. S80 realizes the following of the present invention: "When the server computer determines that the audio is not received or is unclear, it switches to another audio device."

[0056] Thereafter, the process proceeds to S30, where the server computer 150 selects a speaker (individual) from the directional speakers 121 to 124 that is suited to the situation, for example, the speaker that can most easily transmit sound to the operator 5.

[0057] In this example, since the operator 5A is selected in S10, the directional speaker 121 closest to the operator 5A is selected as shown in FIG.

[0058] Next, the process proceeds to S40, where the server computer 150 uses the changed audio device (directional speaker 121) to notify the operator 5A of the confirmation audio again.

[0059] Thereafter, the process proceeds to S50, where the process is changed depending on the reaction of the operator 5A. Specifically, if the reaction is positive, S60 is executed, if the reaction is negative, S80 is executed, if the reaction is confirmatory, S90 is executed, and if there is no reaction, S100 is executed.

[0060] When the process proceeds to S100, the confirmation voice is re-issued until the upper limit is reached. When the upper limit is reached, the process proceeds to S80, and an operator 5 is reselected.

[0061] 3. Effects This technology makes it possible to convey information to the operator 5 without interrupting work. It also avoids distracting the operator 5. Furthermore, because the voice output is specific to the operator 5, the voice hardly reaches other operators 5. This makes it possible to prevent other operators 5 from misunderstanding the voice and interrupting their work.

[0062] Second Embodiment In the first embodiment, an example is disclosed in which the directional speakers 121 to 124 installed on the ceiling, the built-in speaker 135 of the portable information terminal 130 carried by the operator 5, and the wireless earphone 140 are used as audio devices.

[0063] The second embodiment discloses an example in which existing speakers 210 installed in the machines 11 to 16 are used as audio devices. "Existing" means that the speakers have been installed in the machines 11 to 16 since their manufacture or construction.

[0064] 8 is a plan view of a mounting line 10 installed in a factory. In this example, two rows of mounting lines 10A and 10B are installed in the Y direction in a factory 1, and three aisles are provided around them.

[0065] Each of the machines 11 to 16 on the mounting lines 10A and 10B has an existing speaker 210. The existing speaker 210 is a directional speaker.

[0066] As shown in FIG. 8, if an error or warning occurs on the first mounting lines 10A and 10B in the factory, the server computer 150 selects an operator 5 who is available to perform the work, selects an existing speaker 210 on a machine 11 to 16 that is closest to the selected operator's location, and notifies the operator of the details of the error or warning and the associated work instructions.

[0067] In this example, an alarm occurs on machine 13B on the first mounting line 10A, and operator 5A, who is available to work, is located near the right end of the first aisle, so the existing speaker 210 on machine 11 or 12 on the first mounting line 10A, which is close to operator 5A, is used as the audio device.

[0068] By using the existing speakers 210 of the machines 11 to 16 as audio devices, there is no need for renovation work to install speakers. Therefore, when adopting the information notification system S, the initial investment can be reduced.

[0069] Third Embodiment In a third embodiment, the volume of the audio device 210 is automatically adjusted in accordance with the distance D from the audio device 210 to the operator 5 .

[0070] FIG. 9 is a plan view of the mounting line 10, and shows an example in which work instructions are transmitted to the operator 5 using the existing speakers 210 of the machines 11 to 16, as in the second embodiment.

[0071] The server computer 150 uses the detection device 100 to detect the distance D from the existing speaker 210 to the operator 5. For example, the detection is performed using a stereo camera method using multiple cameras 110. Then, the volume of the existing speaker 210 is automatically adjusted according to the distance D to the operator 5. Specifically, the longer the distance D, the higher the volume is made.

[0072] By automatically adjusting the volume of the audio device (existing speaker 210) according to the distance D to the operator 5, audio can be transmitted to the operator 5 regardless of the distance D to the operator 5.

[0073] This technique is also effective for audio devices in which audio is easily attenuated with distance, and if the volume is constant, the audio suddenly becomes less likely to reach the operator 5 depending on the distance D.

[0074] Furthermore, if the audio device has a function for adjusting the sound collection position, the sound collection position may be automatically adjusted according to the distance D to the operator 5. The sound collection position may be adjusted together with the volume adjustment, or may be adjusted independently.

[0075] The sound collection position (listening position) is an ideal position for the operator 5 to hear the sound emitted by the audio device.

[0076] Fourth Embodiment In the first embodiment, it has been disclosed that the directional speakers 121 to 124 installed on the ceiling are used as audio devices.

[0077] When the operator 5 moves, there is a possibility that the operator 5 may move out of the effective range (range over which the sound can be transmitted) of the directional speakers 121 to 124 and the sound may not reach the operator 5. The same applies if the operator 5 is in a blind spot.

[0078] The server computer 150 acquires movement information of the operator 5 from the detection device 100. The movement information includes, for example, information on the current location of the operator, information on the direction of movement, information on the movement speed, and the like.

[0079] The server computer 150 then dynamically switches the directional speakers 121 to 124 in accordance with the movement of the operator 5 so that the operator 5 is included in the effective range (range in which sound can be transmitted) U1 of the directional speakers 121 to 124.

[0080] For example, as shown in FIG. 10, when the operator 5 moves in the X direction, the directional speakers used as the audio device are switched from the combination of 323 and 324 to the combination of 324 and 325.

[0081] By switching between the directional speakers 321 to 326, it is possible to prevent the moving operator 5 from going outside the effective range U1 of the directional speakers 321 to 326, and to transmit voice to the moving operator 5.

[0082] Furthermore, when switching between the directional speakers 321 to 326, the server computer 150 may also transfer responses (interactions and conversations) with the operator 5 to the new directional speaker 321 to 326, thereby avoiding repetition of the same responses.

[0083] Switching of audio devices can also be applied to cases where the communication range is limited, such as when using wireless earphones 140. In Fig. 11, the operator moves out of the effective range (wireless communication range) U2 of the wireless earphones 140, and so the device is switched from the wireless earphones 140 to the directional speaker 326 installed on the ceiling.

[0084] Fifth Embodiment The directional speakers 321 to 326 may have areas where it is difficult to hear the sound due to reflections. If the operator 5 makes a gesture (such as shaking their hand) to indicate that they are having difficulty hearing the sound from the directional speakers 321 to 326, the server computer 150 changes the conditions and attempts to notify them by sound again.

[0085] For example, if the angle of each of the speakers 321 to 326 installed on the ceiling 2 can be changed, the angle may be changed and another audio notification may be attempted using another of the directional speakers 321 to 326, as shown in FIG.

[0086] Also, if the operator 5 makes a gesture requesting that the volume be adjusted (for example, by pointing a finger at a directional speaker), the volume may be automatically adjusted and an audio notification may be attempted again, as shown in FIG. 13 .

[0087] Sixth Embodiment In the first embodiment, the detection device 100 is the camera 110, but it can also be replaced by a communication device 190 carried by each operator 5 working in the factory. The communication device 190 can be in any form as long as it can wirelessly communicate with the server computer 150 and can at least identify each operator 5 and confirm their current location.

[0088] For example, as shown in FIG. 14, a wireless ID tag 191 and a beacon device 192 may be combined.

[0089] The wireless ID tag 191 is a device that wirelessly transmits the ID information of the operator 5 to the server computer 150. The beacon device 192 is a location identification device that uses short-range wireless communication. By installing these receivers in the server computer 150, the ID information and current location information of the operator 5 can be obtained.

[0090] Furthermore, an operator microphone 193 for voice confirmation and response may be added to the communication device 190. Also, instead of the operator microphone 193, an acceleration sensor device or a device with a switch may be added.

[0091] Although the embodiments have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above.

[0092] (1) In the first to fifth embodiments, the mounting line 10 is shown as an example of production equipment installed in the factory 1. The production equipment is not limited to the mounting line 10 and may be another manufacturing line. Furthermore, the production equipment may be a supply facility that supplies parts and the like to the mounting line 10 or the manufacturing line.

[0093] (2) In the first to fifth embodiments, the operator 5 to perform the work is selected based on multiple pieces of information, such as the current location and the work content. The operator to perform the work may be selected based on only one piece of information, such as the current location or the work content. Furthermore, the number of selected operators is not limited to one, and multiple operators may be selected.

[0094] 10 Mounting line (production equipment) 11 to 16 Machine 100 Detection device 110 Camera 115 Image analysis unit 121 to 124 Directional speaker (audio device) 135 Built-in speaker (audio device) 140 Wireless earphone (audio device) 150 Server computer 160 Database 170 Control device 321 to 326 Directional speaker (audio device)

Claims

1. An information notification system that notifies information to operators working on production equipment in a factory, comprising: a detection device that detects operator information of operators working in the factory; an audio device that can output audio to a specific operator working in the factory; and a server computer, wherein the server computer selects an operator to talk to from among operators in the factory with whom information can be communicated, based on the operator information detected by the detection device, and notifies information about the work by audio using the audio device.

2. An information notification system according to claim 1, wherein the operator information includes two or more of the operator's current location, working posture, work content, and gestures, and the server computer selects an operator to converse with based on two or more pieces of operator information.

3. An information notification system according to claim 1, wherein the server computer sends a confirmation voice to the selected operator and changes the processing depending on the operator's response.

4. An information notification system according to claim 1, wherein the server computer switches to another audio device when it determines that the audio cannot be received or is unclear.

5. An information notification system according to claim 1, wherein the server computer automatically adjusts the volume when the operator makes a gesture requesting volume adjustment.

6. An information notification system according to claim 1, wherein the operator information includes movement information of the operator, and the server computer switches the audio device in accordance with the movement of the operator.

7. An information notification system according to claim 1, wherein the audio device is one of a directional speaker installed in a structure or production facility, a built-in speaker of a mobile terminal, and a wireless earphone.

8. An information notification system according to claim 7, wherein the audio device is the directional speaker, and the server computer automatically adjusts the volume or sound collection position of the directional speaker depending on the distance from the directional speaker to the operator.

9. An information notification system as described in claim 1, wherein the audio device can be selected from a directional speaker installed in a structure or production facility, a built-in speaker of a mobile terminal, or a wireless earphone, and the server computer selects the built-in speaker of the mobile terminal or the wireless earphone in preference to the directional speaker.

Images