Determination device, system, and determination method

Abstract

To determine an operation result to a switch, on the basis of a vibration signal at the time of operation of the switch.SOLUTION: A determination device includes: an acquisition part for acquiring a vibration signal generated at the time of position switching of an object; and a determination part for determining a state of the object using the vibration signal.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a determination device, a system, and a determination method.

Background Art

[0002] Conventionally, in machines, equipment, etc., various switches for performing various controls have been used. The operation of the switch is performed manually or using a robot or the like. Whether the operation on the switch is appropriately performed can be considered by taking an image around the switch using a camera and determining based on the image.

[0003] For example, Patent Document 1 discloses a configuration for determining the positional relationship with a component to be gripped during the operation of a robot using a captured image by a camera.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, depending on the shooting conditions, such as in a relatively dark place, it may not be possible to take an image around the switch, or there may be a state where it cannot be discriminated from the image. Therefore, a method for grasping the state of the switch by a method other than the captured image is required.

[0006] The present disclosure has been devised in view of the above-described conventional circumstances, and an object thereof is to provide a determination device and a determination method for determining the state of an object based on a vibration signal when the position of the object such as a switch is switched.

Means for Solving the Problems

[0007] This disclosure comprises an acquisition unit that acquires vibration signals generated when an object changes position, and a determination unit that determines the state of the object using the vibration signals. The object is a switch, the state includes the position of the switch, the determination unit determines the state of the object by comparing the input signal acquired by the acquisition unit with the output signal output from the trained model, and the trained model learns the vibration signal of a predetermined switch and outputs a signal indicating the vibration that occurs when the position of the predetermined switch is switched. A determination device is provided.

[0008] This disclosure provides a system comprising a determination device, a robot for manipulating an object, a sensor for detecting vibration signals, and a control unit for controlling the movement of the robot, wherein the determination device includes an acquisition unit for acquiring vibration signals generated when the object changes position, and a determination unit for determining the state of the object using the vibration signals. The object is a switch, the state includes the position of the switch, the determination unit determines the state of the object by comparing the input signal acquired by the acquisition unit with the output signal output from the trained model, the trained model learns the vibration signal of a predetermined switch and outputs a signal indicating the vibration that occurs when the position of the predetermined switch is switched. We provide the system.

[0009] This disclosure includes an acquisition step for acquiring a vibration signal generated when an object changes position, and a determination step for determining the state of the object using the vibration signal. death , The object is a switch, the state includes the position of the switch, the determination step is a step of determining the state of the object by comparing the input signal acquired by the acquisition unit with the output signal output from the trained model, the trained model learns the vibration signal of a predetermined switch and outputs a signal indicating the vibration that occurs when the position of the predetermined switch is switched. A method for making a determination is provided.

[0010] Furthermore, any combination of the above components, as well as any conversion of the expressions of this disclosure between methods, apparatus, systems, storage media, computer programs, etc., are also valid forms of this disclosure. [Effects of the Invention]

[0011] According to this disclosure, a determination device and determination method can be provided for determining the state of an object based on vibration signals when the object changes position. [Brief explanation of the drawing]

[0012] [Figure 1] Block diagram showing an example of the system configuration according to Embodiment 1 [Figure 2] External perspective view showing an example of the configuration of the hand member of the robot according to Embodiment 1. [Figure 3] External perspective view illustrating an example of operation of a toggle switch according to Embodiment 1 [Figure 4]External perspective view illustrating an example of operation of a push switch according to Embodiment 1 [Figure 5] External perspective view illustrating an example of rocker switch operation according to Embodiment 1 [Figure 6] A graph showing an example of a vibration waveform generated when operating the switch according to Embodiment 1. [Figure 7] Schematic diagram illustrating the learning process and judgment process according to Embodiment 1 [Figure 8] Flowchart of the process according to Embodiment 1 [Figure 9] Block diagram showing an example of the system configuration according to Embodiment 2 [Figure 10] Sequence diagram of the process according to Embodiment 2 [Figure 11] Flowchart of the process according to Embodiment 3 [Modes for carrying out the invention]

[0013] The following description will detail embodiments specifically disclosing the determination device and determination method according to this disclosure, with appropriate reference to the attached drawings. However, unnecessarily detailed explanations may be omitted. For example, detailed explanations of already well-known matters or redundant explanations of substantially identical configurations may be omitted. This is to avoid the following description becoming unnecessarily verbose and to facilitate understanding by those skilled in the art. The attached drawings and the following description are provided to enable those skilled in the art to fully understand this disclosure and are not intended to limit the subject matter described in the claims.

[0014] In each of the embodiments described below, various types of switches will be taken as examples of the objects to be determined by the determination device. Therefore, in each determination described later, the determination is made based on the data obtained by operating the switch. However, the features of the present invention are not limited to being applied to switches. For example, when there is a correlation between the vibration information generated during operation or state change and its content (operation or state change), and determination can be made based on this in the same manner as in the embodiments described later, the present invention can be applied to objects other than switches.

[0015] <Embodiment 1> [System Configuration] FIG. 1 is a block diagram showing an example of the overall configuration of a system including a switch operation device 100 according to the present embodiment. The switch operation device 100 is a device configured to include the functions of the determination device according to the present invention. The switch operation device 100 is connected to a robot 300 and a vibration sensor 200 for operating a switch group 500. Further, the switch operation device 100 is configured to be communicable with an external server 400 via a network 600.

[0016] The switch operation device 100 operates the switch group 500 by controlling the operation of the robot 300 according to user operations, preset operation information, etc. The switch operation device 100 includes an IF (Interface) unit 101, a UI (User Interface) unit 102, a control unit 103, a memory 104, a communication unit 105, and an HDD (Hard Disk Drive) 106. Each part within the switch operation device 100 is communicably connected by an internal bus or the like. The IF unit 101 is an interface for communicably connecting to the vibration sensor 200 and the robot 300. Further, the IF unit 101 transmits a control signal for controlling the robot 300 to the robot 300 and receives a signal from the robot 300 side.

[0017] The UI unit 102 accepts user input and displays the results of operations on the switch group 500. The UI unit 102 may include, for example, a mouse or keyboard, or it may be composed of a touch panel display that integrates a display unit and an operation unit. The control unit 103 realizes the various functions according to this embodiment by reading various programs and data stored in the memory 104 and HDD 106 and executing processing. The control unit 103 may be composed of at least one of the following: CPU (Central Processing Unit), MPU (Micro Processing Unit), DSP (Digital Signal Processor), or FPGA (Field Programmable Gate Array). The memory 104 is a storage area for storing and holding data of various information, and may be composed of, for example, a non-volatile storage area such as ROM (Read Only Memory) or a volatile storage area such as RAM (Random Access Memory).

[0018] The communication unit 105 is a network interface for communicating with external devices (in this example, an external server 400, etc.) via the network 600. The communication method used by the communication unit 105 is not particularly limited and can be wired or wireless. The HDD 106 is a non-volatile storage area. The network 600 consists of one or more networks and may include, for example, a wireless LAN (Local Area Network) or the Internet.

[0019] The vibration sensor 200 is a sensor that detects vibrations generated when the switch group 500 is operated by the robot 300 and acquires them as vibration information. An example configuration of the vibration sensor 200 will be described later with reference to the drawings. The robot 300 is a robot positioned in a location where the switch group 500 can be operated. The robot 300 is configured so that the position coordinates of the operating part (for example, the tip of the robot 300) relative to the switches can be adjusted by multiple drive axes (for example, 4 axes or 6 axes). An example configuration of the robot 300 will be described later with reference to the drawings.

[0020] The external server 400 is a server for storing and managing history information and operation instruction information related to switch operations, which will be described later. The external server 400 provides and records various information in response to requests from the switch operating device 100. The external server 400 may be an on-premise or cloud-based server device. In this embodiment, an example is shown in which the switch operating device 100 and the external server 400 are configured as separate devices, but this is not the only option, and they may be configured as an integrated unit.

[0021] The switch group 500 consists of one or more switches operated by the robot 300. Switches can have various structures and shapes. In this embodiment, toggle switches, push switches, and rocker switches will be described as examples of switches.

[0022] [Handle component] Figure 2 shows an external perspective view of an example of a hand member 310 provided on the robot 300 according to this embodiment. The hand member 310 is provided at the tip of the robot 300 and operates each switch by contacting the switch group 500. The orientations of the xyz coordinate systems shown in the figures used in the following description correspond to each other. Also, the x, y, and z axes in the xyz coordinate system are orthogonal to each other. Note that the xyz coordinate system shown in Figure 2 does not necessarily coincide with the three-dimensional coordinate system (robot coordinate system) used when controlling the robot 300.

[0023] The hand member 310 includes a contact detection unit 311 and a switch operating unit 312 on the side that contacts the switch group 500. The contact detection unit 311 detects contact with an object located in the x-axis direction and includes, for example, a tactile sensor (not shown). The contact detection unit 311 may be integrated with the switch operating unit 312. Information detected by the contact detection unit 311 is notified to the switch operating device 100. Further sensors may be used to obtain information necessary for controlling the robot 300. For example, known gyro sensors, slip sensors, proximity sensors, torque sensors, etc., may be used. The switch operating unit 312 is the part for operating the individual switches that make up the switch group 500. Specific examples of switch operation will be described later with reference to Figures 3 to 5.

[0024] The hand member 310 is connected to the arm portion of the robot 300 by a connecting portion 313. A vibration sensor 200 is installed on the hand member 310, and vibration information detected by the vibration sensor 200 is notified to the switch operating device 100 via a connecting cable 201. The installation location of the vibration sensor 200 is not particularly limited, but it is desirable to install the vibration sensor 200 around the switch operating portion 312 in order to more accurately detect vibrations generated when the switch operating portion 312 comes into contact with the switch.

[0025] Furthermore, the hand member 310 may be configured to rotate around the x-axis due to the movement of the robot 300. In the example shown in Figure 2, the vibration sensor 200 is located on the upper side of the hand member 310 in the z-axis direction, but it is also possible to orient the vibration sensor 200 sideways by rotating the hand member 310. Therefore, in order to detect vibrations when operating the switch with greater accuracy, the hand member 310 may be rotated as appropriate according to its relative position to the switch. In this case, a gyro sensor may be attached to the hand member 310 to determine the position of the vibration sensor 200, that is, its orientation and placement relative to the contact surface with the switch, from the orientation of the hand member 310.

[0026] [Example of switch operation] (Toggle switch) Figure 3 is a schematic diagram illustrating an example of operating a toggle switch using the hand member 310 of the robot 300 according to this embodiment. In Figure 3, the switch group 500 shows a state in which multiple toggle switches 501 (16 in total in this example, arranged in a 4x4 configuration) are mounted on the wall. The toggle switches 501 are configured to be switchable up and down. The shape of the switch operating part 312 of the hand member 310 is not limited to a rod shape, but may be plate-shaped, hemispherical, hook-shaped, etc., depending on the arrangement and dimensions of the toggle switches 501.

[0027] The tip of the robot 300 is equipped with a hand member 310 connected to the arm portion 320. Based on instructions from the switch operating device 100, the robot 300 moves and rotates in the up, down, left, and right directions to approach, contact, and operate the target toggle switch 501 in the switch group 500. To raise the toggle switch 501, the robot 300 moves the hand member 310 from the lower side to the upper side in the z-axis direction. Conversely, to lower the toggle switch 501, the robot 300 moves the hand member 310 from the upper side to the lower side in the z-axis direction. In the example in Figure 3, the robot 300 is shown making contact with a toggle switch 501 from the upper side in the z-axis direction and operating it downwards. At this time, it is desirable that the vibration sensor 200 be placed on the surface of the hand member 310 that is opposite to the surface that contacts the toggle switch 501 via the hand member 310, or on the same side as the surface that contacts the toggle switch 501, when viewed from the tip side of the hand member 310. In Figure 3, the surface on which the vibration sensor 200 is located is the upper surface facing the contact surface with the toggle switch 501, or the lower surface which is the same as the contact surface, when the hand member 310 is viewed from the wall side along the x-axis.

[0028] This allows the vibration sensor 200 to be positioned in accordance with the direction of vibration transmitted from the switch to the hand member 310, thereby enabling more accurate detection of vibrations generated when the switch is contacted. In other words, by positioning the vibration sensor 200 in accordance with the direction of vibration along the switching direction of the switch (in this example, the z-axis direction), the detection accuracy of the vibration sensor 200 can be improved.

[0029] Furthermore, before the hand member 310 operates the switch, the contact surface between the switch and the hand member 310 may be anticipated based on the content of the next operation to be performed and information from sensors attached to the hand member.

[0030] If the vibration sensor 200 is not located on the surface opposite to the expected contact surface or on the same side as the expected contact surface, the hand member 310 may be rotated so that the vibration sensor 200 is located on the surface opposite to the contact surface or on the same side as the contact surface.

[0031] As a result, even when the hand member 310 operates in different directions when multiple switches are being operated, the vibration sensor 200 can detect vibrations generated when it comes into contact with the switches with greater accuracy.

[0032] (Push switch) Figure 4 is a schematic diagram illustrating an example of operating a push switch using the hand member 310 of the robot 300 according to this embodiment. The arm portion 320 of the robot 300 is omitted in this diagram. In Figure 4, a push switch 502 is shown as part of the switch group 500. The push switch 502 is configured to be operable by pressing it along a predetermined direction (in this example, the z-axis direction). Depending on the on / off state, the position of the upper surface (in this example, the contact surface with the switch operating part 312) of the push switch 502 may change or remain unchanged, but either is acceptable here. The shape of the switch operating part 312 of the hand member 310 may be configured to match the dimensions of the push switch 502.

[0033] Based on instructions from the switch operating device 100, the robot 300 moves up, down, left, and right, and rotates to approach, contact, and operate the target push switch 502 in the switch group 500. The robot 300 operates the push switch 502 by pressing it along the z-axis direction with a predetermined amount of pressure.

[0034] (Rocker switch) Figure 5 is a schematic diagram illustrating an example of operating a rocker switch using the hand member 310 of the robot 300 according to this embodiment. The arm portion 320 of the robot 300 is omitted in this diagram. In Figure 5, the rocker switch 503 is shown as part of the switch group 500. The rocker switch 503 is configured to be operable by pressing it along a predetermined direction (in this example, the z-axis direction). The position of the upper surface (in this example, the contact surface with the switch operating portion 312) of the rocker switch 503 changes depending on the on / off state. The shape of the switch operating portion 312 of the hand member 310 may be configured to match the dimensions of the rocker switch 503.

[0035] Based on instructions from the switch operating device 100, the robot 300 moves up, down, left, and right, and rotates to approach, contact, and operate the target rocker switch 503 in the switch group 500. The robot 300 operates the rocker switch 503 by pressing it along the z-axis direction with a predetermined amount of pressure.

[0036] The robot 300 may be configured to be installed in the vicinity of the switch group 500 (for example, on the wall of the switch group shown in Figure 3, or on the stand on which the switches are installed). Furthermore, the robot 300 may be configured to be movable only in some of the three axes related to the operation of the switches.

[0037] [Vibration waveform] The vibration waveform of the vibration information acquired by the vibration sensor 200 by operating the switch group 500 will be explained. Figure 6 shows an example of a vibration waveform acquired by the vibration sensor 200, where the vertical axis represents amplitude and the horizontal axis represents time.

[0038] Figure 6(a) shows an example of a vibration waveform generated when the push switch 502 is operated and switching is performed normally. Figure 6(b) shows an example of a vibration waveform generated when the rocker switch 503 is operated and switching is performed normally. Figure 6(c) shows an example of a vibration waveform generated when the toggle switch 501 is operated and switching is performed normally. Figure 6(d) shows an example of a vibration waveform generated when the toggle switch 501 is not operated properly due to insufficient force. Figure 6(e) shows an example of a vibration waveform generated when attempting to operate an already operated toggle switch 501 and making contact. These vibration waveforms are affected by the shape and internal structure of the switches. The internal structure includes elastic energy and may be, for example, a spring such as a coil spring or leaf spring, or a piston structure.

[0039] Therefore, as shown in Figure 6, the vibration waveform generated differs depending on the type of switch being operated and the result of the operation. In this embodiment, the switch operation is controlled and judged by focusing on this difference.

[0040] [Learning Process] In this embodiment, the operation result of the switch is determined using a trained model generated using a predetermined learning algorithm with vibration information as training data. As explained with reference to Figure 6, this embodiment handles vibration information with different vibration waveforms depending on the switch operation. It is assumed that a learning process has been performed and a trained model has been generated before the processing flow of this embodiment described later.

[0041] The learning process for generating a trained model used in the switch operating device 100 according to this embodiment will be described. Figure 7 is a conceptual diagram showing the flow for generating a trained model according to this embodiment. Here, the process is divided into a learning process for generating a trained model using training data (step S700: learning phase) and a determination process using the generated trained model (step S710: determination phase). The determination phase is included as part of the processing of the switch operating device 100, which will be described later.

[0042] In this embodiment, "learning" or "machine learning" refers to generating a "trained model" by performing learning using training data and an arbitrary learning algorithm. The trained model is updated as learning progresses using multiple training data sets, and its output changes even with the same input. Therefore, the state of the trained model is not limited to any specific point in time. Here, the model used in learning is referred to as the "learning model," and the learned model that has undergone a certain degree of learning is referred to as the "trained model." Specific examples of "learning data" will be described later, but its composition may be changed depending on the learning algorithm used. Furthermore, the learning data may include training data used for learning itself, validation data used to validate the trained model, and test data used to test the trained model. In the following description, when comprehensively referring to data related to learning, it will be referred to as "learning data," and when referring to data used when performing learning itself, it will be referred to as "training data." It should be noted that this is not intended to clearly classify the training data, validation data, and test data included in the learning data; for example, depending on the learning, validation, and testing methods, all of the learning data may also be training data.

[0043] The learning phase is performed by the processing unit of an information processing device (not shown) reading and executing various programs stored in the memory unit. The information processing device may be, for example, a PC (Personal Computer). The processing unit may consist of a CPU or a GPU (Graphical Processing Unit). The memory unit may consist of ROM, RAM, HDD, etc.

[0044] In the learning phase, learning processing and verification operations are repeatedly performed using a learning model 702 based on a predetermined learning algorithm, with learning data 701 consisting of multiple vibration information, thereby generating a trained model 715 with a certain level of accuracy. In this embodiment, a neural network-based autoencoder is used as the learning algorithm. The autoencoder can be implemented using known methods, and a detailed explanation is omitted here. In this embodiment, the vibration signal from a predetermined switch is used as the learning data 701. Here, the vibration signal when the toggle switch 501 is operated and switched normally (corresponding to Figure 6(c)) is used.

[0045] By performing a learning process using an autoencoder with the vibration signal generated by operating the toggle switch 501 as learning data 701, a trained model 715 is generated that outputs a vibration signal (estimated value) that approximates the vibration waveform generated when the toggle switch 501 is successfully operated. Note that the trained model used in the switch operating device 100 does not restrict which trained model is used. Therefore, learning processing may be performed as needed, and the trained model held by the switch operating device 100 may be updated with the updated trained model.

[0046] Furthermore, while this embodiment demonstrates the use of an autoencoder, a machine learning technique, as the learning method, it does not particularly limit the specific autoencoder algorithm used. Deep learning techniques using neural networks may be used, such as well as known methods like convolutional neural networks (CNNs), or other algorithms may be used.

[0047] The determination process (step S710) is performed during the operation of the switch operating device 100. First, as described above, a vibration signal 714 (detected value) is acquired by the vibration sensor 200 when the switch is operated. By applying the trained model 715 generated by the learning process (step S700) to the vibration signal 714 as input, a vibration signal 716 (estimated value) that approximates the vibration waveform of the toggle switch 501 is output.

[0048] Then, the error is detected by comparing vibration signal 714 and vibration signal 716 (step S712). The error detection here may be performed using, for example, a known mean squared error.

[0049] Furthermore, based on the error obtained in step S712, the operation result on the switch is determined (step S713). In step S713, for example, a threshold for the error is set in advance, and the operation result is determined by comparison.

[0050] The above example illustrates the process of determining the type of object being operated on and the success or failure of the operation, but it is not limited to this. If the characteristics of the generated vibration signal can be identified, the system may be configured to detect switch failures or malfunctions.

[0051] [Processing flow] Figure 8 shows a flowchart of the process during switch operation of the switch operating device 100 according to this embodiment. This processing flow may be realized by the control unit 103 of the switch operating device 100 reading and executing a program stored in a storage unit such as an HDD 106. Furthermore, it is assumed that a trained model has been generated by a training process and is available for use by the switch operating device 100 before this processing flow starts.

[0052] The switch operating device 100 reads an operation target list that defines the operation content of the switch group 500 (step S801). The operation target list may specify the arrangement and position coordinates of the switches to be operated, the operation content (whether to turn on or off), the number of operations, etc. The operation target list may be read from a DB 401 on an external server 400, or from an HDD 106 within the switch operating device 100. Alternatively, the operation target list may be read from a user specified via the UI unit 102.

[0053] The switch operating device 100 controls the robot 300 based on the list of target switches read in step S801, and adjusts the position of the hand member 310 around the target switch (step S802). The amount of control here may be predetermined according to the operation content of the target switch.

[0054] The switch operating device 100 performs operations on the switches to be operated by the robot 300 based on the list of targets read in step S801 (step S803).

[0055] The switch operating device 100 acquires vibration information generated by the operation performed in step S803 via the vibration sensor 200 (step S804).

[0056] The switch operating device 100 analyzes the vibration information using the vibration information acquired in step S804 and the trained model 715 (step S805). In this analysis, the content of the judgment process (step S710) explained with reference to Figure 7 is performed.

[0057] The switch operating device 100 determines whether the switch operation was successful or not based on the analysis results in step S805. If the switch operation was successful (step S806; YES), the switch operating device 100 proceeds to step S807. On the other hand, if the switch operation was unsuccessful (step S806; NO), the switch operating device 100 proceeds to step S809.

[0058] The switch operating device 100 generates management metadata regarding the operation result (success) (step S807). This metadata may include information about the switch to be operated, the date and time of operation, the operation result (success), and information about the robot that performed the operation.

[0059] The switch operating device 100 records the metadata generated so far in a database (step S808). The destination for this recording may be the HDD 106 within the switch operating device 100, or it may be DB 401 located on the external server 400. At this time, the switch operating device 100 may store the metadata generated in step S807 in association with the list of operation targets read in step S801. Then, this processing flow is terminated.

[0060] The switch operating device 100 generates metadata for management purposes regarding the operation result (failure) (step S809). This metadata may include information about the switch being operated, the date and time of operation, the operation result (failure), the cause of the failure, and information about the robot that performed the operation.

[0061] The switch operating device 100 displays a message on the UI unit 102 indicating that the operation on the switch failed (step S810). The method of display here is not particularly limited, but for example, it may be displayed on the screen or a lamp may be blinked. Then, the process of the switch operating device 100 returns to step S802 and the process is repeated.

[0062] Furthermore, if the operation target list specifies operations on multiple switches, or multiple operations on a single switch, the steps shown in Figure 8 may be repeated for each operation. In the example in Figure 8, if an operation on a switch fails, the robot 300's operations are repeated, and then, when the operation on the switch is successful, the data is recorded in the database. However, this flow is not limited to this example; a configuration in which data is recorded in the database each time an operation on a switch fails is also possible.

[0063] As described above, the switch operating device 100 includes a vibration sensor 200 that acquires vibration signals generated when the switch group 500 is switched, and a control unit 103 that determines the state of the switch group 500 using the vibration signals. This makes it possible to determine the operation result of the switch based on the vibration signals when the switch is operated.

[0064] The switch group 500 has a spring, and the vibration signal includes vibrations generated by the spring when the position is switched. This makes it possible to accurately detect vibrations caused by the spring when the position is switched and to accurately determine the state.

[0065] This system targets switches and determines their position. This makes it possible to accurately determine the state of the switch, including its position.

[0066] Furthermore, the control unit 103 determines the state of the switch group 500 by comparing the input vibration signal with the vibration signal output from the trained model 715. The trained model 715 learns the vibration signals of predetermined switches and outputs a signal indicating the vibration that occurs when the predetermined switch is switched to a different position. This makes it possible to determine the operation result on the switches using a trained model generated by machine learning.

[0067] Furthermore, the control unit 103 can determine the type of switch whose position was switched, or the state of the operated switch (e.g., successful operation, unsuccessful operation, or operated) as a result of operating the switch group 500. The state of the switch may be the on / off position of the switch, or a malfunction or defect of the switch. The position of the switch may be an intermediate switching position, such as the top position when a push switch or rotary switch is pressed. Therefore, the state and position may be defined according to the configuration of the switch.

[0068] Furthermore, the switch group 500 includes either toggle switches, push switches, or rocker switches. This allows for operation detection for various types of switches.

[0069] Furthermore, the switch operating device 100 also includes an HDD 106 for recording the operation results. This makes it possible to record the switch operation information as a history.

[0070] Furthermore, the robot 300 is equipped with a hand member 310 having a contact surface that contacts the switch when the switch is operated, and the vibration sensor 200 is positioned on the same side as the contact surface, or on the side opposite the contact surface via the hand member, when viewed from the tip side of the hand member 310. This makes it possible to acquire vibration information with high accuracy.

[0071] Furthermore, the robot 300 is equipped with a hand member 310 having a contact surface that contacts the switch when the switch is operated. The control unit 103 controls the vibration sensor 200 installed on the robot 300 so that, when viewed from the tip side of the hand member 310, it is on the same side as the contact surface, or on the side opposite the contact surface via the hand member 310. This makes it possible to adjust the vibration sensor 200 to an orientation and position that allows for more accurate acquisition of vibration information.

[0072] <Embodiment 2> Embodiment 2 of the present invention will now be described. In this embodiment, assuming remote operation of a switch, a configuration will be described in which the switch operating device 100 and the remote operating device 900 operated by the user are located in different positions. Furthermore, in Embodiment 1, a configuration was described in which a trained model generated by a learning process was used to perform a decision process. In this embodiment, a configuration will be described in which a rule-based decision process is performed. Note that the explanation of content that overlaps with Embodiment 1 will be omitted, and the explanation will focus on the differences.

[0073] [System Configuration] Figure 9 is a block diagram showing an example of the overall configuration of a system comprising a switch operating device 100 and a remote control device 900 according to this embodiment. The switch operating device 100 and the remote control device 900 are connected to communicate via a network 600. The configuration of the switch operating device 100 is the same as the configuration described in Embodiment 1.

[0074] The remote control device 900 is an information processing device for issuing switch operation instructions to a switch operation device 100 located at a remote location, and is composed of, for example, a PC or a mobile terminal. The remote control device 900 includes a control unit 901, a UI unit 902, a memory 903, an HDD 904, and a communication unit 905. Each part of the remote control device 900 is connected to communicate via an internal bus or the like.

[0075] The control unit 901 realizes the various functions according to this embodiment by reading various programs and data stored in the memory 903 and HDD 904 and executing processing. The control unit 901 may be configured using at least one of a CPU, MPU, DSP, or FPGA. The UI unit 902 accepts operations from the user and displays the results of operations on the switch group 500. The UI unit 902 may include, for example, a mouse or keyboard, or it may be configured as a touch panel display in which the display unit and operation unit are integrated.

[0076] Memory 903 is a storage area for storing and holding data of various types of information, and may consist of, for example, a non-volatile storage area such as ROM or a volatile storage area such as RAM. HDD 904 is a non-volatile storage area. Communication unit 905 is a network interface for communicating with external devices (in this example, a switch operating device 100, etc.) via network 600. The communication method by communication unit 905 is not particularly limited and can be wired or wireless.

[0077] In this embodiment, when the remote control device 900 operates the switch control device 100, an application program stored in the HDD 904 or the like may be used, or a web application provided by accessing the switch control device 100 via a web browser (not shown) may be used.

[0078] [Processing Sequence] Figure 10 shows the processing sequence for switch operation according to this embodiment. This processing sequence may be realized by the control units of the switch operating device 100 and the remote control device 900, which are the processing units for each processing step, reading and executing programs stored in the memory unit.

[0079] The remote control device 900 accepts the user's selection of the switch to be operated via the UI unit 902 (step S1001). At this time, the remote control device 900 may also accept the operation content for the switch to be operated. The information accepted here may be displayed on an operation screen (not shown) for remote control that can be selected by the user. Alternatively, the user may specify an operation target list as described in Embodiment 1.

[0080] The remote control device 900 transmits the instruction received in step S1001 to the switch control device 100 (step S1002).

[0081] The switch operating device 100 controls the robot 300 based on instructions transmitted from the remote control device 900 and adjusts the position of the hand member 310 around the switch to be operated (step S1003). The amount of control here may be predetermined according to the operation content of the target switch.

[0082] The switch operating device 100 performs operations on the switch to be operated by the robot 300 based on instructions transmitted from the remote control device 900 (step S1004).

[0083] The switch operating device 100 acquires vibration information generated by the operation performed in step S1004 via the vibration sensor 200 (step S1005).

[0084] The switch operating device 100 analyzes the vibration information based on the vibration information acquired in step S1005 and predefined rules for vibration information (step S1006). The rules in this embodiment may be defined based on, for example, the peak value of the vibration waveform, the number of peaks per certain period of time, the interval between waveform occurrences, etc. In addition, the rules may define multiple conditions in order to determine the type of switch and the operation result.

[0085] The switch operating device 100 determines whether the switch operation was successful or not based on the analysis results in step S1007. If the switch operation was successful (step S1007; YES), the switch operating device 100 proceeds to step S1008. On the other hand, if the switch operation was unsuccessful (step S1007; NO), the switch operating device 100 proceeds to step S1009.

[0086] The switch operating device 100 generates management metadata regarding the operation result (success) (step S1008). This metadata may include information about the switch to be operated, the date and time of operation, the operation result (success), and information about the robot that performed the operation. The process of the switch operating device 100 then proceeds to step S1010.

[0087] The switch operating device 100 generates metadata for management purposes regarding the operation result (failure) (step S1009). This metadata may include information about the switch being operated, the date and time of operation, the operation result (failure), the cause of the failure, and information about the robot that performed the operation. The process of the switch operating device 100 then proceeds to step S1010.

[0088] The switch operating device 100 transmits the switch operation result to the remote operating device 900 (step S1010). At this time, the switch operating device 100 transmits the result including the metadata generated in step S1008 or step S1009.

[0089] The remote control device 900 displays the operation result in the UI unit 902 based on the operation result transmitted from the switch control device 100 (step S1011). The method of display here is not particularly limited, but for example, it may be displayed on a screen or a lamp or the like may be blinked.

[0090] The remote control device 900 records the operation results transmitted from the switch control device 100 to the HDD 904 (step S1012).

[0091] The remote control device 900 determines whether the operation on the switch has been completed (step S1013). For example, if it receives further operation instructions from the user, it may determine that the operation on the switch has not been completed. If it receives an instruction from the user to end the remote operation, it may determine that the operation on the switch has been completed. If the operation has been completed (step S1013; YES), this processing sequence ends. On the other hand, if the operation has not been completed (step S1013; NO), the processing of the remote control device 900 returns to step S1001 and the processing is repeated.

[0092] In this embodiment, the analysis process in step S1006 was performed using a rule-based approach, but a pre-trained model may be used, as in Embodiment 1. Furthermore, the analysis process may also be performed using a rule-based approach in the configuration of Embodiment 1.

[0093] As described above, in this embodiment, the switch operating device 100 determines the operation result of the switch group 500 based on the vibration signal and a predetermined rule. This makes it possible to determine the operation result of the switch based on the vibration signal and a predetermined rule.

[0094] Furthermore, the switch operating device 100 is connected to the remote control device 900 via the network 600, and the control unit 103 controls the operation of the switch group 500 by the robot 300 based on instructions input via the remote control device 900. This makes it possible to perform switch operations based on instructions from a remote location.

[0095] Furthermore, if the control unit 103 determines that the operation on the switch group 500 has failed, it controls the robot 300 to attempt the operation on the switch group 500 again. This allows for automatic retries even if the operation on the switch group 500 fails, thus saving the user time and effort.

[0096] Furthermore, if the control unit 103 determines that the operation on the switch group 500 has failed, it will accept instructions for operation on the switch group 500 again. This ensures that the switches can be reliably operated by accepting user instructions when the operation on the switch group 500 fails.

[0097] <Embodiment 3> Embodiment 3 of the present invention will now be described. In this embodiment, a configuration is described in which the accuracy of controlling the robot 300 is improved by providing feedback on the operation of the robot 300 based on the operation results of the switch. Note that the explanation will be omitted for content that overlaps with Embodiment 1, and the explanation will focus on the differences. The system configuration is the same as in Figure 1 of Embodiment 1.

[0098] [Processing flow] Figure 11 shows a portion of the flowchart of the process when the switch is operated by the switch operating device 100 according to this embodiment. The configuration in Figure 11 is a part of the processing flow described using Figure 8 in Embodiment 1. This processing flow may be realized by the control unit 103 of the switch operating device 100 reading and executing a program stored in a storage unit such as the HDD 106. Furthermore, it is assumed that a trained model is generated by a training process before this processing flow starts and is available for use by the switch operating device 100.

[0099] The switch operating device 100 determines whether the switch operation was successful or not based on the analysis results in step S805. If the switch operation was successful (step S806; YES), the switch operating device 100 proceeds to step S1101. On the other hand, if the switch operation was unsuccessful (step S1101; NO), the switch operating device 100 proceeds to step S1103.

[0100] Based on the analysis results in step S805, the switch operating device 100 determines whether the operating force (pressing amount) applied to the switch was appropriate (step S1101). The resulting vibration waveform differs depending on the operating force applied to the switch. In other words, by analyzing the vibration waveform generated when the switch is successfully operated, it is possible to determine whether the operating force was appropriate. More specifically, the operating force is determined to be either excessive, appropriate, or insufficient, and feedback is provided if it is excessive or insufficient. This classification may be performed, for example, using the classification by the trained model described in Embodiment 1. If the operating force is appropriate (step S1101; YES), the switch operating device 100 proceeds to step S807. On the other hand, if the operating force is inappropriate (step S1101; NO), the switch operating device 100 proceeds to step S1102.

[0101] The switch operating device 100 provides feedback based on the operation results by adjusting the control parameters related to the operation of the robot 300 (step S1102). This adjustment may be performed stepwise using preset values, or it may be performed according to the shape of the vibration waveform. After that, the process of the switch operating device 100 proceeds to step S807.

[0102] Based on the analysis results from step S805, the switch operating device 100 further analyzes the cause of the failure (step S1103). Examples of causes of failure include insufficient operating force, the switch already being pressed, and failure to make contact with the switch to be operated. The classification here may be performed using, for example, the classification by the trained model described in Embodiment 1.

[0103] The switch operating device 100 provides feedback based on the results of the switch operation. The adjustment content may differ depending on the nature of the failure cause analyzed in step S1103. In addition, manual adjustment may be required depending on the cause of the failure. Therefore, the device may be configured to switch the content of the feedback according to the analysis results in step S1103. After that, the process of the switch operating device 100 proceeds to step S809.

[0104] The following are examples of feedback depending on the cause of failure when operating a switch: If the failure is due to insufficient operating force, coordinate control may be performed to bring the relative coordinates of the switch and the hand member 310 closer together, or current control may be performed to increase the movement speed of the hand member 310 in order to increase the pressing force. In addition, if contact is made with a switch that has already been pressed, the system may be configured to notify the user without changing the control parameters. If contact with the target switch fails or if the system grazes the target switch, the system may be configured to perform coordinate control to bring the relative coordinates of the switch and the hand member 310 closer together, or to notify the user.

[0105] As described above, in this embodiment, the control unit 103 provides feedback regarding the operation of the switch group 500 based on the determination result. This makes it possible to further improve the accuracy of operation of the switch group 500 according to the determination result of the switch.

[0106] Furthermore, the feedback involves adjusting the control parameters of the robot 300 or notifying the user of the results of the operation. This makes it possible to improve the accuracy of operations on the switch group 500.

[0107] <Other Embodiments> The above embodiment shows an example using vibration information from a vibration sensor, but it is not limited to this. For example, a microphone may be used instead of a vibration sensor, and audio information may be acquired by the microphone. In this case, the IF unit 101 may be configured to include an audio interface for receiving detection results from the microphone.

[0108] Furthermore, while toggle switches, push switches, and rocker switches were given as examples of switches in the above embodiments, the present invention may also be applied to other types of switches. For example, rotary switches and slide switches may be applied. In this case, the hand member 310 may be modified to a shape that can operate rotary switches or slide switches. For example, when operating a rotary switch, the hand member 310 may be provided with a two-finger configuration that grips the rotary switch. Also, toggle switches, push switches, and rocker switches may have diverse internal structures. For example, the internal structure of each switch may be a spring structure, or a seesaw structure using a leaf spring or the like. In addition, each switch is not limited to those that involve electrical switching, but may also be a lever or the like that involves mechanical switching.

[0109] Furthermore, the above embodiment describes a configuration in which the result of operating the switch is determined based on the vibrations produced when the robot 300 operates the switch. However, the configuration is not limited to this, and the switch may be operated by a person. In this case, for example, a vibration sensor may be installed around the switch to acquire the vibration signal generated when a person operates the switch, and the type of switch that was operated and the result of the operation may be determined based on that vibration signal. Alternatively, the operation may be determined not by operation by a person or robot, but simply by the switching of the switch.

[0110] Furthermore, the above embodiments may be used in combination with other determination methods. For example, they may be combined with a method that uses an imaging device such as a camera to capture an image of the area around the switch and determines whether the switch has been operated based on that image. For example, if it is difficult to make a determination using an image, the system may switch to the determination method of the present invention, or vice versa. Alternatively, the system may be configured to use an imaging device such as a camera to operate a robot and then use the determination method of the present invention to determine whether the operation on the switch was performed correctly.

[0111] Furthermore, the present invention determines the state of an object whose state transitions by switching its position. In the above embodiment, the on / off state of a switch was used as an example of a position. Although a switch was used as an example of an object, the present invention is applicable to any object whose state is switched electrically or mechanically by a position. Here, the position of the object refers to the position of a predetermined mechanism of the object.

[0112] Furthermore, although the position was described as the state of the object, the state of the object may also be an abnormality of the object. An abnormality of the object may, for example, be the absence of mechanical changes in the object or the object being malfunctioning. For example, if the object is a door, the position may be the position when it is open or closed, or the position of the mechanism depending on whether it is locked or not. In this case, vibrations during opening, closing, and locking are acquired to determine whether it is closed properly, ajar, etc.

[0113] Furthermore, this can also be achieved by supplying programs and applications for realizing the functions of one or more embodiments described above to a system or device using a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the programs.

[0114] Alternatively, it may be implemented by a circuit that performs one or more functions (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array)).

[0115] Furthermore, the above functions may be configured on a network using cloud computing technology. In this case, terminal devices can access the above functions via the network.

[0116] Although various embodiments have been described above with reference to the drawings, it goes without saying that this disclosure is not limited to these examples. It will be clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can occur within the scope of the claims, and these will naturally fall within the technical scope of this disclosure. Furthermore, the components of the various embodiments described above can be combined arbitrarily without departing from the spirit of the invention. [Industrial applicability]

[0117] This disclosure is useful in switch operating devices for operating various switches. [Explanation of Symbols]

[0118] 100... Switch operating device 101...IF section 102...UI section 103... Control Unit 104...Memory 105... Communications Department 106…HDD 200...Vibration sensor 300... Robots 310...Handle component 400…External Server 401...DB 500... Switch group 501...Toggle switch 502... Push switch 503... Rocker switch 600…Network 900... Remote control device 901... Control Unit 902...UI section 903...Memory 904…HDD 905... Communications Department

Claims

1. An acquisition unit that acquires vibration signals generated when the position of the object is changed, A determination unit that determines the state of the object using the vibration signal, Equipped with, The object is a switch, and the state includes the position of the switch. The determination unit determines the state of the object by comparing the input signal acquired by the acquisition unit with the output signal output from the trained model. The aforementioned trained model is a determination device that learns the vibration signals of a predetermined switch and outputs a signal indicating the vibration that occurs when the predetermined switch is switched to a different position.

2. The object has a spring, The determination device according to claim 1, wherein the vibration signal includes vibrations generated by a spring when the position is switched.

3. The determination device according to claim 1, wherein the determination unit determines the operation result of the switch based on the vibration signal and a predetermined rule.

4. The determination unit determines the type of switch or the state of the switch whose position has been switched, according to claim 1.

5. The determination device according to claim 1, wherein the switch includes a toggle switch, a push switch, or a rocker switch.

6. The determination device according to any one of claims 1 to 5, further comprising a recording unit for recording the results of operations performed by the determination unit.

7. A determination device according to any one of claims 1 to 6, A robot for manipulating the aforementioned object, A sensor for detecting the aforementioned vibration signal, A control unit that controls the operation of the robot, A system equipped with these features.

8. The robot comprises a hand portion having a contact surface that comes into contact with the object when manipulating the object, The sensor is positioned on the same side as the contact surface when viewed from the tip side of the hand portion, or on the surface facing the contact surface via the hand portion. The system according to claim 7.

9. The robot comprises a hand portion having a contact surface that comes into contact with the object when manipulating the object, The system according to claim 7, wherein the control unit controls the sensor installed on the robot to be positioned on the same side as the contact surface, or on the surface facing the contact surface via the hand portion, when viewed from the tip side of the hand portion, according to the operation performed by the robot on the object.

10. The system according to any one of claims 7 to 9, wherein the control unit provides feedback regarding the operation of the object based on the determination result of the determination unit.

11. The system according to claim 10, wherein the feedback is the adjustment of the control parameters of the robot or notification of the operation result to the user.

12. If the determination unit determines that the operation on the object has failed, the control unit controls the robot to perform the operation on the object again, according to any one of claims 7 to 11.

13. The system according to any one of claims 7 to 11, wherein if the determination unit determines that the operation on the object has failed, the control unit receives instructions for the operation on the object again.

14. The aforementioned system is connected to a remote control device via a network, The system according to any one of claims 7 to 13, wherein the control unit controls the robot's operation on the object based on instructions input via the remote control device.

15. The acquisition process involves acquiring vibration signals generated when the object's position changes, and A determination step of determining the state of the object using the vibration signal, It has, The object is a switch, and the state includes the position of the switch. The determination step is a step of determining the state of the object by comparing the input signal acquired in the acquisition step with the output signal output from the trained model. The aforementioned trained model learns the vibration signal of a predetermined switch and outputs a signal indicating the vibration that occurs when the predetermined switch is switched to a different position.

Images