Information processing devices, measurement systems, robotic systems, data processing methods, and computer programs

Abstract

The processing circuit of the information processing apparatus (600) of the present invention performs: a three-dimensional conversion process, which converts two-dimensional sound data containing the motion sound of a robot (10) performing a prescribed action twice or more, and the sound signal is expressed in terms of intensity and generation time, to generate converted data as three-dimensional sound data expressing the sound signal in terms of frequency, intensity and generation time; an extraction process, which extracts two or more interval data from the converted data, including data of the time interval in which the prescribed action was performed; and a detection process, which detects motion sound data representing the motion sound of the robot contained in the object frequency band based on the result of comparing the data contained in the object frequency band, which is the same frequency band as each other, between the interval data.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and benefits from Japanese Patent Application No. 2021-127560, filed with the Japan Patent Office on August 3, 2021, which is incorporated herein by reference in its entirety as part of this application. Technical Field

[0003] This disclosure relates to an information processing apparatus, a measurement system, a robotic system, a data processing method, and a computer program for detecting specific sound data based on sound data. Background Technology

[0004] For example, Japanese Patent Application Publication No. 2018-147390 discloses an abnormal waveform detection system for monitoring the periodic movements of a robotic arm. This abnormal waveform detection system detects anomalies in the waveform of the measured values ​​from vibration sensors installed on the robotic arm, i.e., the object waveform, based on a reference waveform.

[0005] In Japanese Patent Application Publication No. 2018-147390, for example, it is very difficult to use the waveform of the robot's motion sound as the object waveform. The sound signal data detected by a sound detector such as a noise meter, i.e., the sound data, may also include sound signals from sound sources other than the robotic arm. This disclosure provides an information processing apparatus, a measurement system, a robot system, and a computer program for detecting robot motion sound data based on sound data. Summary of the Invention

[0006] One aspect of the information processing apparatus disclosed herein includes a processing circuit, wherein the processing circuit performs: a three-dimensional conversion process, converting two-dimensional sound data containing the motion sound of a robot performing a predetermined action twice or more, and expressing the sound signal in terms of intensity and generation time, to generate converted data as three-dimensional sound data expressing the sound signal in terms of frequency, intensity, and generation time; an extraction process, extracting two or more interval data from the converted data, including data containing the time interval of the predetermined action; and a detection process, detecting motion sound data representing the motion sound of the robot contained in the object frequency band based on the result of comparing data contained in an object frequency band that is the same as each other in the interval data. Attached Figure Description

[0007] Figure 1 This is a top view showing an example of the structure of the robot system involved in an illustrative implementation.

[0008] Figure 2 This is a block diagram illustrating an example of the hardware structure of a robot controller and a measurement controller according to an exemplary implementation.

[0009] Figure 3 This is a block diagram illustrating an example of the functional structure of a robot controller, measurement controller, and information processing device involved in an exemplary implementation.

[0010] Figure 4 It is a graph that represents an example of 3D transformed data.

[0011] Figure 5 This is a diagram showing an example of the waveform and envelope of the sound signal contained in the frequency band of an object.

[0012] Figure 6 This is a diagram illustrating an example of envelope overlap processing.

[0013] Figure 7 This is a diagram representing an example of the object removed from the difference between envelopes.

[0014] Figure 8 This is a diagram showing an example of the waveform of a sound signal after the difference between the envelopes has been removed.

[0015] Figure 9 This is a flowchart illustrating an example of the actions of a robot system involved in an exemplary implementation.

[0016] Figure 10 This is a top view showing an example of the structure of a robot system involved in a variation of an exemplary implementation.

[0017] Figure 11 This is a block diagram illustrating an example of the functional structure of an information processing apparatus involved in a variant of an exemplary implementation.

[0018] Figure 12 This is a graph showing a comparison of the time difference values ​​of motion sound data.

[0019] Figure 13 This is a graph representing another comparative example of the time difference in motion sound data. Detailed Implementation

[0020] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings. Furthermore, the embodiments described below are general or specific examples. Elements in the following embodiments that are not described in the independent claims representing the highest-level concept are described as arbitrary elements. The figures in the accompanying drawings are schematic and not necessarily strictly illustrative. Furthermore, in the figures, the same reference numerals are used for substantially identical elements, and repeated descriptions are sometimes omitted or simplified. In this specification and claims, "apparatus" can mean not only a single apparatus but also a system composed of multiple apparatuses.

[0021] [Structure of the Robot System]

[0022] Side reference Figure 1 The structure of the robot system 1 according to the exemplary embodiment will be described. Figure 1 This is a top view illustrating an example of the structure of a robot system 1 according to an exemplary embodiment. The robot system 1 includes a robot 100, peripheral devices 200, a robot controller 300, a sound detector 400, a measurement controller 500, an information processing device 600, an output device 700, and an input device 800. The sound detector 400, the measurement controller 500, and the information processing device 600 are also configured as components of a measurement system 10. The measurement controller 500 is an example of a controller. Neither the robot system 1 nor the measurement system 10 necessarily includes all of the aforementioned components.

[0023] In this embodiment, the information processing device 600 is included in the measurement controller 500. However, the information processing device 600 may also be a device separately from the measurement controller 500. For example, the information processing device 600 may be a separate device or may be included in the robot controller 300 or other devices. The measurement controller 500 may be a device separately from the robot controller 300, but it may also be included in the robot controller 300.

[0024] Although not limited, in this embodiment, robot 100 is an industrial robot. Robot 100 includes one or more robotic arms 101 and one or more end effectors 102. The robotic arm 101 has one or more joints, but in this embodiment, it is a multi-joint robotic arm with two or more joints. The end effector 102 is detachably mounted to the distal end of the robotic arm 101. The end effector 102 is capable of applying action to the object being handled by robot 100. The end effector 102 has a configuration corresponding to the action; in this embodiment, it has a configuration for holding the object. The robotic arm 101 can freely change the position and posture of the end effector 102. The robotic arm 101 includes two or more servo motors as power sources for two or more joints. The end effector 102 includes servo motors as power sources for the part performing the holding action. Robot 100 generates operating sounds.

[0025] The robotic arm 101 is a vertical multi-joint type, but it can also be a horizontal multi-joint type, polar coordinate type, cylindrical coordinate type, rectangular coordinate type, or other types. Robot 100 is not limited to industrial robots; it can also be a service robot or humanoid robot that provides services to users. Examples of services include nursing, medical care, cleaning, security, guidance, rescue, cooking, sales, rental, and goods provision.

[0026] Peripheral device 200 is configured in the same space as robot 100, such as in the work area where robot 100 performs operations in a factory or warehouse. Although not limited to this, in this embodiment, peripheral device 200 is a belt conveyor configured near robot 100. Peripheral device 200 generates operating noise. The belt conveyor is capable of transporting objects to and from robot 100, or both.

[0027] The robot controller 300 is configured in the same space as the robot 100, specifically within the work area, but may also be configured outside the work area. The robot controller 300 is connected to the robot 100, peripheral devices 200, and measurement controller 500 via wired communication, wireless communication, or a combination of both. The robot controller 300 controls the movement of the motors and other power sources of the robot 100 and peripheral devices 200. The robot controller 300 can send and receive commands, information, and data from the measurement controller 500. For example, the robot controller 300 causes the robot 100 to move according to received commands. The robot controller 300 includes a computer and may also include circuitry for controlling the power supply to the power sources of the robot 100 and peripheral devices 200.

[0028] A sound detector 400 is configured in the same space as the robot 100, specifically within the work area. The sound detector 400 is positioned to detect the sound generated by the robot 100 during its movements. The sound detector 400 is connected to the measurement controller 500 via wired communication, wireless communication, or a combination of both. The sound detector 400 can send and receive commands, information, and data from the measurement controller 500. For example, the sound detector 400 performs sound detection based on received commands and sends the detection results to the measurement controller 500.

[0029] The sound detector 400 includes a microphone capable of detecting ambient sound. The type of microphone is not particularly limited. Examples of microphone types include electrodynamic, piezoelectric, and capacitive types. The sound detector 400 may also include a converter that converts the sound signal detected by the microphone into data that can be sent to the measurement controller 500, but this converter is not mandatory. The converter may include circuitry, a computer, or a combination of circuitry and a computer. The sound detector 400 can be an existing detector such as a noise meter, or a detector specifically designed for the robot system 1.

[0030] The measurement controller 500 can be configured in any location. The measurement controller 500 connects to the robot controller 300, sound detector 400, output device 700, and input device 800 via wired communication, wireless communication, or a combination of both. The measurement controller 500 can send and receive commands, information, and data from the robot controller 300, sound detector 400, output device 700, and input device 800. The measurement controller 500 can also send and receive commands, information, and data from the information processing device 600. The measurement controller 500 includes a computer. Examples of the measurement controller 500 may include electronic circuit boards, electronic control units, microcomputers, personal computers, workstations, smartphones, tablets, and other smart devices and electronic equipment.

[0031] For example, the measurement controller 500 receives various instructions from the input device 800. Based on the instructions received from the input device 800, the measurement controller 500 generates instructions to cause the robot 100 to perform a predetermined action, i.e., a predetermined action execution instruction, and sends it to the robot controller 300. The measurement controller 500 may also receive information indicating the motion state of the robot 100 from the robot controller 300. Based on the instructions received from the input device 800, the measurement controller 500 generates instructions to perform sound detection, i.e., a detection execution instruction, and sends it to the sound detector 400. For example, the measurement controller 500 may also send the sound detection instruction to the sound detector 400 in a manner that synchronizes the execution timing of the robot 100's predetermined action with the execution timing of the sound detection. The measurement controller 500 may also use the motion state of the robot 100 received from the robot controller 300 to control the synchronization.

[0032] The prescribed action may also include one action or a combination of actions of the robot 100. The prescribed action may be an action of the robotic arm 101. Examples of an action may include bending, rotating, twisting, or a combination of two or more of the above actions of the robotic arm 101. Although not limited, in this embodiment, the prescribed action is the action used to measure the motion sound of the robot 100, for example, it may include bending and rotating actions of the robotic arm 101 at high speed. Hereinafter, the prescribed action used for measuring this motion sound may also be described as a "prescribed measurement action".

[0033] The prescribed action execution instruction may also include instructions on the type of prescribed measurement action, the timing of the prescribed measurement action, the number of times the prescribed measurement action is executed, and the interval between the prescribed measurement actions. In this embodiment, the number of executions is two or more. In this case, the robot controller 300, based on the instruction content of the prescribed action execution instruction, causes the robot 100 to autonomously perform the prescribed measurement action two or more times. The prescribed action execution instruction may also only instruct the execution of the prescribed measurement action. In this case, the robot controller 300, based on a pre-stored program, causes the robot 100 to autonomously perform the prescribed measurement action at a prescribed timing, a prescribed number of executions (two or more), and a prescribed interval.

[0034] The measurement controller 500 receives signals or data representing the detection results of the sound detector 400 from the sound detector 400, converts the signals or data to generate data that the information processing device 600 can process, i.e., converted sound data, and sends the converted sound data to the information processing device 600. The measurement controller 500 can also output information and data received from the information processing device 600 to the output device 700 and external devices. The external devices can also be devices external to the robot system 1.

[0035] The information processing device 600 can be configured as a separate device within the measurement controller 500, or it can be integrated with the measurement controller 500. In the latter case, the functions of the information processing device 600 can also be implemented through the functions of the measurement controller 500. The information processing device 600 can send and receive commands, information, and data from the measurement controller 500. The information processing device 600 includes a computer. In this embodiment, examples of the information processing device 600 include electronic circuit boards, electronic control units, and microcomputers, but it can also include smart devices such as personal computers, workstations, smartphones, and tablet computers, as well as other electronic devices.

[0036] For example, the information processing device 600 processes the converted sound data received from the measurement controller 500, and detects motion sound data of the robot 100 from the converted sound data. The information processing device 600 then sends the motion sound data of the robot 100 to the measurement controller 500.

[0037] The output device 700 can be configured in any location. The output device 700 can send and receive commands, information, and data from the measurement controller 500. For example, the output device 700 can output information and data received from the measurement controller 500 in a visual, auditory, or both manner. The output device 700 may also include one or more of a display, projector, speaker, and printing apparatus. The output device 700 can output images via one or more of the display and projector, output sound via the speaker, and output a recording medium printed with information and data via the printing apparatus. Examples of displays may include liquid crystal displays, organic EL (Electroluminescence), and inorganic EL. Examples of recording media may include, but are not limited to, sheets such as paper, cloth, film, and plates.

[0038] The input device 800 can be configured in any location. The input device 800 can send and receive commands, information, and data from the measurement controller 500. The input device 800 accepts input from the user of the robot system 1 and sends commands, information, and data corresponding to the input content to the measurement controller 500. The input device 800 may also include one or more devices such as a device that provides input via user operation, a camera that provides input via the user's image, and a microphone that provides input via the user's voice. The device that provides input via user operation may also include one or more of the following: a button, joystick, dial, joystick, mouse, key, touch panel, and motion capture device.

[0039] Devices performing functions, such as the robot controller 300, sound detector 400, measurement controller 500, and information processing device 600, may also include circuits or processing circuits. Circuits may also include processing circuits. Processing circuits or circuits include processors and storage devices. Processing circuits or circuits can function as components of a computer. Processing circuits or circuits transmit and receive instructions, information, and data from other devices. Processing circuits or circuits accept input signals from various devices and output control signals to the controlled object.

[0040] Storage devices may also include memory, storage, or both. Examples of memory include RAM (Random Access Memory), which is volatile semiconductor memory, and ROM (Read-Only Memory), which is non-volatile semiconductor memory. Examples of storage include semiconductor memory such as flash memory, hard disks, and SSDs (Solid State Drives). For example, storage devices store programs executed by processing circuits or circuits, as well as various data.

[0041] At least some of the functions of the apparatus performing the above-described functions can also be implemented through the cooperation of a processor, memory, and storage devices. The processor and memory, including RAM and ROM, can function as components of a computer system. For example, a computer system can also implement the above-described functions by having a processor use RAM as a working area to execute programs recorded in ROM.

[0042] The functions of the device performing the above-mentioned functions can be partially or entirely implemented by a computer system, or by dedicated hardware circuits such as electronic circuits or integrated circuits, or by a combination of a computer system and hardware circuits. The device performing the above-mentioned functions can be processed by a single processing circuit or by centralized control of circuits, or by distributed control of multiple processing circuits or by the collaborative cooperation of circuits.

[0043] While not limited to a single device, a processor can include, for example, a CPU (Central Processing Unit), an MPU (Microprocessing Unit), a GPU (Graphics Processing Unit), a microprocessor, a processor core, a multiprocessor, an ASIC (Application-Specific Integrated Circuit), a FPGA (Field-Programmable Gate Array), and a reconfigurable processor. Processing can also be achieved through hardware circuitry, i.e., logic circuitry or application-specific circuitry, formed on integrated circuits such as IC (Integrated Circuit) chips and LSI (Large Scale Integration) chips. Multiple functions of the aforementioned device can be implemented through integrated circuits integrated individually on a single chip, or through integrated circuits that integrate some or all functions of a single chip.

[0044] [Hardware Structure]

[0045] Reference Figure 2 An example of the hardware structure of the robot controller 300 and the measurement controller 500 according to an exemplary embodiment will be described. Figure 2 This is a block diagram illustrating an example of the hardware structure of the robot controller 300 and measurement controller 500 according to an exemplary embodiment. The hardware structure described below is an example, and the hardware structure of the robot controller 300, measurement controller 500, and information processing device 600 is not limited to the following structure and can be appropriately modified.

[0046] The robot controller 300 includes a control unit 310 and a drive circuit 320. The control unit 310 includes a processor P, a memory M, a storage S, an input / output (I / F) interface 311, and a drive I / F 312 as its components. The components of the control unit 310 are interconnected via a bus B, but can also be connected via other wired communication, wireless communication, or a combination of wired and wireless communication. Examples of the control unit 310 may include an electronic circuit board, an electronic control unit, a microcomputer, and other electronic devices. The processor P and the memory M may be included in the processing circuitry that the control unit 310 may include.

[0047] Input / output I / O 311 is connected to the measurement controller 500 and controls the transmission and reception of signals between the control unit 310 and the measurement controller 500. Drive I / O 312 is connected to the drive circuit 320 and controls the transmission and reception of signals, etc., between the control unit 310 and the drive circuit 320. The drive circuit 320 controls the current supplied to the robot 100.

[0048] The measurement controller 500 includes a processor P, a memory M, a storage unit S, and input / output (I / F) units 501 to 504. The processor P, memory M, storage unit S, and I / F units 501 to 504 are interconnected via a bus B, but can also be connected via other wired communication, wireless communication, or a combination of wired and wireless communication. Although not limited to this embodiment, in this case, the processor P, memory M, and storage unit S of the measurement controller 500 implement the functions of both the measurement controller 500 and the information processing device 600. The processor P and memory M may be included in the processing circuitry that the measurement controller 500 may include.

[0049] Input / output I / F 501 is connected to input / output I / F 311 of the robot controller 300, controlling the transmission and reception of signals between the measurement controller 500 and the robot controller 300. Input / output I / F 502 is connected to the sound detector 400, controlling the transmission and reception of signals between the measurement controller 500 and the sound detector 400. Input / output I / F 503 is connected to the output device 700, controlling the transmission and reception of signals between the measurement controller 500 and the output device 700. Input / output I / F 504 is connected to the input device 800, controlling the transmission and reception of signals between the measurement controller 500 and the input device 800.

[0050] [Functional Structure]

[0051] Side reference Figure 3 An example of the functional structure of the robot controller 300, measurement controller 500, and information processing device 600 according to an exemplary embodiment will be described. Figure 3 This is a block diagram illustrating an example of the functional structure of the robot controller 300, measurement controller 500, and information processing device 600 according to an exemplary embodiment. The functional structure described below is an example, and the functional structure of the robot controller 300, measurement controller 500, and information processing device 600 is not limited to the following structure and can be appropriately modified.

[0052] The robot controller 300 includes a motion control unit 300a and a storage unit 300b as functional structural elements. The measurement controller 500 includes an instruction unit 500a, a conversion unit 500b, an output processing unit 500c, and a storage unit 500d as functional structural elements. The information processing device 600 includes a first conversion unit 600a, an extraction unit 600b, a detection unit 600c, a second conversion unit 600d, and a storage unit 600e as functional structural elements.

[0053] In the robot controller 300, the functions of the motion control unit 300a can be realized by the processor P and the memory M, and the functions of the storage unit 300b can be realized by the memory M and the storage S.

[0054] The storage unit 300b stores programs for enabling the robot 100 to move autonomously. For example, the storage unit 300b stores programs for enabling the robot 100 to autonomously perform specified measurement actions, or programs for performing specified measurement actions at specified execution times, at specified number of executions, and at specified intervals.

[0055] The motion control unit 300a controls the movements of the robot 100. The motion control unit 300a controls the autonomous movements of the robot 100 according to the program stored in the storage unit 300b. The motion control unit 300a enables the robot 100 to autonomously execute predetermined measurement actions according to the predetermined action execution instructions received from the measurement controller 500. The motion control unit 300a can obtain information such as rotation amount and current value from the servo motors of the robot 100 and use this information to perform feedback control on the servo motors.

[0056] In the measurement controller 500, the functions of the instruction unit 500a, the conversion unit 500b, and the output processing unit 500c can be implemented by the processor P and the memory M, etc., and the functions of the storage unit 500d can be implemented by the memory M and the storage S.

[0057] The instruction unit 500a generates instructions to be output to the sound detector 400 and the robot controller 300 based on the instructions received from the input device 800. An example of an instruction received from the input device 800 is an instruction to measure the motion sound of the robot 100, i.e., a measurement execution instruction. If the instruction unit 500a receives a measurement execution instruction, it sends a specified action execution instruction for performing the specified measurement action to the robot controller 300 and a detection execution instruction for detecting sound to the sound detector 400. The instruction unit 500a receives information about the robot 100's motion state from the robot controller 300 and sends a sound detection instruction to the sound detector 400 to synchronize the execution timing of the specified measurement action of the robot 100 with the execution timing of sound detection by the sound detector 400.

[0058] The conversion unit 500b receives a signal or data representing the detection result of the sound detector 400 from the sound detector 40, converts the signal or data to generate converted sound data that can be processed by the information processing device 600, and sends the converted sound data to the information processing device 600. The converted sound data is two-dimensional sound data representing a sound signal as an intensity and generation time. The two-dimensional sound data includes intensity and time as components of the sound signal. Examples of sound signal intensity can include sound pressure level. The conversion unit 500b can also, based on the synchronous control of the two execution times of the specified measurement action and sound detection by the instruction unit 500a, establish a correlation between the intensity and generation time of the sound signal in the converted sound data and the execution time of the specified measurement action of the robot 100.

[0059] The output processing unit 500c converts information and data received from the information processing device 600 into data that the output device 700 can output, and sends it to the output device 700. For example, the output processing unit 500c converts information and data received from the information processing device 600 into data corresponding to the output format requested by the input device 800.

[0060] The storage unit 500d stores information and data used by the instruction unit 500a, the conversion unit 500b, and the output processing unit 500c during processing. For example, the storage unit 500d stores the program executed by the measurement controller 500.

[0061] In the information processing device 600, the functions of the first conversion unit 600a, the extraction unit 600b, the detection unit 600c and the second conversion unit 600d can be implemented by the processor P and the memory M, etc., and the function of the storage unit 600e can be implemented by the memory M and the storage S.

[0062] Storage unit 600e stores information and data used by the first conversion unit 600a, extraction unit 600b, detection unit 600c, and second conversion unit 600d during processing. Storage unit 600e stores the program executed by information processing device 600.

[0063] The first conversion unit 600a converts the converted sound data, which is two-dimensional sound data, to generate three-dimensional converted sound data, which is three-dimensional sound data representing the sound signal in terms of frequency, intensity, and generation time. Three-dimensional converted data is an example of converted data. The three-dimensional converted data includes frequency, intensity, and generation time as components of the sound signal. For example, the first conversion unit 600a uses conversion methods such as wavelet transform and short-time Fourier transform in the conversion process. Although not limited, in this embodiment, the first conversion unit 600a uses continuous wavelet transform (CWT). In the three-dimensional converted data, the frequency, intensity, and generation time of the sound signal are represented, for example, by a three-dimensional coordinate space where the X-axis is time T, the Y-axis is frequency F, and the Z-axis is intensity SP. Examples of intensity may include sound pressure level. For example, the three-dimensional converted data can be... Figure 4 That's how it's expressed. Figure 4 This is a graphical representation of an example of 3D transformed data. Figure 4 In the example, the 3D conversion data includes data from two specified measurement actions.

[0064] Although not limited, in this embodiment, the first conversion unit 600a divides the frequency F of the Y-axis into multiple frequency bands. Furthermore, the first conversion unit 600a can also, for each of the multiple frequency bands, convert the data contained in the same frequency band into two dimensions on the XZ plane. That is, the first conversion unit 600a can also project the data contained in the same frequency band onto the XZ plane, and determine the two-dimensional data on the projection plane as the data for subsequent processing. Afterwards, the three-dimensional converted data of each frequency band can be processed as two-dimensional data. For example, in... Figure 4 In this example, the frequency F of the Y-axis is divided into nine frequency bands, FB1 to FB9. Data can be represented in two dimensions within each of these frequency bands.

[0065] Extraction unit 600b extracts interval data, including data from the time interval during the execution of a specified measurement action, from the 3D conversion data. Extraction unit 600b extracts two or more interval data. Figure 4 In this example, two interval data are extracted. Specifically, the first interval data D1 of the first time interval T1 and the second interval data D2 of the second time interval T2 are extracted. An interval data includes a time interval in which a specified measurement action is performed one or more times; in this embodiment, it includes a time interval in which a specified measurement action is performed once. For example, the extraction unit 600b may also use the relationship between the execution timing of the specified measurement action of the robot 100 and the execution timing of the sound detection, and the period of the specified measurement action to extract the interval data. In this embodiment, the length of the first time interval T1 and the length of the second time interval T2 are the same, but it is not limited to this and they may be different from each other.

[0066] The detection unit 600c compares data contained in an object frequency band that is the same as each other between interval data, and detects motion sound data representing the motion sound of the robot 100 contained in that object frequency band based on the comparison result. The detection unit 600c can also compare data with the same period among the data contained in the object frequency band that is the same as each other. Figure 4 In the example, the detection unit 600c detects motion sound data based on the comparison result between the first interval data D1 and the second interval data D2. The detection unit 600c performs motion sound data detection processing for all frequency bands.

[0067] The following describes the detailed function of the detection unit 600c, which detects motion sound data in a target frequency band. The detection unit 600c performs processing on each data interval to determine the envelope representing the relationship between the intensity of the sound signal contained in the target frequency band and its generation time. Hereinafter, "the intensity of the sound signal in the envelope" will sometimes be expressed as "the intensity of the envelope." For example, as... Figure 5 As shown, the sound signal contained in the object's frequency band forms a waveform W that vibrates with an extremely short period. Figure 5 This is a diagram illustrating an example of the waveform and envelope of an audio signal contained within a target frequency band. The detection unit 600c calculates the envelope that is tangent to a plurality of convex curve portions contained in such a waveform W and represents the trajectory of the tangent points on those convex curve portions. The detection unit 600c calculates the envelope Ea tangent to a portion of the waveform W with positive intensity, the envelope Eb tangent to a portion of the waveform W with negative intensity, or both envelopes Ea and Eb. Envelopes Ea and Eb represent the approximate shape of the waveform W.

[0068] Here, in this specification and claims, the intensity of a sound signal can mean any one of the intensity in the positive direction, the intensity in the negative direction, and the absolute value of the intensity of the sound signal. The magnitude of the intensity of a sound signal can mean the absolute value of the intensity of the sound signal.

[0069] Furthermore, the detection unit 600c compares the envelopes of the target frequency bands between data intervals. Specifically, the detection unit 600c performs a process to overlap the envelopes and detects the intensity difference between the envelopes. The detection unit 600c can also determine the reference position for the overlap of each envelope based on the relationship between the execution timing of the robot 100's prescribed measurement action and the execution timing of sound detection, the period of the prescribed measurement action, and the shape of the envelopes, and overlap the envelopes in a manner that makes the reference positions consistent. For example, the detection unit 600c can also determine the reference position by overlapping the portions of two envelopes with the same period.

[0070] For example, such as Figure 6As shown, the detection unit 600c makes the envelope E1 of the frequency band FB8 of the first interval data D1 overlap with the envelope E2 of the frequency band FB8 of the second interval data D2. Figure 6 This diagram illustrates an example of envelope overlap processing. In this embodiment, the detection unit 600c uses an envelope with positive intensity, but is not limited to this. The detection unit 600c detects the intensity difference between envelopes E1 and E2 based on the overlap result. The detection unit 600c detects the aforementioned difference between envelopes E1 and E2 at the same time point relative to the reference position, i.e., at the same phase, within the cycle of a defined measurement operation.

[0071] The detection unit 600c modifies the intensity of the sound signal corresponding to one or more envelopes to remove the difference between the envelopes. This difference may correspond to the portion of the sound signal intensity affected by sound from sources other than the robot 100, such as the peripheral device 200. While not limited to this, in this embodiment, the detection unit 600c removes differences exceeding a threshold Th. In this case, when removing the difference, the detection unit 600c modifies the intensity of the sound signal corresponding to an envelope with a larger sound signal intensity to an envelope with a smaller sound signal intensity. That is, the detection unit 600c modifies the sound signal by reducing the amplitude of the larger sound signal. The detection unit 600c modifies the sound signal intensity so that the difference becomes below the threshold Th.

[0072] For example, such as Figure 6 As shown, in the interval E1a of envelope E1, the difference between the intensity of envelope E1 and the intensity of envelope E2 exceeds the threshold Th, meaning the intensity of envelope E1 is greater than the intensity of envelope E2. The detection unit 600c, for example, reduces... Figure 7 The intensity of the sound signal in the positive direction contained in the portion W1a within the interval E1a of the waveform W1 corresponding to the envelope E1, as shown, is such that it is below the intensity obtained by adding a threshold Th to the intensity of the envelope E2. Figure 7 In the example, the detection unit 600c changes the intensity of the positive direction of the sound signal contained in part W1a to the intensity obtained by adding a threshold Th to the intensity of the envelope E2. Figure 7 This is a diagram representing an example of the object removed from the difference between envelopes.

[0073] Furthermore, the detection unit 600c modifies the negative direction intensity of the sound signal contained in portion W1a so that the absolute value of the negative direction intensity is lower than or equal to the intensity of envelope E2 plus a threshold Th. For example, the absolute value becomes the intensity obtained by adding the threshold Th to the intensity of envelope E2. Alternatively, the detection unit 600c may calculate the negative direction envelope of the waveform W2 of the sound signal corresponding to envelope E2 and modify the negative direction intensity of the sound signal in portion W1a to be higher than or equal to the intensity obtained by subtracting the threshold Th from the intensity of the envelope. Alternatively, the detection unit 600c may use the intensity obtained by modifying the positive direction intensity of the modified sound signal in portion W1a to a negative value as the negative direction intensity of the sound signal in portion W1a.

[0074] For example, such as Figure 8 As shown, the detection unit 600c, through the above processing, changes the waveforms W1 and W2 of the sound signal to waveforms W1A and W2A, respectively. Figure 8 This is a diagram illustrating an example waveform of an audio signal after removing the differences between the envelopes. In waveform W1A, the portion of waveform W1A within interval E1a is modified from portion W1a of waveform W1. Waveform W2A remains unchanged and is identical to waveform W2.

[0075] The detection unit 600c uses interval data of the object frequency band containing the intensity of the altered sound signal—that is, interval data of the object frequency band reflecting the intensity of the altered sound signal—to detect motion sound data contained in the object frequency band. The detection unit 600c determines the data of the sound signal contained in the object frequency band, after processing to alter the intensity of the sound signal by removing the difference between envelopes, as motion sound data. Figure 8 In the example, the detection unit 600c determines the intensity and generation time of the sound signal that forms waveform W1A as the motion sound data of frequency band FB8 in the first interval data D1. The detection unit 600c determines the intensity and generation time of the sound signal that forms waveform W2A as the motion sound data of frequency band FB8 in the second interval data D2.

[0076] The second conversion unit 600d converts a data set containing motion sound data of different frequency bands to generate two-dimensional motion sound data, which is two-dimensional sound data representing sound signals in terms of intensity and generation time. For example, the second conversion unit 600d uses an inverse transform method relative to transformation methods such as wavelet transform and short-time Fourier transform for conversion processing. The second conversion unit 600d sends the two-dimensional motion sound data to the measurement controller 500. For example, the second conversion unit 600d can convert a data set containing motion sound data of two or more frequency bands within a data interval to generate two-dimensional motion sound data, or it can convert a data set containing motion sound data of all frequency bands within a data interval to generate two-dimensional motion sound data. The second conversion unit 600d can convert a data set containing motion sound data of two or more different frequency bands within two or more data intervals to generate two-dimensional motion sound data, or it can convert a data set containing motion sound data of all frequency bands within two or more data intervals to generate two-dimensional motion sound data. The two-dimensional motion sound data is ordinary sound data and can be used to reproduce the motion sound of the robot 100.

[0077] For example, in the illustrated example, the second conversion unit 600d converts a data group including motion sound data in frequency bands FB1 to FB9 containing first interval data D1 to generate first two-dimensional motion sound data, and converts a data group including motion sound data in frequency bands FB1 to FB9 containing second interval data D2 to generate second two-dimensional motion sound data. Furthermore, the second conversion unit 600d can also combine the first two-dimensional motion sound data and the second two-dimensional motion sound data to generate a single two-dimensional motion sound data. For example, in the generated two-dimensional motion sound data, the second interval data D2 is appended to the first interval data D1.

[0078] [The actions of the robot system]

[0079] Side reference Figure 1 and Figure 9 The operation of the robot system 1 according to the exemplary embodiment will be described. Figure 9 This is a flowchart illustrating an example of the operation of the robot system 1 according to an exemplary embodiment. First, if the user of the robot system 1 receives an input command to measure the sound of the robot 100's movement, the input device 800 sends a measurement execution command to the measurement controller 500 (step S101). At this time, the peripheral device 200 operates.

[0080] Next, the measurement controller 500 sends a detection execution command for detecting sound to the sound detector 400, and a prescribed action execution command for performing a prescribed measurement action to the robot controller 300 (step S102). The detection execution command may include information on the start time of sound detection, and the prescribed action execution command may include information on the start time of the prescribed measurement action. The measurement controller 500 may also determine two start times to synchronize sound detection with the prescribed measurement action.

[0081] The sound detector 400 begins sound detection (step S103).

[0082] The robot controller 300 causes the robot 100 to perform a specified measurement action (step S104).

[0083] The sound detector 400 continuously detects sound during the execution of the prescribed measurement action by the robot 100 and sends data indicating the detection result to the measurement controller 500. The measurement controller 500 stores and accumulates this data (step S105).

[0084] If robot 100 completes the specified measurement action, robot controller 300 stops robot 100 and sends the stop information of robot 100 to measurement controller 500 (step S106).

[0085] If the measurement controller 500 receives a stop message from the robot 100, it sends a command to the sound detector 400 to stop sound detection (step S107). The sound detector 400 stops sound detection.

[0086] The measurement controller 500 converts the data representing the detection result of the sound detector 400 to generate converted sound data, and sends the converted sound data and the instruction to process the converted sound data to detect the motion sound data of the robot 100 to the information processing device 600 (step S108).

[0087] The information processing device 600 converts the converted sound data to generate three-dimensional conversion data (step S109).

[0088] The information processing device 600 extracts two or more interval data from the three-dimensional conversion data (step S110).

[0089] The information processing device 600 calculates the envelope of the sound signals contained in all frequency bands of the interval data (step S111).

[0090] The information processing device 600 performs processing on the envelope of the same object frequency band that overlaps between interval data (step S112).

[0091] The information processing device 600 changes the intensity of the sound signal corresponding to the envelope to remove the difference of overlapping envelopes (step S113). As a result, the portion of the sound signal intensity affected by sound sources other than the robot 100, such as the peripheral device 200, is removed.

[0092] The information processing device 600 uses interval data of the object frequency band containing the intensity of the altered sound signal to detect motion sound data contained in the object frequency band (step S114).

[0093] The information processing device 600 converts the data set, which includes motion sound data of different frequency bands, to generate two-dimensional motion sound data (step S115).

[0094] The information processing device 600 sends the two-dimensional motion sound data to the measurement controller 500. The measurement controller 500 converts the two-dimensional motion sound data into data that can be sent to the output device 700, and sends the converted data to the output device 700. The output device 700 outputs the data corresponding to the two-dimensional motion sound data in a form that can be perceived by the user (step S116).

[0095] [Variation Example]

[0096] A variation of the robot system 1 according to the exemplary embodiment will be described. The robot system 1A of this variation differs from the embodiment in that it further determines motion sound data from the motion sound data detected by the embodiment. Hereinafter, this variation will be described focusing on the aspects that differ from the embodiment, while descriptions of aspects that are the same as the embodiment will be appropriately omitted.

[0097] Figure 10 This is a top view illustrating an example of the structure of a robot system 1A according to a variant of an exemplary embodiment. Robot system 1A differs from robot system 1 according to the embodiment in that it includes two or more sound detectors 400. In this variant, robot system 1A includes two sound detectors 400A and 400B. Sound detector 400A is disposed at a first position P1, and sound detector 400B is disposed at a second position P2, different from the first position P1. The first position P1 and the second position P2 are located at equidistant from robot 100. In this variant, the first position P1 and the second position P2 are located at equidistant from robot 100 in the horizontal direction, but they can also be located at equidistant in any direction in the three-dimensional direction. The distance between position P1 and robot 100 can be the distance L1 between position P1 and a reference point PS set on robot 100, and the distance between position P2 and robot 100 can be the distance L2 between position P2 and reference point PS. The reference point PS can also be located in a stationary part of robot 100 and fixed.

[0098] During the execution of a specified measurement action by the robot 100, the measurement controller 500 simultaneously causes the sound detectors 400A and 400B to detect sound. The measurement controller 500 receives the detection results from the sound detectors 400A and 400B respectively, processes and stores the data.

[0099] Figure 11 This is a block diagram illustrating an example of the functional structure of an information processing apparatus 600A according to a modified embodiment of an exemplary implementation. For example... Figure 11 As shown, the information processing device 600A of the robot system 1A includes the same functional structural elements as in the embodiment.

[0100] The conversion unit 500b of the measurement controller 500 receives a signal or data representing the detection result of the sound detector 400A from the sound detector 400A, and converts the signal or data to generate first converted sound data as converted sound data. The conversion unit 500b receives a signal or data representing the detection result of the sound detector 400B from the sound detector 400B, and converts the signal or data to generate second converted sound data as converted sound data. The conversion unit 500b associates the first converted sound data with the second converted sound data and sends them to the information processing device 600. For example, the conversion unit 500b may also associate the first converted sound data with the second converted sound data regarding the start time of sound detection and the start time of a predetermined measurement action. The first converted sound data is an example of first sound data, and the second converted sound data is an example of second sound data.

[0101] The first conversion unit 600Aa of the information processing device 600A converts the first converted sound data to generate first position three-dimensional conversion data as three-dimensional conversion data, and converts the second converted sound data to generate second position three-dimensional conversion data as three-dimensional conversion data.

[0102] The extraction unit 600Ab extracts two or more first position interval data as interval data from the first position three-dimensional transformation data, and extracts two or more second position interval data as interval data from the second position three-dimensional transformation data.

[0103] Similar to the embodiment, the detection unit 600Ac processes data contained in the same object frequency band within the first position interval data, thereby detecting motion sound data contained in that object frequency band. Similarly, the detection unit 600Ac processes data contained in the same object frequency band within the second position interval data, thereby detecting motion sound data contained in that object frequency band. The detection unit 600Ac performs motion sound data detection processing for all frequency bands of the first position interval data, and performs motion sound data detection processing for all frequency bands of the second position interval data.

[0104] Furthermore, the detection unit 600Ac selects two different frequency band pairs from the frequency bands in the first position interval data and the second position interval data, respectively. It then compares the time difference values ​​of the motion sound data contained in each frequency band pair between the first and second position interval data. Based on the comparison results, the detection unit 600Ac selects which motion sound data to discard. The detection unit 600Ac performs the same processing for other combinations of frequency band pairs. For example, the combinations of frequency band pairs processed by the detection unit 600Ac may include all combinations of frequency band pairs, or only a portion of them.

[0105] For example, the detection unit 600Ac selects pairs of distinct first and second frequency bands. The detection unit 600Ac detects first motion sound data contained in the first frequency band within the first position interval data, and detects second motion sound data contained in the second frequency band. The detection unit 600Ac detects third motion sound data contained in the first frequency band within the second position interval data, and detects fourth motion sound data contained in the second frequency band.

[0106] The first and third motion sound data contained in the first frequency band can be considered to originate from the same sound source. The second and fourth motion sound data contained in the second frequency band can be considered to originate from the same sound source.

[0107] The detection unit 600Ac compares the time difference between the first action sound data and the second action sound data (i.e., the first time difference) and the time difference between the third action sound data and the fourth action sound data (i.e., the second time difference). If the difference between the first time difference and the second time difference is greater than or equal to the threshold Th2, the first action sound data and the third action sound data, or the second action sound data and the fourth action sound data, are removed from the action sound data.

[0108] For example, in Figure 12 In the example shown, the first frequency band is frequency band FB8, and the second frequency band is frequency band FB6. Figure 12 This is a graph showing a comparison of the time differences in motion sound data. In Figure 12In this system, the waveforms of the sound signals are represented by an envelope, with a common time axis as the reference. The common time axis can be the elapsed time from a specific point in time, or it can be a single moment. For example, a specific point in time could be a point in time where a specific sound is generated, such as the start of a specified measurement action. The detection unit 600Ac calculates the deviation in the timing of the detection of the same action sound from the robot 100 by the sound detectors 400A and 400B based on the difference between distances L1 and L2; that is, the detection time difference. Based on the detection time difference, the detection unit 600Ac establishes a temporal association between the first position interval data SDA and the second position interval data SDB. In this modified example, distances L1 and L2 are the same, therefore the detection time difference is 0 or almost 0, and the time difference between a specific point in time of the first position interval data SDA and a specific point in time of the second position interval data SDB is 0 or almost 0.

[0109] like Figure 12 As shown, for example, in the first interval data DA1 of the first time interval T1 of the first position interval data SDA, the detection unit 600Ac detects the data of the sound signal that forms the waveform WA1 as the first motion sound data contained in the detection frequency band FB8, and detects the data of the sound signal that forms the waveform WA2 as the second motion sound data contained in the frequency band FB6.

[0110] The detection unit 600Ac detects the data of the sound signal that forms waveform WB1 in the first interval data DB1 of the first time interval T1 of the second position interval data SDB as the third motion sound data contained in the detection frequency band FB8, and detects the data of the sound signal that forms waveform WB2 as the fourth motion sound data contained in the frequency band FB6.

[0111] The detection unit 600Ac detects the time difference TD1 between waveforms WA1 and WA2 as the first time difference, and detects the time difference TD2 between waveforms WB1 and WB2 as the second time difference. The time difference between the two waveforms corresponds to the phase difference between them. For example, the time difference between the two waveforms can be the time difference between reference time points of the two waveforms. While the reference time point is not particularly limited, it can be, for example, a start time point, a time point corresponding to a pole or inflection point in the envelope, or a time point corresponding to a specified value of the tangent's inclination angle in the envelope. Figure 12 In the example, the reference time point is set as the start time point of the waveform. In this variation, the start time point, i.e., the start moment, of the first time interval T1 between the first position interval data SDA and the second position interval data SDB is the same moment.

[0112] like Figure 12As shown, the difference between time difference TD1 and time difference TD2 is greater than or equal to the threshold Th2. The detection unit 600Ac determines that the sound source of waveforms WA1 and WB1, or waveforms WA2 and WB2, is not robot 100. Threshold Th2 is an example of the first threshold.

[0113] For example, if the sound sources of waveforms WA1 and WB1, and waveforms WA2 and WB2 are both located on robot 100, then the upper limits of the time differences TD1 and TD2 are restricted. Therefore, the difference between time differences TD1 and TD2 can converge to less than the threshold Th2.

[0114] Furthermore, if the time difference between the first motion sound data and the third motion sound data, i.e., the third time difference, is greater than or equal to the second threshold, the detection unit 600Ac removes both the first and third motion sound data from the motion sound data. If the time difference between the second and fourth motion sound data, i.e., the fourth time difference, is greater than or equal to the second threshold, the detection unit 600Ac removes both the second and fourth motion sound data from the motion sound data.

[0115] For example, in Figure 12 In the example, the detection unit 600Ac detects the time difference TD3 between waveforms WA1 and WB1 as the third time difference, and detects the time difference TD4 between waveforms WA2 and WB2 as the fourth time difference. Figure 12 In the example, the time difference TD3 is 0 and less than the threshold Th3, while the time difference TD4 is greater than the threshold Th3. Detection unit 600Ac determines that the sound source of waveforms WA2 and WB2 is not robot 100. Threshold Th3 is an example of the second threshold.

[0116] The first position P1 and the second position P2 are located at equal distances from the robot 100. Therefore, for example, when the sound source of waveforms WA1 and WB1 is the robot 100, the time difference TD3 is less than the threshold Th3. When the sound source of waveforms WA2 and WB2 is the robot 100, the time difference TD4 is less than the threshold Th3.

[0117] The detection unit 600Ac can also determine the motion sound data to be removed by using one or both of the processing using a first time difference and a second time difference, and the processing using a third time difference and a fourth time difference. Furthermore, the detection unit 600Ac performs the same processing as described above for other combinations of frequency bands to remove motion sound data originating from devices other than the robot 100. For example, the combinations of frequency bands processed by the detection unit 600Ac can include all combinations of frequency bands, or only a portion of them. This enables high-precision detection of motion sound data.

[0118] In the above description, the distance L1 from the first position P1 to the robot 100 is the same as the distance L2 from the second position P2 to the robot 100, but they can also be different. In this case, the detection unit 600Ac can also use further information such as distances L1 and L2 to determine the motion sound data to be removed. For example, as Figure 13 As shown, the detection unit 600Ac detects the first interval data DB1, which corresponds to the first interval data DA1 of the first position interval data SDA, from the second position interval data SDB. Figure 13 This is a graph representing another comparative example of the time difference values ​​of motion sound data. In Figure 13 In this diagram, all waveforms are based on a common time axis and represented by an envelope. The common time axis can be the elapsed time from a specific point in time, or it can be a single instant.

[0119] The corresponding first interval data DA1 and first interval data DB1 include motion sound data of the robot 100 performing a specified measurement action at the same time. The detection unit 600Ac uses information from distances L1 and L2 to detect the detection time difference TA between the first position interval data SDA and the second position interval data SDB. The detection unit 600Ac can use the detection time difference TA to detect the first interval data DB1 corresponding to the first interval data DA1.

[0120] The detection unit 600Ac detects the time difference TD1 between waveforms WA1 and WA2 of the first interval data DA1 as the first time difference, and detects the time difference TD2 between waveforms WB1 and WB2 of the first interval data DB1 as the second time difference. The detection unit 600Ac detects the time difference TD3A obtained by subtracting the detection time difference TA from the time difference TD3 between waveforms WA1 and WB1 as the third time difference, and detects the time difference TD4A obtained by subtracting the detection time difference TA from the time difference TD4 between waveforms WA2 and WB2 as the fourth time difference. The detection unit 600Ac can use time differences TD1 and TD2 to perform the same processing as described above for the first and second time differences, and can use time differences TD3A and TD4A to perform the same processing as described above for the third and fourth time differences. Therefore, the detection unit 600Ac can select and discard motion sound data under different distances L1 and L2.

[0121] In this modified example, the information processing device 600A performs motion sound data selection processing based on the detection results of two sound detectors 400A and 400B located at different positions. However, it can also perform motion sound data selection processing based on the detection results of three or more sound detectors 400. In this case, the information processing device 600A can also select two sound detectors 400 from the three or more sound detectors 400 and perform motion sound data selection processing using motion sound data detected from the detection results of one sound detector 400 and motion sound data detected from the detection results of the other sound detector 400. The information processing device 600A can also perform motion sound data selection processing among motion sound data detected from the detection results of three or more sound detectors 400.

[0122] [Other implementation methods]

[0123] The exemplary embodiments and modifications of this disclosure have been described above, but this disclosure is not limited to the above-described embodiments and modifications. That is, various modifications and improvements can be made within the scope of this disclosure. For example, various modifications to the embodiments and modifications, and ways of constructing by combining different components of the embodiments and modifications, are also included within the scope of this disclosure.

[0124] For example, in the implementation and variations, the information processing device detects motion sound data to remove the difference between two envelopes of the same target frequency band between two data intervals, but is not limited to this. The information processing device may also detect motion sound data to remove the difference between multiple envelopes of the same target frequency band between three or more data intervals. The difference to be removed can be the difference between the envelopes of two selected data intervals from three or more data intervals, or it can be the difference between the envelopes of three or more data intervals. In the former case, the difference between the envelopes of a single combination of data intervals can be used, or the difference between the envelopes of multiple combinations of data intervals can be used. In the latter case, the difference can also be calculated using known statistical methods.

[0125] For example, in the embodiments and variations, the information processing device uses interval data extracted from a series of three-dimensional conversion data, including sounds containing two or more specified measurement actions, but is not limited to this. The information processing device may also convert multiple conversion sound data detected at different times, different dates, or other opportune moments to generate multiple three-dimensional conversion data. The information processing device 600 may also use multiple interval data extracted from multiple three-dimensional conversion data to detect action sound data.

[0126] Various embodiments of the technology disclosed herein can be listed below. One aspect of the information processing apparatus disclosed herein includes a processing circuit, wherein the processing circuit performs: a three-dimensional conversion process, converting two-dimensional sound data containing the motion sound of a robot performing a predetermined action twice or more, and expressing the sound signal in terms of intensity and generation time, to generate converted data as three-dimensional sound data expressing the sound signal in terms of frequency, intensity, and generation time; an extraction process, extracting two or more interval data from the converted data, including data from the time interval in which the predetermined action was performed; and a detection process, detecting motion sound data representing the robot's motion sound contained in the object frequency band based on a comparison of data contained in an object frequency band that is the same as the other two interval data.

[0127] According to the above method, in the converted data, for each frequency band, the relationship between the intensity of the sound signal and the generation time can be shown. The time intervals of the interval data can be different. Within the interval data, the motion sounds of the robot's prescribed actions contained in the same object frequency band have the same period. By comparing the data contained in the object frequency band between interval data, the information processing device can detect data with the same period, thereby detecting the robot's motion sound data. Thus, the information processing device can detect the robot's motion sound data from the sound data. Furthermore, by analyzing the motion sound data, it is also possible to detect the degree, type, and frequency band of the noise generated by the robot, as well as any abnormalities occurring in the robot.

[0128] In one aspect of the information processing apparatus disclosed herein, the processing circuit may also be configured to perform the following in the detection process: detecting first motion sound data as motion sound data based on the result of comparing data contained in a first target frequency band between the data intervals; and detecting second motion sound data as motion sound data based on the result of comparing data contained in a second target frequency band different from the first target frequency band between the data intervals. The processing circuit further performs a two-dimensional conversion process, which converts a data set including the first motion sound data and the second motion sound data to generate two-dimensional motion sound data as two-dimensional sound data representing sound signals in terms of intensity and generation time.

[0129] According to the above method, the robot's motion sound may be contained in multiple frequency bands. The information processing device can detect the robot's motion sound data contained in multiple frequency bands. Furthermore, the information processing device centrally converts a set of robot motion sound data contained in multiple frequency bands into two-dimensional data, thereby generating two-dimensional motion sound data as ordinary sound data. Such two-dimensional motion sound data can be used to reproduce the robot's motion sound. Moreover, by analyzing the two-dimensional motion sound data, it is also possible to detect the degree and type of noise generated by the robot, as well as anomalies arising from the robot.

[0130] In one aspect of the information processing apparatus disclosed herein, the processing circuit may also be configured to perform the following in the detection process: determining an envelope representing the relationship between the intensity of the sound signal contained in the target frequency band of the interval data and the generation time; and detecting the motion sound data based on the result of comparing the envelope of the target frequency band between the interval data.

[0131] According to the above method, by using an envelope, it becomes easier to compare data contained in the object frequency band between data intervals. For example, it is possible to compare data with reduced effects caused by the short-period waveform of the sound signal. As a result, it is possible to reduce the processing load of the information processing device and improve the accuracy of the processing results.

[0132] In one aspect of the information processing apparatus disclosed herein, the processing circuit may also be configured such that the intensity of the sound signal corresponding to the envelope is changed by removing the difference between the envelopes, and the interval data of the target frequency band, including the intensity of the changed sound signal, is used to detect the motion sound data.

[0133] According to the above method, the difference between the envelopes may be caused by sound sources other than the robot. In motion sound data detected using sound signals modified to remove the difference between the envelopes, the content of signals caused by sound sources other than the robot is reduced. Therefore, the information processing device can detect motion sound data with high accuracy.

[0134] In one embodiment of the information processing apparatus disclosed herein, the processing circuit may also be configured such that, when removing the difference, it changes the intensity of the sound signal corresponding to the envelope with a greater intensity to an envelope with an intensity closer to that of the sound signal. According to this method, signals caused by sound sources other than the robot can be reliably removed from motion sound data.

[0135] In one aspect of the information processing apparatus disclosed herein, the processing circuit may also be configured to perform the aforementioned three-dimensional conversion processing, the aforementioned extraction processing, and the aforementioned detection processing on two-dimensional sound data, namely first sound data and second sound data, which are detected at different first and second positions and include the motion sound of the robot performing the aforementioned predetermined action more than twice. In the aforementioned detection processing, the processing circuit performs: detecting first motion sound data as the motion sound data based on the result of comparing data contained in a first object frequency band between the interval data of the first sound data; detecting second motion sound data as the motion sound data based on the result of comparing data contained in a second object frequency band between the interval data of the first sound data; and detecting second motion sound data based on the result of comparing data contained in a second object frequency band between the interval data of the first sound data. Based on the comparison of the data contained in the first object frequency band between the aforementioned interval data of the data, third motion sound data is detected as the motion sound data; based on the comparison of the data contained in the second object frequency band between the aforementioned interval data of the second sound data, fourth motion sound data is detected as the motion sound data; the time difference between the first motion sound data and the second motion sound data, i.e., the first time difference, and the time difference between the third motion sound data and the fourth motion sound data, i.e., the second time difference, are compared; and if the difference between the first time difference and the second time difference is greater than or equal to a first threshold, the first motion sound data and the third motion sound data, or the second motion sound data and the fourth motion sound data, are removed from the motion sound data.

[0136] According to the above method, the sound source of the first motion sound data and the sound source of the third motion sound data can be the same. The sound source of the second motion sound data and the sound source of the fourth motion sound data can be the same. When the sound sources of the first and third motion sound data, as well as the sound sources of the second and fourth motion sound data, are all robots, the difference between the first time difference and the second time difference is small, less than the first threshold. However, when the difference is greater than the first threshold, it is very likely that one of the sound sources is not a robot. By removing motion sound data from such sound sources, the information processing device can detect robot motion sound data with high accuracy.

[0137] In one aspect of the information processing apparatus disclosed herein, it may also be configured such that: when the time difference between the first action sound data and the third action sound data, i.e., the third time difference, is a second threshold or higher, the processing circuit removes the first action sound data and the third action sound data from the action sound data; and when the time difference between the second action sound data and the fourth action sound data, i.e., the fourth time difference, is a second threshold or higher, the processing circuit removes the second action sound data and the fourth action sound data from the action sound data.

[0138] According to the above method, when both the sound source of the first and third motion sound data is a robot, the third time difference is very small, less than the second threshold. When both the sound source of the second and fourth motion sound data is a robot, the fourth time difference is very small, less than the second threshold. The sound sources of two motion sound data sets with time differences exceeding the second threshold are very likely not robots. By removing motion sound data from such sources, the information processing device can detect high-precision robot motion sound data.

[0139] In one embodiment of the information processing apparatus disclosed herein, the first position and the second position may be configured such that they are located at equal distances from the robot. According to this configuration, it becomes easier to detect motion sound data where the difference between the first time difference and the second time difference is greater than or equal to a first threshold, and the robot is not the sound source. This allows for a reduction in the processing load of the information processing apparatus and an improvement in the accuracy of the processing results.

[0140] One aspect of the measurement system disclosed herein includes: an information processing device according to one aspect of the present disclosure; and a sound detector for detecting the motion sound of the robot, wherein the information processing device detects the motion sound data from sound data representing the sound signal detected by the sound detector.

[0141] According to the above method, the measurement system can achieve the same effect as the information processing device involved in one aspect of this disclosure. The measurement system can detect sound and can detect the robot's motion sound data from sound data representing the detected sound signals. The measurement system can perform both sound detection and motion sound data detection.

[0142] A robot system according to one aspect of this disclosure includes: an information processing device according to one aspect of this disclosure; the robot; a sound detector for detecting the movement sound of the robot; and a controller for controlling the robot to perform the predetermined action and, during the execution of the predetermined action, causing the sound detector to perform sound detection, wherein the information processing device detects the movement sound data from sound data representing the sound signal detected by the sound detector.

[0143] According to the above method, the robot system can achieve the same effect as the information processing device and the measurement system involved in one aspect of this disclosure. The robot system can enable the robot to perform a predetermined action, detect the robot's motion sound during the execution of the predetermined action, and detect the robot's motion sound data from the sound data representing the detected sound signals. The robot system can perform the execution of predetermined actions based on the robot, sound detection, and motion sound data detection.

[0144] One aspect of this disclosure relates to a data processing method comprising: converting two-dimensional sound data containing the motion sounds of a robot performing a prescribed action more than twice, and representing sound signals in terms of intensity and generation time, to generate converted data as three-dimensional sound data representing sound signals in terms of frequency, intensity, and generation time; extracting two or more interval data from the converted data, including data containing time intervals in which the prescribed actions were performed; and detecting motion sound data representing the motion sounds of the robot contained in the object frequency band based on the result of comparing data contained in an object frequency band that is the same as each other between the interval data.

[0145] According to the above method, this method can achieve the same effect as the information processing apparatus involved in one aspect of this disclosure. This method can be implemented, for example, by a CPU, LSI circuits, an IC card, or a single module.

[0146] One aspect of this disclosure involves a computer program that causes a computer to: convert two-dimensional sound data containing the motion sounds of a robot performing a prescribed action more than twice, and representing sound signals in terms of intensity and generation time, to generate converted data as three-dimensional sound data representing sound signals in terms of frequency, intensity, and generation time; extract two or more interval data from the converted data, including data containing the time interval of the prescribed action; and detect motion sound data representing the motion sounds of the robot contained in the object frequency band based on the result of comparing data contained in an object frequency band that is the same as each other between the interval data.

[0147] According to the above method, the computer program can achieve the same effect as the information processing apparatus involved in one aspect of this disclosure. The computer program may be, for example, a program recorded on a non-transitory computer-readable recording medium, or it may be configured to be read from the recording medium and installed on the computer using a driver device for the recording medium. The computer program may be, for example, a program that can be transmitted via a transmission medium such as the Internet, or it may be configured to be downloaded and installed on the computer.

[0148] The functions of the elements disclosed in this specification can be executed using circuits or processing circuits, including general-purpose processors, special-purpose processors, integrated circuits, ASICs, existing circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuitry. In this disclosure, a circuit, unit, or mechanism is hardware that performs the listed functions, or hardware programmed to perform the listed functions. The hardware can be the hardware disclosed in this specification, or it can be other known hardware programmed or configured to perform the listed functions. Where the hardware is a processor considered a type of circuit, the circuit, mechanism, or unit is a combination of hardware and software, with the software used in the structure of the hardware and / or processor.

[0149] The serial numbers, quantities, and other figures used in this specification are illustrative for the purpose of specifically illustrating the technology of this disclosure, and this disclosure is not limited to the illustrative figures. The connecting relationships between the constituent elements are illustrative for the purpose of specifically illustrating the technology of this disclosure, and the connecting relationships for realizing the functions of this disclosure are not limited thereto.

[0150] The scope of this disclosure is defined by the appended claims rather than by the description, so that this disclosure can be practiced in various forms without departing from its essential spirit. Therefore, illustrative embodiments and variations are exemplary and not limiting. All modifications intended within the scope of the claims, or equivalents thereof, are included in the claims.

Claims

1. An information processing device comprising a processing circuit, characterized in that, The processing circuit performs: The three-dimensional conversion processing converts two-dimensional sound data containing the motion sounds of a robot performing a specified action more than twice, and the sound signals are expressed in terms of intensity and generation time, to generate converted data as three-dimensional sound data in terms of frequency, intensity and generation time. Extraction processing: Extracting two or more interval data from the transformed data, including data within the time interval in which the specified action was performed; as well as The detection process, based on the result of comparing data contained in object frequency bands that are the same frequency band as each other within the data intervals, detects motion sound data representing the robot's motion sounds contained in the object frequency bands. The processing circuit performs the following in the detection process: The envelope that determines the relationship between the intensity of the sound signal contained in the object frequency band of the interval data and the generation time; and Based on the result of comparing the envelope of the target frequency band between the interval data, the motion sound data is detected, that is, the intensity of the sound signal corresponding to the envelope is changed in a way that removes the difference between the envelopes, and the motion sound data is detected using the interval data of the target frequency band that includes the intensity of the changed sound signal.

2. The information processing device according to claim 1, characterized in that, The processing circuit performs the following in the detection process: Based on the result of comparing the data contained in the first object frequency band between the data intervals, the first motion sound data as the motion sound data is detected; and Based on the comparison of data contained in a second object frequency band that is different from the first object frequency band within the data interval, second motion sound data, which is the motion sound data, is detected. The processing circuit also performs a two-dimensional conversion process, which converts the data set including the first motion sound data and the second motion sound data to generate two-dimensional motion sound data as two-dimensional sound data representing sound signals in terms of intensity and generation time.

3. The information processing device according to claim 1, characterized in that, When removing the difference, the processing circuit changes the intensity of the sound signal corresponding to the envelope with a larger intensity to the envelope with a smaller intensity, which is closer to the intensity of the sound signal.

4. The information processing apparatus according to any one of claims 1 to 3, characterized in that, The processing circuit performs the three-dimensional conversion processing, the extraction processing, and the detection processing on the two-dimensional sound data, namely the first sound data and the second sound data, which are detected at different first and second positions and contain the robot's motion sounds that have been performed more than twice. The processing circuit performs the following in the detection process: Based on the result of comparing the data contained in the first object frequency band between the interval data of the first sound data, the first motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the second object frequency band between the interval data of the first sound data, the second motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the first object frequency band between the interval data of the second sound data, third motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the second object frequency band between the interval data of the second sound data, fourth motion sound data as the motion sound data is detected; The time difference between the first action sound data and the second action sound data, i.e., the first time difference, and the time difference between the third action sound data and the fourth action sound data, i.e., the second time difference, are compared. as well as If the difference between the first time difference and the second time difference is greater than or equal to a first threshold, the first action sound data and the third action sound data, or the second action sound data and the fourth action sound data, are removed from the action sound data.

5. The information processing apparatus according to claim 4, characterized in that, If the time difference between the first action sound data and the third action sound data, i.e., the third time difference value, is greater than or equal to a second threshold, the processing circuit removes both the first action sound data and the third action sound data from the action sound data. If the time difference between the second action sound data and the fourth action sound data, i.e., the fourth time difference, is greater than or equal to the second threshold, the processing circuit removes the second action sound data and the fourth action sound data from the action sound data.

6. The information processing apparatus according to claim 4, characterized in that, The first position and the second position are located at equal distances from the robot.

7. An information processing device comprising a processing circuit, characterized in that, The processing circuit performs: The three-dimensional conversion processing converts two-dimensional sound data containing the motion sounds of a robot performing a specified action more than twice, and the sound signals are expressed in terms of intensity and generation time, to generate converted data as three-dimensional sound data in terms of frequency, intensity and generation time. Extraction processing: Extracting two or more interval data from the transformed data, including data within the time interval in which the specified action was performed; as well as The detection process, based on the result of comparing data contained in object frequency bands that are the same frequency band as each other within the data intervals, detects motion sound data representing the robot's motion sounds contained in the object frequency bands. The processing circuit performs the three-dimensional conversion processing, the extraction processing, and the detection processing on the two-dimensional sound data, namely the first sound data and the second sound data, which are detected at different first and second positions and contain the robot's motion sounds that have been performed more than twice. The processing circuit performs the following in the detection process: Based on the result of comparing the data contained in the first object frequency band between the interval data of the first sound data, the first motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the second object frequency band between the interval data of the first sound data, the second motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the first object frequency band between the interval data of the second sound data, third motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the second object frequency band between the interval data of the second sound data, fourth motion sound data as the motion sound data is detected; The time difference between the first action sound data and the second action sound data, i.e., the first time difference, and the time difference between the third action sound data and the fourth action sound data, i.e., the second time difference, are compared. If the difference between the first time difference and the second time difference is greater than or equal to a first threshold, the first action sound data and the third action sound data, or the second action sound data and the fourth action sound data, are removed from the action sound data. If the time difference between the first action sound data and the third action sound data, i.e., the third time difference value is greater than or equal to the second threshold, the processing circuit removes the first action sound data and the third action sound data from the action sound data. as well as If the time difference between the second action sound data and the fourth action sound data, i.e., the fourth time difference, is greater than or equal to the second threshold, the processing circuit removes the second action sound data and the fourth action sound data from the action sound data.

8. The information processing apparatus according to claim 7, characterized in that, The first position and the second position are located at equal distances from the robot.

9. A measurement system, characterized in that, The measurement system includes: The information processing apparatus according to any one of claims 1 to 8; and A sound detector detects the sounds of the robot's movements. The information processing device detects the motion sound data from sound data representing sound signals detected by the sound detector.

10. A robot system, characterized in that, The robot system has the following features: The information processing apparatus according to any one of claims 1 to 8; The robot; A sound detector detects the sounds of the robot's movements; as well as The controller performs control to cause the robot to execute the prescribed action, and during the execution of the prescribed action, the sound detector performs sound detection. The information processing device detects the motion sound data from sound data representing sound signals detected by the sound detector.

11. A method, a method for processing data, characterized in that, The method includes: Two-dimensional sound data, which contains the sound of a robot performing a specified action more than twice and is expressed in terms of intensity and generation time, is converted to generate three-dimensional sound data, which is expressed in terms of frequency, intensity and generation time. Extract two or more interval data from the transformed data, including data from the time interval in which the specified action was performed; and Based on the comparison of data contained in object frequency bands that are the same frequency bands among the data in the intervals, motion sound data representing the robot's motion sound contained in the object frequency band is detected. The processing of the motion sound data includes: The envelope that determines the relationship between the intensity of the sound signal contained in the object frequency band of the said interval data and the time of generation; and Based on the result of comparing the envelope of the target frequency band between the interval data, the motion sound data is detected, that is, the intensity of the sound signal corresponding to the envelope is changed in a way that removes the difference between the envelopes, and the motion sound data is detected using the interval data of the target frequency band that includes the intensity of the changed sound signal.

12. A method, a method for processing data, characterized in that, The method includes: Two-dimensional sound data, which contains the sound of a robot performing a specified action more than twice and is expressed in terms of intensity and generation time, is converted to generate three-dimensional sound data, which is expressed in terms of frequency, intensity and generation time. Extract two or more interval data from the transformed data, including data from the time interval in which the specified action was performed; and Based on the comparison of data contained in object frequency bands that are the same frequency bands among the data in the intervals, motion sound data representing the robot's motion sound contained in the object frequency band is detected. For two-dimensional sound data, namely first sound data and second sound data, which are detected at different first and second positions and include the robot's motion sounds that have been performed more than twice, the following processes are performed: generating the converted data, extracting the interval data, and detecting the motion sound data. The processing of the motion sound data includes: Based on the result of comparing the data contained in the first object frequency band between the interval data of the first sound data, the first motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the second object frequency band between the interval data of the first sound data, the second motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the first object frequency band between the interval data of the second sound data, third motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the second object frequency band between the interval data of the second sound data, fourth motion sound data as the motion sound data is detected; The time difference between the first action sound data and the second action sound data, i.e., the first time difference, and the time difference between the third action sound data and the fourth action sound data, i.e., the second time difference, are compared. If the difference between the first time difference and the second time difference is greater than or equal to a first threshold, the first action sound data and the third action sound data, or the second action sound data and the fourth action sound data, are removed from the action sound data. If the time difference between the first motion sound data and the third motion sound data, i.e., the third time difference value, is greater than or equal to a second threshold, the processing circuit removes both the first motion sound data and the third motion sound data from the motion sound data; and If the time difference between the second action sound data and the fourth action sound data, i.e., the fourth time difference, is greater than or equal to the second threshold, the processing circuit removes the second action sound data and the fourth action sound data from the action sound data.

13. A computer-readable storage medium storing a computer program, characterized in that, The computer program causes the computer to execute: Two-dimensional sound data, which contains the sound of a robot performing a specified action more than twice and is expressed in terms of intensity and generation time, is converted to generate three-dimensional sound data, which is expressed in terms of frequency, intensity and generation time. Extract two or more interval data from the transformed data, including data from the time interval in which the specified action was performed; as well as Based on the comparison of data contained in object frequency bands that are the same frequency bands among the data in the intervals, motion sound data representing the robot's motion sound contained in the object frequency band is detected. In the processing of the motion sound data, the computer performs the following: The envelope that determines the relationship between the intensity of the sound signal contained in the object frequency band of the interval data and the generation time; and Based on the result of comparing the envelope of the target frequency band between the interval data, the motion sound data is detected, that is, the intensity of the sound signal corresponding to the envelope is changed in a way that removes the difference between the envelopes, and the motion sound data is detected using the interval data of the target frequency band that includes the intensity of the changed sound signal.

14. A computer-readable storage medium storing a computer program, characterized in that, The computer program causes the computer to execute: Two-dimensional sound data, which contains the sound of a robot performing a specified action more than twice and is expressed in terms of intensity and generation time, is converted to generate three-dimensional sound data, which is expressed in terms of frequency, intensity and generation time. Extract two or more interval data from the transformed data, including data from the time interval in which the specified action was performed; as well as Based on the comparison of data contained in object frequency bands that are the same frequency bands among the data in the intervals, motion sound data representing the robot's motion sound contained in the object frequency band is detected. The computer performs the following processes on two-dimensional sound data—namely, first sound data and second sound data—that are detected at different first and second positions and contain the robot's motion sounds that have been performed more than twice: generating the converted data, extracting the interval data, and detecting the motion sound data. In the processing of the motion sound data, the computer performs the following: Based on the result of comparing the data contained in the first object frequency band between the interval data of the first sound data, the first motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the second object frequency band between the interval data of the first sound data, the second motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the first object frequency band between the interval data of the second sound data, third motion sound data as the motion sound data is detected; Based on the result of comparing the data contained in the second object frequency band between the interval data of the second sound data, fourth motion sound data as the motion sound data is detected; The time difference between the first action sound data and the second action sound data, i.e., the first time difference, and the time difference between the third action sound data and the fourth action sound data, i.e., the second time difference, are compared. If the difference between the first time difference and the second time difference is greater than or equal to a first threshold, the first action sound data and the third action sound data, or the second action sound data and the fourth action sound data, are removed from the action sound data. If the time difference between the first action sound data and the third action sound data, i.e., the third time difference value is greater than or equal to the second threshold, the processing circuit removes the first action sound data and the third action sound data from the action sound data. as well as If the time difference between the second action sound data and the fourth action sound data, i.e., the fourth time difference, is greater than or equal to the second threshold, the processing circuit removes the second action sound data and the fourth action sound data from the action sound data.

Images