Object movement state detection device and object movement state detection method

Abstract

The timing of a change in the movement path is detected based on a large change in a detection signal from a sensor (12) attached to the object (10). Then, merged interval data composed of sensor data at the timing of the change, i.e., change timing data, and subsequent timing data continuing thereafter, is extracted, and compared with collation data of a database (20g), thereby detecting the "movement path" and the "movement pattern". Further, segmented data after the merged interval data is extracted, and the merged interval data and the segmented data are compared with a learning model, thereby detecting the movement path state of the object (10).

Description

Cross-references to related applications

[0001] This application is based on Japanese Patent Application No. 2024-37305, filed on March 11, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to an object movement state detection device and a method for detecting the movement state of an object equipped with a sensor. Background Technology

[0003] Previously, Patent Document 1 disclosed a device for detecting anomalies in equipment that transports objects by installing sensors on them. This device causes the sensors mounted on the object to transmit sensor data, and multiple devices are equipped with a shared sensor node. Sensor data transmitted from a communication unit is received by a receiver. Then, sensor data from normal operation is pre-learned as training data. By comparing the received sensor data with the training data, anomalies or signs of anomalies in multiple devices are detected. For example, when a moving transport path is divided into multiple sections, the normal transport path data corresponding to each section is stored as training data. By comparing the detected sensor data with the training data, the position of the transport path is determined.

[0004] Existing technical documents Patent documents Patent Document 1: Japanese Patent Application Publication No. 2023-007350 Summary of the Invention The aforementioned conventional devices are effective when the learning data path and sensing time are the same. However, in cases such as fault diagnosis, due to the nature of fault diagnosis, sudden stops or malfunctions, impacts caused by foreign objects attached to the transport path, etc., will also be reflected in the time series data, resulting in significant time deviations even in identical data. Therefore, the following technical problem exists: if repeatedly acquired time series data are superimposed, machine learning will be performed based on a biased and low-quality data set, making it impossible to obtain high-precision learning data, resulting in a decrease in anomaly detection accuracy.

[0005] The purpose of this disclosure is to provide an object movement state detection device and method that can detect the movement state of an object with higher accuracy.

[0006] The first aspect of this disclosure is an object movement state detection device that detects the object's movement state by acquiring sensor data showing the sensing result of the sensor from an object equipped with a sensor that outputs a detection signal corresponding to the movement state. The device comprises: The data acquisition unit acquires the sensor data; The timing of change detection unit detects the timing of changes in the movement mode based on the sensor data acquired by the data acquisition unit. The interval data generation unit takes the detected change timing and a portion of the interval from the change timing when the movement mode is maintained as a merged interval, and generates merged interval data, which is the time series data of the sensor data in the merged interval; The database stores verification data, namely training data used to identify the movement state, movement pattern, and movement path of the object. The identification unit identifies the movement pattern and the movement path by comparing the merged interval data with the verification data. The segmented data generation unit generates multiple segmented data from the sensor data after the merged interval when the movement mode is maintained; The learning model storage unit stores a learning model obtained by performing machine learning on the merged interval data and the segmented data corresponding to the movement mode and the movement path; The label assignment unit assigns labels to the merged interval data generated by the interval data generation unit and the segmented data generated by the segmented data generation unit, corresponding to the movement pattern and movement path identified by the identification unit; The diagnostic unit detects the movement path status of the object by comparing the merged interval data and the segmented data with the learning model corresponding to the label stored in the learning model storage unit.

[0007] This method detects the timing of changes in the movement path during movement mode switching and generates merged interval data containing these changes. By comparing the merged interval data with verification data, the type of "movement path" or "movement mode" can be identified. Simultaneously, by comparing the merged interval data with the learned model, the movement path state at the timing of the movement mode change can be detected. Furthermore, segmented data is generated by extracting portions after the merged interval from the sensor data at arbitrary time intervals. By comparing each segmented data with the learned model, the movement path state when the movement mode is maintained can be detected.

[0008] Then, after identifying the type of "movement path" or "movement pattern" at the point of change in the movement pattern, the merged interval data or segmented data is extracted as time series data through interval segmentation. Therefore, machine learning can be performed at times not anticipated based on the type of "movement path" or "movement pattern" (e.g., the time when the object's movement completely stops). Furthermore, by generating segmented data under stable conditions where the movement pattern is maintained, temporal biases in the time series data can be suppressed. Therefore, the accuracy of machine learning can be improved, thereby improving the accuracy of movement path state detection.

[0009] The second aspect of this disclosure is a method for detecting the movement state of an object, which acquires sensor data showing the sensing results of the sensor from an object equipped with a sensor that outputs a detection signal corresponding to the movement state, thereby detecting the movement state of the object. The method includes: The steps for acquiring the sensor data; The step of detecting the timing of changes in the movement pattern based on the acquired sensor data; The step of generating merged interval data by taking the detected timing of the change and a portion of the interval in which the movement mode is maintained from the timing of the change as a merged interval, wherein the merged interval data is the time series data of the sensor data in the merged interval; The step of using check data, i.e. training data used to identify the movement state, i.e., movement pattern and movement path of the object, to compare the merged interval data with the check data, thereby identifying the movement pattern and the movement path; The step of generating multiple segmented data from the sensor data after the merged interval, when the movement mode is maintained; The step of assigning labels to the merged interval data generated by the interval data generation unit and the segmented data generated by the segmented data generation unit, corresponding to the movement pattern and movement path identified by the identification unit; The step of using a learning model obtained by machine learning on the merged interval data and the segmented data corresponding to the movement pattern and the movement path, and comparing the merged interval data and the segmented data with the learning model corresponding to the label in the learning model, to detect the movement path state of the object.

[0010] Thus, this disclosure can also be understood as a method for detecting the motion state of an object. Even in this case, the same effect as the first aspect of this disclosure can be achieved. Attached Figure Description

[0011] [Figure 1 [Illustration] is a block diagram showing the object movement state detection system according to the first embodiment of this disclosure.

[0012] [ Figure 2 This is an example illustrating how an object can move without its own power source.

[0013] [ Figure 3 The image is an example illustrating a situation where an object has power and its path of movement is not determined.

[0014] [ Figure 4 [This is a flowchart of the interval segmentation process at the point where the movement mode changes.]

[0015] [ Figure 5 This is a flowchart of the segmented interval segmentation process in the maintain state of the moving mode.

[0016] [ Figure 6 [This is a diagram showing the pattern when the movement path RA changes to the movement path RB.]

[0017] [ Figure 7 [ ] is a timing diagram showing the changes in the detection signals of the microphone and vibration sensor as the object moves along the movement path RA and the movement path RB.

[0018] [ Figure 8 The figure shows the changes in the detection signal and feature quantity of the Z-axis when the movement path changes from RA to RB, as well as the situation of extracting and merging interval data and segmented data.

[0019] [ Figure 9 [ ] is a diagram showing a variation of the segmentation method for data.

[0020] [ Figure 10 [ ] is a timing diagram showing sensor data during normal operation and during a fault.

[0021] [ Figure 11 [Illustration] is a block diagram illustrating the object movement state detection system described in other embodiments. Detailed Implementation

[0022] Hereinafter, embodiments of the present disclosure will be described based on the accompanying drawings. Furthermore, in other embodiments, including those described below, the same or equivalent parts will be given the same reference numerals and described accordingly.

[0023] (First Embodiment) The first embodiment will be described. The object movement state detection system of this embodiment is a system that detects the movement path state of an object; specifically, it performs fault diagnosis, displays the detection results, and thus communicates to the operator whether the object's movement path is normal or abnormal. Here, "movement path" refers to the route the object takes. For example, when moving an object from a starting point to an end point, the movement path refers to the trajectory of the object from the starting point to the end point, or the transport path of the device that moves the object along its trajectory.

[0024] like Figure 1 As shown, the object movement state detection system comprises an object 10, a state detection device 20, and a display device 30. The system is configured such that sensors 12 within sensor nodes 11 mounted on the object 10 output sensing results indicating the movement path state of the object 10. The state detection device 20 compares these results with a learned model for fault diagnosis, and displays the results on the display device 30. Based on this, the operator can maintain the equipment in a good condition by controlling, repairing, and replacing parts of the equipment constituting the movement path.

[0025] The following describes the components of an object movement detection system.

[0026] [Structure of object 10] Object 10 is an object moving along a movement path that is being detected, and it is equipped with a sensor node 11 having a sensor 12. Object 10 can be an actual product or a monitoring object prepared for detecting the state of the movement path.

[0027] The object 10 can exist in either the case of being unpowered or powered, but either case is acceptable. For example, an example of the object 10 being unpowered is the case where the object 10 is moved by the movement of a device that forms a movement path in contact with the object 10. Movement by grasping a part of the object 10 or the surface in contact with the object 10 is also considered a case where the object 10 is unpowered.

[0028] Methods of moving object 10 when it lacks power include, for example, suspension, lifting, and placement. Figure 2The diagram illustrates the movement paths for switching from a suspended type to a lifting type, and then to a placement type. The suspended type involves suspending the object 10 below the first movement path 41 and moving it. The lifting type uses a second movement path 42 that is pulled vertically to move the object 10 from an upper position to a lower position or vice versa. The placement type involves placing the object 10 on a third movement path 43 for placement and moving it. Examples of suspended types include wafer handling systems (SMIF, Standard Mechanical Interface) and ceiling-mounted conveyors. Examples of lifting types include wafer handling systems (SMIF) and elevators. Examples of placement types include automated guided vehicles (AGVs), belt conveyors, roller conveyors, screw conveyors, and electric transport vehicles.

[0029] On the other hand, when object 10 is powered, examples can be given of objects that can move autonomously, such as self-propelled mobile bodies. For example, objects 10 moving on their own along guide rails or moving autonomously based on program control fall under the category of objects 10 being powered. Furthermore, if object 10 is a mobile robot such as a food preparation robot or a cleaning robot, and that robot is powered, then this also falls under this category.

[0030] The movement path of object 10 can be either a predetermined path or an undetermined path. When object 10 is not powered, the movement path of each device is determined. Furthermore, when object 10 is powered, its movement path is also determined, whether it is along a path with guide rails or a predetermined path controlled by a program. As for cases where the movement path is not determined, for example... Figure 3 As shown, an example of a program can be given that, when planning a movement path after determining the starting point P1 and ending point P2 of the object 10's movement, if an obstacle 50 exists along the way, a path 44 that avoids the obstacle 50 can be selected. For example, a mobile robot can be considered as an example where the movement path is not determined.

[0031] [Structure of sensor node 11] Sensor node 11 is mounted on object 10 moving along a path for which state detection is desired, and outputs sensing results showing the state of the movement path. Sensor node 11 is equipped with at least one sensor 12, a power supply unit 13, and a communication unit 14.

[0032] Sensor 12 detects one or more of the following: sound, vibration, acceleration, angular velocity, etc. Sensor 12 can be any type of sensor, such as a semiconductor sensor, and can output the detection signal showing the sensing result or the data after signal processing as sensor data to the communication unit 14. The sensor data output by sensor 12 is equivalent to data representing the physical quantity used for state detection of the movement path.

[0033] Regarding sensor 12, a sensor corresponding to the aforementioned detection object is applicable. For example, in this embodiment, a composite sensor is used, comprising a microphone 12a for detecting sound, a vibration sensor 12b for detecting vibration or acceleration, and an angular velocity sensor 12c for detecting angular velocity. Sensor 12 can be a variety of sensors with different detection objects, or it can include multiple sensors with the same detection object. Even if multiple sensors 12 detect the same object, sensor data corresponding to that direction or position can be obtained if the detection direction or position is different.

[0034] Regarding microphone 12a, only one can be used, or multiple microphones can be used to determine the location of the sound source. Microphone 12a is used to sense the sound generated when object 10 moves along the moving path, and the sensing result is output as sensor data.

[0035] Regarding the vibration sensor 12b, it senses the vibration or acceleration applied to the object 10 as it moves along the movement path, and outputs the sensing result as sensor data. The vibration sensor 12b can be, for example, a triaxial accelerometer that detects the vibration or acceleration along the mutually orthogonal X-axis, Y-axis, and Z-axis respectively. For example, depending on the swaying state of the movement path, vibration or acceleration may occur on the object 10, and this can be detected by the vibration sensor 12b. Furthermore, in the following description, a horizontal direction and a direction orthogonal to it will be referred to as the X-axis and Y-axis, and the vertical direction as the Z-axis.

[0036] Regarding the angular velocity sensor 12c, the rotation and orientation changes of the object 10 are sensed as angular velocities, and the sensing results are output as sensor data. When the object 10 is positioned on a movement path, the angular velocity sensor 12c can detect changes in the movement path and the tilt or attitude of the object 10. The angular velocity sensor 12c can be at least one of the following: a single-axis angular velocity sensor that detects rotation in one direction, a dual-axis angular velocity sensor that detects rotation in two mutually orthogonal directions, or a triaxial angular velocity sensor that detects rotation in three mutually orthogonal directions. Here, the angular velocity sensor 12c uses a triaxial accelerometer to detect the angular velocities of the object 10 when it is positioned on a horizontal plane: yaw angle about the vertical axis, roll angle about the longitudinal axis, and pitch angle about the left-right axis.

[0037] The power supply unit 13 is the power source for the various components equipped within the sensor node 11, and is composed of, for example, a button battery or a lithium battery. Alternatively, the power supply unit 13 may be configured using vibration power generation, wireless power supply, solar power generation, or other similar methods.

[0038] The communication unit 14 transmits the sensor data from the sensor 12 to the status detection device 20 via wireless communication or the like. The communication unit 14 selects the frequency band to be used for transmission based on factors such as communication speed, communication distance, and the most suitable frequency diffraction characteristics.

[0039] Furthermore, as sensor data transmitted from sensor 12 to communication unit 14, the sensor data detected by sensor 12 can be transmitted as is, that is, communication unit 14 can use the raw value of the received sensor data. However, communication unit 14 can also process the sensor data through signal processing and then send it to state detection device 20.

[0040] [Structure of the condition detection device 20] The state detection device 20 is equivalent to an object movement state detection device, constituting the control unit of the object movement state detection system. The state detection device 20 consists of a microcomputer equipped with a CPU, ROM, RAM, I / O, etc., and detects the movement state of the object 10 based on sensor data transmitted from the sensor node 11. The state detection device 20 includes a data acquisition unit 20a, a first feature quantity calculation unit 20b, a change timing detection unit 20c, an interval data generation unit 20d, a segmented data generation unit 20e, an identification unit 20f, a database 20g, a tag assignment unit 20h, a tag and serial number assignment unit 20i, a second feature quantity calculation unit 20j, a diagnostic unit 20k, and a learning model storage unit 20m. Furthermore, in this embodiment, although the state detection device 20 is a separate device from the sensor node 11, thus miniaturizing the object 10 equipped with the sensor node 11, the state detection device 20 can also be built into the object 10, thereby forming a communication-free structure. If this is done, a structure that eliminates unstable data communication can be achieved.

[0041] The data acquisition unit 20a receives sensor data transmitted from the communication unit 14 via wireless communication and acquires the sensor data sent from the sensor node 11 at any time as time-series data. In this embodiment, the data acquisition unit 20a acquires seven detection signals from the microphone 12a, the triaxial vibration sensor 12b, and the three angular velocity sensors 12c.

[0042] The first feature calculation unit 20b calculates the feature quantity, i.e., the first feature quantity, after quantifying the characteristic components contained in the sensor data. The calculation method for the first feature quantity is arbitrary. For example, various methods for obtaining feature quantities from time series data can be used, such as moving average, root mean square (RMS), peak-to-average value of waveform, mean absolute error, and mean square error.

[0043] The timing of the change in movement pattern is detected by the timing of the change in movement pattern 20c. Details will be described later. "Movement pattern" refers to the way the object 10 moves; when the movement pattern changes, the signal contained in the sensor data also changes accordingly. Specifically, at the timing of the change in movement pattern, the detection signal increases, or a larger signal with a specific waveform or frequency is generated. Therefore, the timing of the change in movement pattern can be detected based on the magnitude of the detection signal. Specifically, at the timing of the change in movement pattern, a characteristic component appears in the detection signal. Since the first characteristic quantity, quantified in the first characteristic quantity calculation unit 20b, is equivalent to the quantity obtained by quantifying this characteristic component, the timing of the change in movement pattern can be detected based on this first characteristic quantity. For example, by comparing the first characteristic quantity with a threshold, the timing of the change in movement pattern can be detected if the first characteristic quantity exceeds the threshold, or if the time exceeding the threshold reaches a predetermined time or more.

[0044] If the timing of the change in movement pattern is detected by the timing of the change in movement pattern detection unit 20c, the interval data generation unit 20d performs processing to extract time-series data containing the timing of the movement pattern change from the sensor data. Specifically, the interval data generation unit 20d extracts sensor data for a predetermined interval containing the timing of the movement pattern change; that is, it extracts not only the period of the timing of the movement pattern change, but also a portion of the time-series data for the subsequent period when the movement pattern is maintained.

[0045] Furthermore, in the following explanation, the time-series data of sensor data during the period of the movement mode change is referred to as the change-time data. Additionally, the time-series data of sensor data during a portion of the period when the movement mode is maintained, starting from the change-time, is referred to as the subsequent-time data. Moreover, the predetermined range encompassing the change-time data and the subsequent-time data is referred to as the merged interval, and the time-series data of sensor data within this merged interval is referred to as the merged interval data.

[0046] When the movement mode changes, the value of the first feature increases. Therefore, by comparing the first feature with a threshold, sensor data in the interval where the first feature exceeds the threshold can be extracted to obtain the change timing data. Furthermore, after obtaining the change timing data, a certain point in the movement mode being in a sustained state is set as the extraction completion point, and sensor data in the interval up to that extraction completion point is extracted to obtain subsequent timing data.

[0047] Therefore, the interval data generation unit 20d sets the extraction completion time, for example, after a predetermined time has elapsed since the first feature value exceeds the threshold, or if the first feature value exceeds the threshold but then falls below it, it sets the extraction completion time after a predetermined time has elapsed since the value fell below the threshold. Then, the interval data generation unit 20d extracts sensor data from the merged interval from the time the first feature value exceeds the threshold to the extraction completion time, i.e., merged interval data, as time series data of the movement mode change timing. Furthermore, regarding the merged interval data, the raw values ​​of the sensor data can be directly extracted, or the first feature value calculated based on the sensor data in the merged interval can be extracted.

[0048] The segmented data generation unit 20e divides the time-series data of the detection signal during the sustained state of the motion mode into segmented data at arbitrary time intervals. Sensor data also exhibits specific waveform or frequency characteristics and appears as a repetitive signal when the motion mode is in a sustained state. Therefore, segmented data is generated by dividing the sustained state of the motion mode into segments at arbitrary times. Here, the segmented data generation unit 20e continues to generate segmented data after the end of the merging interval, ensuring that subsequent timing data and segmented data do not overlap, although overlap is possible. Regarding the segmentation of arbitrary time intervals, independent windows with non-overlapping and seamless transitions can be set, or sliding windows with partially overlapping consecutive time intervals can be set.

[0049] The identification unit 20f identifies the type of "movement path" and the "movement mode" by comparing the merged interval data with the verification data stored in the database 20g, which will be described later. Thus, the type of "movement path" and the "movement mode" in the current movement of object 10 can be determined. The movement mode can be determined based on the detection signals of at least one sensor 12, or it can be determined by a combination of multiple detection signals.

[0050] Database 20g stores verification data for identifying "movement path" types and "movement modes" as movement path information; in other words, it serves as training data for templates. The "movement path" type refers to the type of movement method used to move object 10 and the type of material it contacts during movement. For example, the type of movement method refers to whether the device moving object 10 from a starting point to an end point is a screw conveyor, roller conveyor, or belt conveyor—that is, the type of equipment constituting the movement path. The type of material it contacts during movement refers to whether the material constituting the movement path is rubber or metal—that is, the material of the movement path that object 10 contacts. Specifically, the waveform and frequency characteristics of the sensor 12's detection signal change accordingly with the type of movement method, the material it contacts during movement, and the switching of the movement mode. Therefore, database 20g uses the waveform and frequency characteristics of the sensor 12's detection signal generated in scenarios corresponding to the combination of "movement mode" and "movement path" as training data, and stores this training data by associating it with the type of "movement mode" or "movement path." Regarding training data, it can be the waveform and frequency characteristics of the detection signals from typical sensors 12 in various scenarios, or it can be obtained by performing machine learning on each scenario separately to acquire the waveform and frequency characteristics of the detection signals from sensors 12. Furthermore, regarding training data, data obtained from only one sensor 12 can be used, but data obtained from a combination of multiple sensors 12 can more accurately identify the type of "movement path" and "movement pattern." Regarding machine learning, it can be any method, such as neural networks or deep learning.

[0051] The tag assignment unit 20h assigns tags corresponding to the "movement pattern" and "movement path" identified by the identification unit 20f to the merged interval data generated by the interval data generation unit 20d, and saves them. Thus, the merged interval data is associated with the "movement pattern" and "movement path".

[0052] The tag and sequence number assignment unit 20i functions as a tag assignment unit that assigns tags to the segmented data generated by the segmented data generation unit 20e, and a sequence number assignment unit that assigns sequence numbers to each segmented data. Regarding tag assignment, it assigns the same tags as the tags assigned by the tag assignment unit 20h to the merged interval data. Regarding sequence numbers, it assigns sequence numbers in a manner that allows for the individual differentiation of multiple segments of data divided by any time interval.

[0053] The second feature calculation unit 20j calculates the second feature based on the segmented data divided by the segmented data generation unit 20e, taking the characteristic components unique to the movement mode during the continuation of the movement mode as the second feature. Regarding the calculation method for the second feature, various methods listed in the calculation method for the first feature can be used; it can be the same as or a different calculation method. Here, the calculation of the second feature performed by the second feature calculation unit 20j can be performed after or before the tag and serial number assignment by the tag and serial number assignment unit 20i.

[0054] The diagnostic unit 20k detects the movement path state of the object 10 based on the tagged merged interval data or the second feature quantity tagged and serial number, and the data stored in the learning model storage unit 20m, and transmits the detection result to the display device 30. For example, as the movement path state, the anomaly degree of the movement path is calculated, a fault diagnosis of the movement path is performed based on the anomaly degree, and the fault diagnosis result is transmitted to the display device 30 for display. As described above, the tagged merged interval data can be the raw value of the sensor data, but here, the first feature quantity calculated based on the sensor data in the merged interval is used.

[0055] The learning model storage unit 20m learns from the data representing the movement path state of object 10 and stores it as a modeled learning model. Machine learning is performed by moving object 10 along a movement path for which the movement path state is to be detected, accumulating sensor data transmitted from sensor 12, and then learning from it. Machine learning can employ any method, such as neural networks or deep learning. Machine learning can be performed on normal states where no movement path anomalies occur, or on abnormal states where some anomaly occurs along the movement path.

[0056] Here, the "movement mode" will be explained. As mentioned above, the "movement mode" is the way the object 10 moves. When the movement mode changes, the signal contained in the sensor data also changes accordingly. Furthermore, when the movement mode continues, the signal contained in the sensor data will also change according to the movement mode. Therefore, while the first feature quantity is generated by the first feature quantity calculation unit 20b when it detects the change in the movement mode, the second feature quantity is generated by the second feature quantity calculation unit 20j when it detects the maintenance state of the movement mode. Thus, the switching of the movement mode and the type of movement mode can be detected. For example, there are movement modes as shown in (1) to (8) below.

[0057] (1) The mode in which a moving object 10 is forced to stop upon contact with an obstacle. For example, examples include using a stop to forcibly stop the moving object 10 during operation, or placing an obstacle in the movement path to stop the object 10. In this forced-stop movement mode, the object 10 collides with the obstacle and is impacted, producing an impact sound, or the movement path spins freely, or the object 10 slides on the movement path. In the case of an impact, the detection signal of the vibration sensor 12b or the angular velocity sensor 12c contains the impact component. In the case of an impact sound, the detection signal of the microphone 12a contains the impact sound component. In the case of the movement path spinning freely or the object 10 sliding on the movement path, the sound or vibration component causing this will be included in the detection signal of the microphone 12a or in the detection signal of the vibration sensor 12b or the angular velocity sensor 12c.

[0058] (2) The mode that makes the object move at a constant speed. For example, in the case of moving the object 10 at a constant speed using the aforementioned suspension, lifting, or placement methods, the acceleration relative to the direction of travel is zero. Furthermore, if the object 10 is subjected to repeated vibrations along the movement path or slides along the movement path, the resulting sound or vibration components will be included in the detection signal of the microphone 12a or in the detection signal of the vibration sensor 12b or the angular velocity sensor 12c.

[0059] (3) Mode of lifting object 10 by transfer For example, when an object 10 is lifted by the robotic arm of an industrial robot and transferred to a pallet or other moving object, the acceleration, vibration, and rotation generated by the movement are included in the detection signal of the vibration sensor 12b or the angular velocity sensor 12c because the movement path is suspended. In addition, the driving sound of the robotic arm may sometimes also be included in the detection signal of the microphone 12a.

[0060] (4) Movement path switching mode For example, the type of movement path that moves object 10 may change, or even if it is the same type of movement path, it may cross a seam in the movement path. For example, when the movement path changes due to moving from a belt conveyor to a screw conveyor, vibrations in the Z-axis direction (vertical direction) will occur at the time of the change, and the rhythm of the repeated vibrations will change accordingly with the difference in the type of movement path. Therefore, when the movement path changes, the sound and vibration components caused by this will be included in the detection signal of microphone 12a or in the detection signal of vibration sensor 12b or angular velocity sensor 12c.

[0061] (5) Stop mode For example, when an anomaly occurs, the movement of object 10 stops for a short period of time, that is, the movement of object 10 stops in a so-called instantaneous stop. In this case, the acceleration becomes zero, which is reflected in the detection signal of vibration sensor 12b.

[0062] (6) Move Start Mode For example, consider the case where object 10 starts moving from a stationary state. When object 10 starts moving, it will experience different accelerations, such as a gradual acceleration with a slow increase in speed or a rapid acceleration with a sharp increase in speed. This will be reflected in the detection signal of vibration sensor 12b.

[0063] (7) Move End Mode For example, consider the case where a moving object 10 is brought to a stop position. When the object 10 stops moving, it will decelerate in different ways, such as a gradual deceleration that slowly reduces its speed or a sharp deceleration that rapidly reduces its speed. This will be reflected in the detection signal of the vibration sensor 12b.

[0064] (8) Direction conversion mode For example, there are cases where orientation changes occur due to rotation of the movement path. For instance, when object 10 is reoriented on a horizontal plane, acceleration occurs in the X-axis or Y-axis direction. When object 10 rotates around the X, Y, or Z axis, angular velocities in the yaw, pitch, and roll directions are generated. Furthermore, movement sound is also generated accompanying the orientation change of object 10. Therefore, when object 10 is reoriented, the resulting sound, acceleration, or angular velocity components will be included in the detection signal of microphone 12a or in the detection signal of vibration sensor 12b or angular velocity sensor 12c.

[0065] [Structure of display device 30] As long as the movement path information of object 10 can be conveyed to the operator, the display device 30 can be of any form, such as a monitor. When the diagnostic unit 20k obtains the detection result of the movement path status, such as calculating the abnormality of the movement path, the result is displayed through the display device 30.

[0066] As described above, this embodiment constitutes an object movement state detection system. Next, the operation of this object movement state detection system will be explained. The object movement state detection system executes the state detection device 20... Figure 4 , Figure 5 The process shown in the flowchart is used to detect the movement path state of object 10.

[0067] [Action of the object movement state detection system] When detecting the status of the movement path, the status detection device 20 executes the following in each predetermined control cycle: Figure 4 The interval segmentation processing at the point of change of the movement pattern shown and Figure 5 The segmented interval segmentation process is shown in the maintenance state of the movement mode. Furthermore, the actions for detecting the movement path state are explained here, but the same processing is also performed when performing machine learning on the training data stored in database 20g or the learning model stored in learning model storage unit 20m, based on the various data obtained therefrom.

[0068] First of all, Figure 4 In step S100, data acquisition processing is performed. Specifically, the data acquisition unit 20a acquires sensor data transmitted from the sensor node 11. For example, sensor data for a predetermined time period is acquired, which is longer than the time during which a large signal can be generated when the movement mode changes.

[0069] Next, step S110 is performed to calculate the first feature value. Specifically, in the first feature value calculation unit 20b, time series data of a certain shorter time period is extracted from the sensor data of the predetermined time period obtained in step S100, and the first feature value of the time series data is calculated. Regarding the first feature value, characteristic components required to extract the timing of changes in the movement pattern from the sensor data are extracted.

[0070] As an example, such as Figure 6 As shown, the first characteristic quantity is explained using the example of an object 10 equipped with sensor node 11 moving from movement path RA to movement path RB. For example, when movement path RA is a screw conveyor and movement path RB is a belt conveyor, the movement mode switches between different conveyors. In this case, the detection signals of sensor 12 are shown as follows: Figure 7 The waveform shown. Furthermore... Figure 7 The detection signals of microphone 12a and triaxial vibration sensor 12b are shown. For simplicity, the detection signal of angular velocity sensor 12c is omitted.

[0071] like Figure 7As shown, when object 10 moves or is positioned on movement path RA, the microphone 12a receives sound from the movement mode switch or the setting on movement path RA, resulting in a signal with a large amplitude detected by the microphone 12a. Similarly, a vibration sensor 12b also receives a signal with a large amplitude. Furthermore, after a short period of time following the initial movement on movement path RA, the movement mode is maintained, and the detection signals of the microphone 12a and vibration sensor 12b exhibit the waveform and frequency characteristics of the movement path RA maintenance state, appearing as repetitive signals. Then, when switching from movement path RA to movement path RB in terms of movement mode and movement path type, the detection signals of the microphone 12a and vibration sensor 12b change according to the switching scenario. In this example, the detection signals of vibration sensor 12b in the X-axis and Z-axis directions become larger signals, while the detection signals of microphone 12a and vibration sensor 12b in the Y-axis direction, although not larger signals, become signals different from those in the movement path RA. Furthermore, after a short period of time following the initial movement of the movement path RB, it enters a movement mode maintenance state. The detection signals from microphone 12a and vibration sensor 12b exhibit the waveform and frequency characteristics of the movement path RB maintenance state, and appear as repetitive signals. Specifically, within the area indicated by the dashed box in the figure, the waveform of the detection signal from vibration sensor 12b in the Z-axis direction is magnified as follows: Figure 8 The waveform shown in the upper part.

[0072] Therefore, in scenarios involving the switching of movement modes and movement path types, there exist characteristic signals corresponding to that scenario. Thus, signal processing is performed on the detection signals from microphone 12a and vibration sensor 12b to calculate a first characteristic quantity. For example, a moving average is performed on the detection signals, and the value obtained by averaging the amplitude of the detection signals is used as the first characteristic quantity. Although this first characteristic quantity is an averaged value of the detection signals, its value is still greater than the value when the movement mode is maintained in the scenario involving the switching of movement modes and movement path types.

[0073] Next, step S120 is performed to extract the timing of the change. Specifically, the timing of the change is detected by the timing of the change detection unit 20c, and the interval data generation unit 20d performs the process of extracting merged interval data containing the timing of the change and subsequent timing data from the sensor data.

[0074] In other words, extract the contents contained in Figure 8The time series data is within the range Rx highlighted in bold solid line. When the movement mode changes, since the value of the first feature calculated in step S110 increases, sensor data in the interval where the first feature exceeds the threshold can be extracted by comparing the first feature with a threshold, thus obtaining the change timing data. Furthermore, subsequent timing data is obtained as follows: when the first feature exceeds the threshold and then decreases below it, sensor data in the interval from the moment it falls below the threshold until a predetermined time is extracted.

[0075] Therefore, the timing when the first feature quantity exceeds the threshold is first detected by the timing detection unit 20c as the timing of the change in the movement mode. The first feature quantity is as follows: Figure 8 The lower part shows a waveform that smooths the amplitude of the acceleration detection signal in the Z-axis direction. When the acceleration detection signal in the Z-axis direction increases, the first feature value also becomes larger; when the detection signal decreases, the first feature value becomes smaller. Figure 8 Similar to timing t1, when a change in movement pattern occurs, if the first feature value exceeds a threshold, a change in movement pattern is detected. Upon detection, the interval data generation unit 20d sets an extraction completion timing, for example, after a predetermined time has elapsed since the first feature value exceeded the threshold, or after a predetermined time has elapsed since the first feature value fell below the threshold after exceeding it. Then, the interval data generation unit 20d uses the interval from the time the first feature value exceeded the threshold until the extraction completion timing as a merging interval, and extracts the sensor data from this merging interval. In this way, merged interval data is extracted.

[0076] In addition, if the threshold of the first feature quantity is not exceeded at this time, the data truncation of the merged interval will not be performed in step S120, and the process will proceed directly to step S130.

[0077] Subsequently, in step S130, identification processing is performed. Specifically, in the identification unit 20f, the verification data stored in the database 20g is compared with the change timing data and subsequent timing data contained in the merged interval data extracted in step S120, thereby identifying the type of "movement path" and the "movement pattern". For example, data with high similarity to the change timing data and subsequent timing data are extracted from the verification data, and the "movement pattern" and "movement path" associated with the extracted verification data are identified as the current "movement pattern" and "movement path". When the merged interval data uses the first feature quantity calculated in the merged interval, since the verification data is also data of the feature quantity corresponding to the first feature quantity, it is only necessary to compare the first feature quantity with the feature quantity stored as the verification data.

[0078] Next, proceed to step S140 to perform the label assignment process. In this process, labels corresponding to the "movement mode" and "movement path" identified in step S130 are assigned to the merged interval data and saved. Alternatively, if no merged interval data is extracted in step S120, label assignment is not performed, and the process proceeds to the next step.

[0079] Then, proceed to step S150 to determine if the data processing has ended. If it has ended, the interval segmentation process at the point of change in movement mode is completed; otherwise, proceed to step S160. In other words, regarding the data in the independent window corresponding to the merged interval, if the processing in steps S110 to S140 has ended and the merged interval data with assigned labels has been saved, a positive determination is made in step S150; otherwise, a negative determination is made in step S150. Then, in step S160, data for the next predetermined time consecutive with the predetermined time used in this step is extracted from the sensor data obtained in step S100, and the processing from step S110 is repeated for the next predetermined time. In this way, the interval segmentation process at the point of change in movement mode is completed.

[0080] On the other hand, Figure 4 In the interval segmentation process at the point of change of movement mode, if labels are assigned and data is saved after the merged interval data is acquired, then the following steps are executed: Figure 5 The segmented interval division process shown is illustrated.

[0081] First, in step S200, it is determined whether the movement mode has stopped. Since the segmentation interval division process generates segmented data while the movement mode is in a sustained state, there is no need to execute it if the movement mode has stopped. Therefore, if it is determined in step S200 that the movement mode has not stopped, the process proceeds to step S210.

[0082] In the subsequent step S210, the method used in Figure 4 The sensor data acquired in step S100 is processed into segmented data generation. In this segmented data generation process, segmented data is generated at arbitrary time intervals. Specifically, segments are extracted... Figure 8 The time series data contained within the range Ry1 to Ry4 enclosed by the dashed line are used to generate segmented data. Here, although segmented data is generated through multiple independent windows with consecutive non-overlapping time intervals, it can also be done as... Figure 9 The data is segmented by setting up a sliding window with overlapping portions of continuous time intervals as shown.

[0083] Following this, proceed to step S220, where the segmented data is assigned... Figure 4In step S140, the label is assigned the same label as the label assigned in the processing, and a sequence number is assigned to each segment of data for storage. For example, if it is a constant speed movement mode on the movement path RB, sequence numbers such as "constant speed RB1, constant speed RB2, constant speed RB3..." are assigned sequentially starting from the earlier segment of the time series.

[0084] Then, in step S230, it is determined that... Figure 4 In step S100, the sensor data acquired is checked to determine if it contains data for the next movement mode. Here, "next movement mode data" refers to the data for the movement mode after the mode switch. If the determination in step S200 is negative, or if the acquired sensor data contains data for the next movement mode, the process proceeds to step S240, where the processing after step S200 is repeated for that next movement mode data. This completes the segmentation process for the movement mode maintenance state.

[0085] In this manner, after generating merged interval data and segmented data from the acquired sensor data, the second feature quantity calculation unit 20j calculates the second feature quantity based on the segmented data. At this time, the second feature quantity is also assigned the same label and sequence number as assigned in step S220. Then, in the diagnostic unit 20k, the calculated second feature quantity and the merged interval data, such as the previously calculated first feature quantity, are compared with the learning model stored in the learning model storage unit 20m to detect the movement path state.

[0086] Specifically, the current "movement mode" and "movement path" can be determined from the labels assigned to the first or second feature. Therefore, the "movement mode" indicated by the label and the corresponding learning model for the "movement path" are read from the learning model storage unit 20m. Then, based on the first feature, it is determined whether there is an anomaly in the movement path state during the movement mode switch, thereby performing fault diagnosis on the movement path. Similarly, based on the second feature, it is determined whether there is an anomaly when the movement mode is in a maintenance state, thereby performing fault diagnosis on the timing of the movement mode maintenance state. Since the second feature is calculated based on the number of segmented data, fault diagnosis can also be performed on each sequence number. In this case, it is also possible to diagnose where the fault occurs on its movement path. For example, the anomaly degree of the movement path can be calculated based on the magnitude of the deviation when comparing the first or second feature with the features shown in the learning model. Alternatively, if the deviation degree is small, the movement path state is determined to be normal; if the deviation degree is large, the movement path state is determined to be abnormal.

[0087] For example, when the guide rail is used as the moving path, various abnormalities in the moving path can be detected, such as deformation of the guide rail connection or wear of the guide rail, deformation at the switching point of the moving path, foreign objects attached to the part constituting the moving path, and malfunction of the motor of the moving device. In addition, abnormalities in the gripping state when the object 10 is moved can also be detected, such as the tilting posture of the object 10 caused by foreign objects stuck in the gripping part, and deviations in the configuration angle when gripping and transferring the object 10.

[0088] Furthermore, during machine learning, each assigned label can be processed using merged interval data and segmented data separately for "movement pattern" and "movement path" to generate a learning model, which is then stored in the learning model storage unit 20m. In this case, the original values ​​of the merged interval data and segmented data can be used, but alternatively, the first feature of the merged interval can be used as the merged interval data, and the second feature derived from subsequent sensor data can be used as the segmented data for machine learning.

[0089] In this way, when the detection result of the movement path status is obtained, the result is transmitted from the diagnostic unit 20k to the display device 30, and the movement path status is communicated to the user by displaying the corresponding detection result on the display device 30.

[0090] As described above, the object movement state detection system according to this embodiment detects situations where a large signal is generated at the moment of change in movement pattern based on a first feature quantity, and extracts a merged interval of change moment data and subsequent moment data. Therefore, by comparing the merged interval data with verification data, the type of "movement path" and "movement pattern" can be identified, and by comparing the merged interval data with a learning model, the movement path state at the moment of change in movement pattern can be detected. Furthermore, segmented data is generated by extracting portions after the merged interval from sensor data at arbitrary time intervals, and a second feature quantity is calculated based on each segmented data. Therefore, by comparing the second feature quantity with the learning model, the movement path state when the movement pattern is maintained can be detected.

[0091] Then, after identifying the type of "movement path" and "movement pattern" based on the timing of changes in movement patterns, interval segmentation is used to extract and merge interval data and segmented data as time series data. Therefore, machine learning can be performed after excluding moments when the object's movement completely stops, which are not anticipated based on the type of "movement path" or "movement pattern." Thus, even if time deviations in sensor data are caused by sudden stops or malfunctions, or impacts from foreign objects adhering to the movement path, these are excluded before machine learning. For example, as... Figure 10As shown, under normal conditions without malfunctions, the same waveform repeats during the maintenance state of the movement mode. However, when a malfunction occurs and object 10 stops, waveform distortion occurs during the repetition of the same waveform. In this case, the signal at the time of the malfunction can be excluded for machine learning. Furthermore, since segmented data is generated under stable conditions such as the maintenance state of the movement mode, time deviations in the time series data can be suppressed. Therefore, the accuracy of machine learning can be improved, thereby improving the accuracy of movement path state detection.

[0092] Furthermore, since the types and patterns of "movement paths" are identified based on the verification data pre-stored in database 20g, it can also be applied when the movement paths are different.

[0093] (Other implementation methods) This disclosure is based on the above-described embodiments, but is not limited to these embodiments, and includes various modifications or modifications within the same scope. Furthermore, various combinations and methods, and even other combinations and methods containing only one element, more elements, or fewer elements, all fall within the scope or spirit of this disclosure.

[0094] For example, in the above embodiment, as an example, it is shown that when switching from movement path RA to movement path RB, the detection signals of vibration sensor 12b in the X-axis and Z-axis directions increase. However, this is only one example, and which of the following sensors included in sensor 12—microphone 12a, vibration sensor 12b, and angular velocity sensor 12c—has a larger detection signal can be determined by a combination of "movement mode" and "type of movement path". Furthermore, in the above embodiment, when the detection signal of any of the sensors 12 becomes a larger signal, machine learning or movement path state detection is performed based on the first and second feature values ​​of the detection signal that becomes the larger signal. In this regard, when the detection signal of any of the sensors 12 becomes a larger signal, the first and second feature values ​​can be calculated not only for the detection signal that becomes the larger signal but also for other detection signals, and these can also be used for machine learning or movement path state detection.

[0095] Furthermore, in database 20g, the verification data is set as data under specific conditions when object 10 is moved. However, if the conditions change, the time series data acquired corresponding to those conditions can be adjusted and compared with the verification data. For example, sometimes the verification data is set assuming that object 10 moves at a certain speed. In this case, if object 10 is moved at different speeds, a speed sensor can be pre-equipped with object 10 as sensor 12, and the time series data can be adjusted based on the speed detected by the speed sensor to be acquired when object 10 moves at a certain speed. That is, by recalculating it to data at a speed corresponding to the verification data in database 20g, it is possible to identify the type of "movement path" or "movement mode" under different conditions. Of course, the verification data in database 20g can also be adjusted to data corresponding to the speed detected by the speed sensor and compared with the time series data acquired at different speeds.

[0096] Furthermore, interferences other than those intended for machine learning or path state detection, such as sound, vibration, and angular velocity generated by autonomous movement, can reduce the accuracy of machine learning or path state detection. Therefore, components corresponding to such interferences can be removed from the detection signal.

[0097] Furthermore, in the event that the movement path is not determined, the diagnostic unit 20k can store the movement history of autonomous movement, generate a movement space map based on the movement history, and display it on the display device 30.

[0098] In addition, such as Figure 11 As shown, the state detection device 20 can also be equipped with a quality evaluation unit 20n to evaluate the quality of the merged interval data or segmented data, such as the quality of the first feature or the second feature, and display the quality evaluation result on the display device 30. In this case, the merged interval data and segmented data are evaluated, and if the quality is poor, they may not be used for machine learning or movement path state detection. For example, a parent group of data for the first feature or the second feature is prepared for each tag, and an allowable range is set based on the parent group data. If the first feature and the second feature are within the allowable range, they are used for machine learning or movement path state detection. The allowable range may be a range from the maximum value to the minimum value contained in the parent group data, or a range set by narrowing the range, or a range of ± a few to tens of percent relative to the average value, etc.

[0099] Furthermore, the object 10 equipped with sensor nodes 11 is not limited to one. Moving multiple objects 10 increases the amount of data for machine learning or movement path state detection, thereby improving accuracy. Moreover, the model used for comparison is not limited to a learning model obtained through machine learning. For example, a model pre-stored with default data for each label for movement path state detection can be used, and the merged interval data or segmented data can be compared with this model to detect the movement path state.

[0100] The control unit and methods described in this disclosure can also be implemented using a dedicated computer, which consists of a processor and memory programmed to perform one or more functions specified by a computer program. Alternatively, the control unit and methods described in this disclosure can also be implemented using a dedicated computer provided with a processor composed of one or more dedicated hardware logic circuits. Furthermore, the control unit and methods described in this disclosure can also be implemented using one or more dedicated computers, which are composed of a processor and memory programmed to perform one or more functions and a processor composed of one or more dedicated hardware logic circuits. In addition, the computer program can also be stored as instructions executable by the computer in a computer-readable, non-transitory tangible recording medium.

[0101] (This is the viewpoint of the publication) The above disclosure can be understood, for example, as shown below.

[0102] [First Opinion] An object movement state detection device acquires sensor data showing the sensing result of a sensor from an object (10) equipped with a sensor (12) that outputs a detection signal corresponding to the movement state, thereby detecting the movement state of the object, characterized in that it comprises: The data acquisition unit (20a) acquires the sensor data; The timing of change detection unit (20c) detects the timing of changes in the movement pattern, which is the movement state of the object, based on the sensor data acquired by the data acquisition unit. The interval data generation unit (20d) takes the detected change timing and a portion of the interval from the change timing when the movement mode is maintained as a merged interval and generates merged interval data, which is the time series data of the sensor data in the merged interval; The database (20g) contains verification data, i.e. training data used to identify the movement patterns and movement paths; The identification unit (20f) identifies the movement pattern and the movement path by comparing the merged interval data with the verification data; The segmented data generation unit (20e) generates multiple segmented data from the sensor data after the merged interval when the movement mode is maintained; The label assignment unit (20h, 20i) assigns labels to the merged interval data generated by the interval data generation unit and the segmented data generated by the segmented data generation unit, corresponding to the movement pattern and movement path identified by the identification unit; and The diagnostic unit (20k) detects the movement path status of the object based on the merged interval data, the segmented data, and the label.

[0103] [Second viewpoint] The object movement state detection device according to the first point of view is characterized by further comprising a learning model storage unit (20m), which stores a learning model of the merged interval data and the segmented data corresponding to the movement mode and the movement path. The diagnostic unit detects the movement path state of the object by comparing the merged interval data and the segmented data with the learning model corresponding to the label stored in the learning model storage unit. The learning model storage unit generates the learning model by performing machine learning on the merged interval data and the segmented data corresponding to the movement mode and the movement path according to each label assigned by the label assignment unit when the interval data generation unit generates the merged interval data and the segmented data and the segmented data generate the segmented data, and stores the learning model.

[0104] [Third Viewpoint] According to the object movement state detection device of the first or second viewpoint, the diagnostic unit calculates the abnormality of the object's movement path as the movement path state, thereby performing fault diagnosis of the movement path.

[0105] [Fourth viewpoint] The object movement state detection device according to any one of the first to third viewpoints is characterized by further comprising a feature quantity calculation unit (20b), which calculates feature quantities that are obtained by quantifying characteristic components that appear in the sensor data at the time of change of the movement pattern. The timing of the change is detected by the feature quantity calculated by the feature quantity calculation unit.

[0106] [Fifth viewpoint] According to the fourth point of view, the object movement state detection device is characterized in that the feature quantity calculation unit extracts the component of the increase in sensor data when the movement mode changes, and thereby calculates the feature quantity.

[0107] [Sixth Viewpoint] According to the object movement state detection device described in the fourth or fifth viewpoint, the feature quantity calculation unit is characterized in that the feature quantity calculation unit is a first feature quantity calculation unit, and the feature quantity calculated by the first feature quantity calculation unit is the first feature quantity calculation unit. The object movement state detection device further includes a second feature quantity calculation unit (20j), which calculates a second feature quantity by quantifying the characteristic components that occur during the duration of the movement mode based on the segmented data generated by the segmented data generation unit. The diagnostic unit detects the movement path status of the object based on the first feature and the second feature.

[0108] [Seventh Viewpoint] The object movement state detection device according to any one of the first to sixth viewpoints is characterized in that the tag assignment unit inherits the tag assigned to the merged interval data by the recognition unit corresponding to the movement mode and the movement path, and assigns tags to the segmented data generated by the segmented data generation unit. The object movement state detection device further includes a tag and serial number assignment unit (20i) that assigns a serial number to each of the plurality of segmented data.

[0109] [8th Viewpoint] The object movement state detection device according to any one of the first to seventh viewpoints is characterized in that the data acquisition unit acquires the detection signals of the microphone, the three-axis accelerometer, and the three-axis angular velocity sensor as sensor data from a composite sensor consisting of a microphone (12a) for detecting sound, a triaxial accelerometer (12b) for detecting acceleration in three mutually orthogonal directions, namely the X-axis, Y-axis, and Z-axis, and a triaxial angular velocity sensor (12c) for detecting angular velocities around the three axes. The timing of the change in movement pattern is detected by the microphone and at least one of the detection signals from the triaxial accelerometer and the triaxial angular velocity sensor.

[0110] [Ninth Viewpoint] The object movement state detection device according to any one of the first to eighth viewpoints is characterized in that the detection result of the movement path state obtained by the diagnostic unit is displayed on the display device (30).

[0111] [10th Viewpoint] The object movement state detection device according to the ninth point of view is characterized by comprising a quality evaluation unit (20n) for evaluating the quality of the merged interval data and the segmented data. The quality evaluation results made by the quality evaluation department are displayed on the display device.

[0112] [11th Viewpoint] According to the 9th or 10th viewpoint, the object movement state detection device is characterized in that it is applicable to the detection of the movement state of any of the following: an object that moves by the movement of a device that forms a movement path in contact with the object and does not have power; or an object that has the power to move autonomously.

[0113] [12th Viewpoint] According to the object movement state detection device of the 11th point, the object has the power to perform the autonomous movement, and the movement path of the object has not been determined. The diagnostic unit stores the movement history of the object's autonomous movement and generates a movement space map based on the movement history, which is then displayed on the display device.

[0114] [13th Viewpoint] A method for detecting the movement state of an object, comprising acquiring sensor data showing the sensing result of a sensor (12) from an object (10) equipped with a sensor (12) that outputs a detection signal corresponding to the movement state, thereby detecting the movement state of the object, characterized in that it includes: The step of acquiring the sensor data (S100); The steps of detecting the timing of changes in the movement pattern, which is the movement state of the object, based on the acquired sensor data (S110, S120). The step of generating merged interval data (S120) is to take the detected timing of the change and a portion of the interval in which the movement mode is maintained from the timing of the change as the merged interval. The merged interval data is the time series data of the sensor data in the merged interval. The step of comparing the merged interval data with the verification data, i.e. the training data used to identify the movement pattern and the movement path, to identify the movement pattern and the movement path (S130). The step of generating multiple segmented data from the sensor data after the merged interval when the movement mode is maintained (S210). The steps of assigning labels to the generated merged interval data and segmented data in accordance with the identified movement pattern and movement path (S140, S220). The step of detecting the movement path status of the object based on the merged interval data, the segmented data, and the label.

Claims

1. An object movement state detection device, wherein sensor data showing the sensing result of a sensor is acquired from an object (10) having a sensor (12) that outputs a detection signal corresponding to the movement state, thereby detecting the movement state of the object, characterized in that, have: The data acquisition unit (20a) acquires the sensor data; The timing of change detection unit (20c) detects the timing of changes in the movement pattern, which is the movement state of the object, based on the sensor data acquired by the data acquisition unit. The interval data generation unit (20d) takes the detected change timing and a portion of the interval from the change timing when the movement mode is maintained as a merged interval and generates merged interval data, which is the time series data of the sensor data in the merged interval; The database (20g) contains verification data, i.e. training data used to identify the movement patterns and movement paths; The identification unit (20f) identifies the movement pattern and the movement path by comparing the merged interval data with the verification data; The segmented data generation unit (20e) generates multiple segmented data from the sensor data after the merged interval when the movement mode is maintained; The label assignment unit (20h, 20i) assigns labels to the merged interval data generated by the interval data generation unit and the segmented data generated by the segmented data generation unit, corresponding to the movement pattern and movement path identified by the identification unit; as well as The diagnostic unit (20k) detects the movement path status of the object based on the merged interval data, the segmented data, and the label.

2. The object movement state detection device according to claim 1, characterized in that, It also includes a learning model storage unit (20m), which stores the learning model of the merged interval data and the segmented data corresponding to the movement mode and the movement path. The diagnostic unit detects the movement path state of the object by comparing the merged interval data and the segmented data with the learning model corresponding to the label stored in the learning model storage unit. The learning model storage unit generates the learning model by performing machine learning on the merged interval data and the segmented data corresponding to the movement mode and the movement path according to each label assigned by the label assignment unit when the interval data generation unit generates the merged interval data and the segmented data and the segmented data generate the segmented data, and stores the learning model.

3. The object movement state detection device according to claim 1 or 2, characterized in that, The diagnostic unit calculates the anomaly degree of the object's movement path as the movement path status, thereby performing fault diagnosis on the movement path.

4. The object movement state detection device according to claim 1, characterized in that, It also includes a feature calculation unit (20b) that calculates feature quantities, which are obtained by quantifying the characteristic components in the sensor data that appear at the time of change of the movement mode. The timing of the change is detected by the feature quantity calculated by the feature quantity calculation unit.

5. The object movement state detection device according to claim 4, characterized in that, The feature calculation unit extracts the component that increases in the sensor data when the movement mode changes, and then calculates the feature quantity.

6. The object movement state detection device according to claim 4 or 5, characterized in that, The feature quantity calculation unit is the first feature quantity calculation unit, and the feature quantity calculated by the first feature quantity calculation unit is the first feature quantity calculation unit. The object movement state detection device further includes a second feature quantity calculation unit (20j), which calculates a second feature quantity by quantifying the characteristic components that occur during the duration of the movement mode based on the segmented data generated by the segmented data generation unit. The diagnostic unit detects the movement path status of the object based on the first feature and the second feature.

7. The object movement state detection device according to claim 1, characterized in that, The tag assignment unit inherits the tags assigned to the merged interval data by the identification unit corresponding to the movement mode and the movement path, and assigns tags to the segmented data generated by the segmented data generation unit. The object movement state detection device also includes a tag and serial number assignment unit (20i) that assigns a serial number to each of the multiple segmented data.

8. The object movement state detection device according to claim 1, characterized in that, The data acquisition unit acquires sensor data from a composite sensor consisting of a microphone (12a) for detecting sound, a triaxial accelerometer (12b) for detecting acceleration in three orthogonal directions (X-axis, Y-axis, and Z-axis), and a triaxial angular velocity sensor (12c) for detecting angular velocities around the X-axis, Y-axis, and Z-axis. The detection signals from the microphone, the triaxial accelerometer, and the triaxial angular velocity sensor are used as sensor data from the sensors. The timing of the change in movement pattern is detected by the microphone and at least one of the detection signals from the triaxial accelerometer and the triaxial angular velocity sensor.

9. The object movement state detection device according to claim 1, characterized in that, The detection results of the movement path status obtained by the diagnostic unit are displayed on the display device (30).

10. The object movement state detection device according to claim 9, characterized in that, It includes a quality evaluation department (20n) to evaluate the quality of the merged interval data and the segmented data. The quality evaluation results made by the quality evaluation department are displayed on the display device.

11. The object movement state detection device according to claim 9 or 10, characterized in that, The object movement state detection device is applicable to the detection of the movement state of any of the following objects: objects that move by means of a device forming a movement path in contact with the object and do not have power; and objects that have the power to move autonomously.

12. The object movement state detection device according to claim 11, characterized in that, When the object possesses the power to perform the autonomous movement, and the object's movement path is not determined, The diagnostic unit stores the movement history of the object's autonomous movement and generates a movement space map based on the movement history, which is then displayed on the display device.

13. A method for detecting the movement state of an object, comprising acquiring sensor data showing the sensing result of a sensor (12) from an object (10) having a sensor (12) that outputs a detection signal corresponding to the movement state, thereby detecting the movement state of the object, characterized in that, include: The step of acquiring the sensor data (S100); The steps of detecting the timing of changes in the movement pattern, which is the movement state of the object, based on the acquired sensor data (S110, S120). The step of generating merged interval data (S120) is to take the detected timing of the change and a portion of the interval in which the movement mode is maintained from the timing of the change as the merged interval. The merged interval data is the time series data of the sensor data in the merged interval. The step of comparing the merged interval data with the verification data, i.e. the training data used to identify the movement pattern and the movement path, to identify the movement pattern and the movement path (S130). The step of generating multiple segmented data from the sensor data after the merged interval when the movement mode is maintained (S210). The steps of assigning labels to the generated merged interval data and segmented data in accordance with the identified movement pattern and movement path (S140, S220). The step of detecting the movement path status of the object based on the merged interval data, the segmented data, and the label.

Images