State determination system and state determination method

Abstract

To provide a state determination system and the like that can widely determine the state of an object including a structure or equipment.SOLUTION: A state determination system for determining the state of an object including a structure or equipment includes a shape acquisition unit that acquires the shape of the object, and a determination unit that uses a trained model to determine the state of the object based on the shape shown in an image acquired by the shape acquisition unit. The determination unit can determine the state based on the shape shown in frames that constitute the image of the object, or based on time-series changes in the shape.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a condition determination system and a condition determination method. [Background technology]

[0002] Patent Document 1 discloses a displacement measuring device that measures the displacement of a measurement target caused by a load. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6883767 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology disclosed in Patent Document 1, there are cases where it is not possible to determine the state of an object including, for example, a structure or equipment, based on displacement alone.

[0005] Therefore, in one aspect, the present invention aims to provide a state determination system and the like that can widely determine the state of an object including a structure or equipment. [Means for solving the problem]

[0006] In one embodiment, A state determination system for determining a state of an object including a structure or equipment, a shape acquisition unit that acquires a shape of the object; a determination unit that determines a state of the object based on the shape acquired by the shape acquisition unit using a trained model; A state determination system is provided, comprising: [Effects of the Invention]

[0007] In one aspect, the present invention makes it possible to determine the state of a wide range of objects, including structures or equipment. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating a configuration of a state determination system according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram showing an image of the vicinity of a winch as an object. [Figure 3] 10 is a flowchart illustrating a process in a determination unit. DETAILED DESCRIPTION OF THE INVENTION

[0009] FIG. 1 is a diagram showing the configuration of a state determination system according to this embodiment.

[0010] As shown in FIG. 1, the state determination system 10 of this embodiment includes a shape acquisition unit 11 that acquires the shape of an object including a structure or equipment based on an image captured by a camera 20, a determination unit 12 that uses a trained model to determine the state of the object based on the shape acquired by the shape acquisition unit 11, and a sound vibration acquisition unit 13 that acquires sound or vibration.

[0011] The determination unit 12 can determine the state of an object based on the shape shown in frames constituting a captured image (video) of the object. The determination unit 12 can also determine the state of an object based on a time-series change in the shape shown in the captured image of the object (a change in the shape shown in each frame).

[0012] The sound vibration acquisition unit 13 acquires sound or vibration detected by a microphone, an acceleration sensor, or other sensor 30. When sound or vibration is acquired via the sound vibration acquisition unit 13, the determination unit 12 can determine the state based on the sound or vibration acquired by the sound vibration acquisition unit 13 in addition to the shape described above.

[0013] The camera 20 and the sensor 30 are installed at various sites, such as construction sites and work sites, and the condition determination system 10 can quickly detect abnormalities that may occur at the sites.

[0014] The state determination system 10 of this embodiment can be configured using one or more computers on which a predetermined program is installed.

[0015] Next, the operation of the state determination system 10 of this embodiment will be described.

[0016] The shape acquisition unit 11 acquires the shape of an object photographed by the camera 20. The object photographed includes a structure or equipment. The structure or equipment is, for example, a structure or equipment installed at various sites including a construction site or a work site, and includes a retaining wall, a vent, a winch, etc. The shape acquisition unit 11 can extract the shape of the object from the image of the object photographed by the camera 20 using well-known image recognition technology.

[0017] Any camera or sensor can be used as the shape acquisition unit 11, and for example, a ToF camera, a lidar, a laser sensor, etc. can be used.

[0018] The determination unit 12 uses the trained model to determine the state of the object based on the shape acquired by the shape acquisition unit 11. The trained model is generated, for example, by using an image of a structure or equipment under normal conditions as a training image.

[0019] FIG. 2 is a diagram showing an image of the vicinity of a winch as an object. When the object includes a winch, an image of a state in which a wire 51 is being properly wound by a winch 50 is used to generate a trained model, for example. Using a large number of images can improve the accuracy of inference, but it is also possible to acquire images of the process of winding the wire 51 in time series and reflect the characteristics of changes in shape shown in the images in the trained model. The shape shown in the image can be acquired, for example, through a process of extracting the outline of the object included in the image.

[0020] Furthermore, an image taken when an abnormality occurs can be used as a teacher image. In this case, for example, if an abnormality with multiple attributes exists, learning can be performed by labeling each image with the attribute of the abnormality. In this way, by reflecting the attribute of the abnormality in the trained model, the determination unit 12 can distinguish and determine the attribute of the abnormality during inference.

[0021] If the target object includes a retaining wall or vent (temporary support platform), the shape shown in the image (teacher image) of the retaining wall or vent in its normal state can be reflected in the trained model. In this case, changes in the shape of the retaining wall or vent as the work progresses are also learned as normal and reflected in the trained model. Even in this case, the accuracy of inference can be improved by using a large number of images.

[0022] As described above, the shape acquisition unit 11 executes a process of extracting a shape shown in an image acquired by the camera 20. Here, for example, by a process of recognizing the contour of an object included in the image, the shape of the recognized contour can be extracted as the relevant shape.

[0023] Furthermore, the determination unit 12 infers the state of the object by inputting the extracted shape into the trained model.

[0024] 3 is a flowchart illustrating the processing in the shape acquisition unit and the determination unit, which illustrates the processing for making inferences based on images acquired sequentially in time series from the camera 20.

[0025] In step S102 of FIG. 3, the shape acquisition unit 11 acquires a new image from the camera 20.

[0026] In step S104, the shape acquisition unit 11 extracts the shape shown in the image acquired in step S102.

[0027] In step S106, the determination unit 12 infers the state of the object by inputting the shape extracted in step S104 into the trained model.

[0028] In step S108, the judgment unit 12 judges whether or not an abnormality has occurred based on the result of the inference in step S106, and if the judgment is positive, the process proceeds to step S110, and if the judgment is negative, the process proceeds to step S102.

[0029] In step S110, a predetermined process corresponding to the occurrence of an abnormality is executed, and the process proceeds to step S102. Here, the predetermined process may include notifying the occurrence of the abnormality, issuing an alarm, and stopping operation related to the object (for example, stopping operation of the winch 50).

[0030] In this way, in this embodiment, the state of the object is determined (inferred) based on the time-series (dynamic) changes in the shape acquired by the shape acquisition unit 11, so that when an abnormality occurs, it can be quickly detected and the necessary processing (step S110) can be performed.

[0031] For example, if the target object includes a winch 50 and a wire 51 (FIG. 2), the normal state in which the shape of the area of ​​the wire 51 gradually changes over time is learned and reflected in the trained model. Therefore, if the winch 50 correctly winds up the wire 51, it is inferred to be normal. On the other hand, if an abnormality occurs in the winding of the wire 51, for example, the shape of the area of ​​the wire 51 suddenly changes or becomes abnormal, and the inference in the determination unit 12 indicates an abnormality, and the necessary processing (step S110) is executed.

[0032] Furthermore, if the target object includes a retaining wall or vent, for example, if the shape displacement exceeds a normal range or the frequency or amplitude of the shape (contour) vibration indicates an abnormality, the inference in the judgment unit 12 indicates an abnormality and the necessary processing (step S110) is executed. This allows for rapid detection of slope collapse, vent deformation, and their signs. Meanwhile, in this embodiment, changes in shape as work progresses are learned and reflected in the trained model, preventing erroneous detection of an abnormality based on normal changes in shape as work progresses. Furthermore, inference using the trained model can effectively suppress the effects of disturbances such as rust and dirt that tend to occur on site.

[0033] The above inference result can be calculated as a parameter value indicating the possibility of an abnormality in the determination unit 12. In this case, for example, if the parameter value exceeds a predetermined threshold, it can be determined that an abnormality has occurred (step S108).

[0034] As described above, when sound or vibration is acquired via the sound and vibration acquisition unit 13, the determination unit 12 can determine the condition based on the sound or vibration acquired by the sound and vibration acquisition unit 13 in addition to the shape. For example, when the sound and vibration acquisition unit 13 acquires an abnormal sound (e.g., a creaking sound) or an abnormal vibration (e.g., a sound related to the collapse of a retaining wall or a precursor thereof) at the site, the determination unit 12 can reflect the acquired abnormal sound or vibration in the inference result. In this case, for example, when an abnormal sound or abnormal vibration is acquired, the determination unit 12 may add a predetermined value to a parameter value indicating the possibility of an abnormality. This allows a safer determination result to be obtained when there is an abnormal sound or abnormal vibration than when there is no abnormal sound or abnormal vibration. Abnormal sound or abnormal vibration can be specified based on, for example, the frequency or amplitude of the sound or vibration. Alternatively, normal sounds and vibrations may be learned and reflected in a trained model (a model related to sound and vibration), and the determination unit 12 may use this trained model to determine whether or not there is an abnormality.

[0035] As described above, in this embodiment, the judgment unit 12 uses a trained model to judge the state of the object based on the shape acquired by the shape acquisition unit 11, and therefore can widely judge the state of objects including structures or equipment.

[0036] Although the embodiments have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications and changes are possible within the scope of the claims. In addition, it is also possible to combine all or a plurality of components of the above-described embodiments. [Explanation of symbols]

[0037] 10. Status Judgment System 11 Shape acquisition section 12 Judgment section 13 Sound and vibration acquisition unit 20 Camera 30 sensors

Claims

1. A state determination system for determining a state of an object including a structure or equipment, a shape acquisition unit that acquires a shape of the object; a determination unit that determines a state of the object based on the shape acquired by the shape acquisition unit using a trained model; A state determination system comprising:

2. The state determination system according to claim 1 , wherein the determination unit determines the state based on the shape shown in a frame constituting an image of the object.

3. The state determination system according to claim 1 , wherein the determination unit determines the state based on a time-series change in the shape.

4. a sound vibration acquisition unit for acquiring sound or vibration; The state determination system according to claim 1 , wherein the determination unit determines the state based on the shape as well as the sound or vibration acquired by the sound vibration acquisition unit.

5. A state determination method for determining a state of an object including a structure or equipment, comprising: a shape acquisition step of acquiring a shape of the object; a determination step of determining a state of the object based on the shape acquired by the shape acquisition step using a trained model; A state determination method comprising:

Images