Condition change detection device and condition change detection program

The state change detection device using optical fibers connected to mobile terminals addresses the high cost and limited deployment of conventional systems by processing optical information through a smartphone camera, enabling cost-effective and widespread monitoring of structural changes.

JP7582597B2Active Publication Date: 2024-11-13KOBE UNIV +1
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2021056305
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-03-31
Filing Date
2021-03-29
Publication Date
2024-11-13
Estimated Expiration
2041-03-29

AI Technical Summary

Technical Problem

Conventional monitoring systems for detecting state changes in natural or artificial structures, such as landslides, are costly and prone to malfunctions, and require expensive electrical sensors and data loggers, limiting their deployment to specific locations.

Method used

A state change detection device utilizing optical fibers connected to a mobile terminal, such as a smartphone, processes optical information through a camera to detect state changes, reducing costs by eliminating the need for dedicated data loggers and enabling widespread monitoring using a mobile device.

Benefits of technology

The system significantly reduces monitoring costs by allowing optical information from multiple optical fibers to be processed by a mobile terminal, facilitating widespread and efficient monitoring of state changes in structures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007582597000002
    Figure 0007582597000002
  • Figure 0007582597000003
    Figure 0007582597000003
  • Figure 0007582597000004
    Figure 0007582597000004
Patent Text Reader

Abstract

To provide a device which can analyze light information transmitted from an optical fiber by an information terminal, detect the state change of a natural or artificial structure and perform monitoring for various purposes.SOLUTION: A state change detection device comprises: a portable terminal 10 having camera means; one or a plurality of optical fibers 11 connected to a measurement object 2; and a positioning jig 12 which superimposes a lens position of the camera means on one end position of the optical fiber. The portable terminal 10 detects the state change of the measurement object 2 from color information on an image by image processing emission light from the optical fiber 11 by an image sensor of the camera means. The end positions of the plurality of optical fibers 11 are aligned, the image sensor performs image processing of the emission light from each end of the optical fiber 11, and the portable terminal detects the state change by specifying the measurement object 2 or measurement place from position information in the image. The positioning jig 12 is provided with a marker that may discriminate the alignment positions of the ends of the plurality of optical fibers 11, and the ends and the marker are fixedly arranged such that the ends of the optical fibers 11 and the marker exist in the photographed image.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a condition change detection device for monitoring the occurrence, progression, and increased danger of condition changes in natural or artificial structures. oh and a program for detecting abnormalities in status. [Background technology]

[0002] Several monitoring systems have been commercialized and introduced to detect signs of disasters such as landslides on dangerous slopes caused by heavy rain. To monitor the risk of landslides on dangerous slopes and their changes over time, it is necessary to monitor groundwater fluctuations, the state of surface flow on the slope, ground movement, rock movement, tree movement, and the soundness of drainage facilities, as shown in Figure 11. In such cases, in the conventional monitoring system shown in Figure 12, various data are acquired using sensors that are assumed to use electricity (hereinafter referred to as "electrical sensors"), and the data is collected in a data logger using wireless technology, and the on-site technical manager often determines the degree of danger and communicates it to nearby residents through administrative agencies such as the local government. Since the deployment of such conventional monitoring systems requires several million yen per slope, it is difficult for small municipalities or individuals to shoulder the costs of deploying them. In addition, there are cases where unforeseen circumstances occur, such as malfunctions of electrical sensors or lightning strikes, and information from the multiple deployed electrical sensors may not be transmitted accurately and promptly. Furthermore, with conventional monitoring systems, the high introduction costs mean that they have to be installed only in limited locations where landslides are likely to occur. Therefore, in order to reduce the system implementation cost, methods and devices for monitoring status changes using light instead of electricity are known (see, for example, Patent Documents 1 and 2). Monitoring status changes using light involves monitoring by matching the colors with status changes using three colors of light, like road traffic lights, green, yellow, and red, based on the widely accepted concepts that green means safety, yellow means caution, and red means danger.

[0003] On the other hand, technological advances in user-owned mobile terminals such as smartphones and tablet terminals have improved the performance of video capture by camera functions, image data recording, and image processing, approaching the functionality of dedicated imaging devices and data analysis devices. For example, an analyte measurement system is known in which a reaction area on a test piece arranged in a peripheral device for test pieces arranged above an image sensor and a light source provided in a mobile terminal is read by the camera function of the mobile terminal, and the characteristics of the analyte are determined based on the color of the reaction area in the image captured by the camera (see Patent Document 3). The system disclosed in Patent Document 3 is equipped with a light guide jig for guiding light from a light source such as the screen or flash of the mobile terminal to the reaction area on the test piece, so that the reaction area on the test piece is illuminated by light from the screen or the like, and the color of the reaction area can be read by the camera function of the mobile terminal. In this way, the mobile terminal can function as a substitute for a dedicated data analysis device, contributing to reducing the cost of introducing the system.

[0004] For example, if data related to the signs of landslides on dangerous slopes could be collected as optical information and an application program could be provided to process and analyze the images on the user's mobile device (smartphone, tablet, etc.), users would be able to monitor dangerous slopes around them from home in order to protect their own lives. Furthermore, even when the work is carried out as part of monitoring rather than for personal use, it is useful for workers to go to the site with their mobile device, or to permanently install the mobile device at the site and carry out long-term measurements, thereby collecting data related to signs of disaster in the area around the site as optical information, and then processing the collected optical information into images on the worker's mobile device to monitor the level of danger. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2008-309784 A [Patent Document 2] Re-tabled publication 2012 / 111317 [Patent Document 3] JP 2018-105886 A Summary of the Invention [Problem to be solved by the invention]

[0006] Optical fibers are used as a means of transmitting optical information. The optical information captured by optical fibers is usually analyzed by dedicated optical fiber sensors and data loggers, but this method incurs data logging costs of several tens of thousands of yen per optical fiber, which poses a major problem when multiple optical fibers are used simultaneously. In light of this situation, this invention provides a technology that can process the optical information captured by multiple optical fibers simultaneously using a camera on a mobile device such as a smartphone, significantly reducing the overall cost of monitoring. It also supports monitoring using a single optical fiber, allowing anyone with a mobile device such as a smartphone to monitor abnormal conditions using optical information through optical fiber.

[0007] In other words, the present invention aims to provide an apparatus, method, and program that can analyze optical information transmitted from an optical fiber using an information terminal such as a mobile terminal, detect changes in the condition of natural or man-made structures, and perform monitoring for a variety of purposes, such as disaster prevention, disaster mitigation, and maintenance and management of structures. [Means for solving the problem]

[0008] In order to solve the above problems, the condition abnormality detection device of the present invention includes a camera means, an information terminal that processes images captured by the camera means, one or more optical fibers connected to the object to be measured, and a positioning jig for aligning the lens position of the camera means with one end position of the optical fiber; Equipped with The positioning jig is provided with markers capable of determining the aligned positions of the ends of the multiple optical fibers, and the ends and the marker are fixedly positioned so that the ends of the multiple optical fibers and the marker are reflected in an image captured by the image sensor of the camera means. The information terminal is The image sensor of the camera means processes the light emitted from each end of the optical fiber, and the object to be measured or the measurement location is identified from the position information in the processed image. Changes in the condition of the object being measured are detected from the color information. pictureBy identifying the object to be measured from the position information of the image area, it becomes possible to effectively identify the detection target. Here, as the information terminal, a mobile terminal such as a smartphone or a tablet terminal is preferable, but a PC or the like may also be used.

[0009] mosquito This configuration makes it possible to effectively utilize optical fiber when performing monitoring for a variety of purposes, while keeping implementation costs down and enabling rational sensing. Here, the camera means is not necessarily limited to being built into the information terminal, but may be externally connected. Also, connecting the optical fiber to the object to be measured is not limited to being directly connected, but may be connected to the object to be measured via another optical fiber, a camera, the Internet, an information terminal such as a PC, etc.

[0010] As for optical fibers, plastic optical fibers are used. POF (Polyester Optical Fiber) is preferably used. POF is an optical fiber whose core is made of plastic, and because it is flexible, it is less expensive than glass optical fibers (the material cost for an optical fiber with a diameter of 1 mm is about 50 yen / m), and because the cable can be cut with a simple tool, cable installation is easy. Also, POF has a higher optical attenuation rate than glass optical fibers, so it is not suitable for long-distance data transmission, but it can be used for short-distance data transmission up to about 200 m.

[0011] The positioning tool can be a socket type (joint type) that connects the lens position of the camera of the information terminal and the end position of the optical fiber so that they face each other, or a hole type in which a hole for the camera lens is provided in the base on which the information terminal itself is installed or attached, and the end of the optical fiber is fixed to the bottom of the hole, but it can be anything that overlaps the lens position of the camera means and the end position of the optical fiber.In addition, as for the color information of the image, the data values ​​(values ​​from 0 to 255) of the three primary colors R (red), G (green), and B (blue) and the values ​​of the three color elements H (Hue), S (Saturation), and V (Value·Brightness) calculated from these values ​​are used to determine the meaning of the light information and detect changes in the condition of the measured object.

[0012] The condition change detection device of the present invention aligns the end positions of multiple optical fibers, and an image sensor processes the light emitted from each end of the optical fibers, and identifies the object to be measured or the measurement location from the positional information in the processed image, thereby detecting a condition change. One end of each of a plurality of optical fibers is connected to a plurality of objects to be measured, the end positions of the other ends of the optical fibers are aligned, and information on changes in the state of the plurality of objects to be measured is captured as optical information in a camera image at one time. Alternatively, one end of each of a plurality of optical fibers is connected to different locations of a single object to be measured, the end positions of the other ends of the optical fibers are aligned, and information on changes in the state of the different locations of the single object to be measured is captured as optical information in a camera image at one time. This makes it possible to detect changes in the state of the plurality of objects to be measured, or changes in the state of different locations of a single object to be measured, and to perform more accurate risk prediction judgments. The end positions of the multiple optical fibers are preferably arranged in a lattice pattern in order to arrange many of them in a certain area, but they may also be arranged in a line or in a circle.

[0013] 1If there is only one optical fiber, the optical information is obviously information originating from the object to be measured to which the optical fiber is connected. On the other hand, when multiple optical fibers are used, in order to identify which measurement location of which object the optical information of each optical fiber originates from, it is necessary to assign individual identification numbers such as addresses to the alignment positions of the ends of the optical fibers and to identify the multiple pieces of optical information in the image. For this reason, the ends and markers are fixedly positioned so that the markers that can identify the alignment positions of the ends of the multiple optical fibers are reflected in the captured image.

[0014] The marker is preferably, for example, a three-point marker such as that used for reading a QR code (registered trademark), but is not limited thereto, and may be a one-point marker such as an arrow or an L-shape, or a two-point marker, as long as it can recognize the XY addresses of the two-dimensionally arranged ends and identify the ends. When the markers are arranged in a line or in a circle, each end can be identified by indicating the starting point with a one-point marker.

[0015] In the condition abnormality detection device of the present invention, it is preferable that the information terminal detects color information of a plurality of divided image areas corresponding to the emitted light from each end of a plurality of optical fibers, and detects a condition abnormality from the detected plurality of color information. This makes it possible to detect color information for each fiber of multiple optical fibers or for a group of fibers, and to detect abnormalities in their condition while looking at their correlations. In other words, it becomes possible to detect abnormalities in the condition of multiple objects to be measured, or abnormalities in the condition of different parts of a single object to be measured, and to make more accurate risk prediction judgments.

[0016] In the condition abnormality detection device of the present invention, it is preferable that the image sensor performs image processing on the real-time image of the emitted light and detects the condition abnormality in real time. The real-time image of the emitted light is acquired by the video capture function of the camera of the information terminal. The acquired image is stored in the memory of the information terminal, but since the memory capacity is limited, the image file is saved for an appropriate time, and when no abnormality is detected, the image file is deleted to prevent the memory capacity from being overloaded. By capturing the optical fibers capturing a variety of information with the camera of an information terminal and analyzing the data from each optical fiber or the information captured by multiple optical fibers in real time using specific rules to define a single piece of data, it becomes possible to perform a variety of monitoring.

[0017] In the condition abnormality detection device of the present invention, it is preferable that the information terminal comprises a tag input means for inputting an identification tag previously assigned to the optical fiber, and stores the image processed by the image sensor in association with the identification tag. By providing a tag input means, it becomes possible to gather one or more bundles of fibers and read the assigned identification tag. The means for reading the tag is not particularly limited, but it is possible to use a method such as reading a QR code (registered trademark) with a camera, inputting a tag code from a key, or selectively inputting by operating an application screen of an information terminal.

[0018] In the condition abnormality detection device of the present invention, it is preferable that the information terminal is provided with a threshold setting means for setting a color threshold for outputting data according to the color information, and outputs data by comparing the color information with the threshold. By providing a threshold setting means, it is possible to output data such as a warning or caution notification according to the color information. Methods for setting the threshold include setting it by operating the application screen of the information terminal, or downloading a setting table via communication. Data output includes screen display, alarm sound output, data transmission via communication, etc.

[0019] The status change detection program of the present invention includes the following 1) to7 ) to cause a computer to execute each step. 1) An image input step of inputting an image obtained by image processing the light emitted from the optical fiber by the image sensor of the camera means while overlapping the position of one end of one or more optical fibers connected to the object to be measured with the position of the lens of the camera means. 2) A tag input step of inputting an identification tag that has been pre-assigned to the optical fiber. 3) A threshold setting step for setting a color threshold for outputting data on the change in the state of the object to be measured according to the color information of the image. 4) A tag association step, which associates the input image with an identifying tag. 5) A data output step of comparing the color information of the input image with a threshold value and outputting data.

[0020] 6 2.) A marker detection step of detecting a marker capable of determining the aligned positions of the ends of a plurality of optical fibers from the input image. 7) A color information detection step of detecting color information of a plurality of divided image regions corresponding to the light emitted from each end of a plurality of optical fibers based on the positions of the markers.

[0022] In the condition abnormality detection device of the present invention, the area in the captured image that is the target of condition abnormality detection may be an image area that is previously associated with the object to be measured among the divided image areas (cells). By detecting only the image area that is previously associated with the object to be measured, image areas that are not previously associated with the object to be measured can be excluded from the detection target, making it possible to efficiently obtain data only from the required locations.

[0023] In the condition abnormality detection device of the present invention, the area to be detected for abnormality in the captured image may be a specific area among the divided image areas. By detecting only a specific area among the divided image areas, it becomes possible to obtain necessary data more efficiently.

[0024] In the condition abnormality detection device of the present invention, the area in the captured image that is the target of condition abnormality detection may be an area within a predetermined pixel value range in a predetermined light band. By detecting only the area within the predetermined light band, it is possible to efficiently obtain data only from the required area. Furthermore, by detecting only the area within the predetermined pixel value range, it is possible to exclude areas that are too bright or too dark from the detection target, and to obtain the required data more efficiently. Effect of the Invention

[0025] According to the present invention, there is an advantage that optical information transmitted from an optical fiber can be analyzed by an information terminal to detect abnormalities in the condition of natural or artificial structures, and monitoring can be performed for various purposes such as disaster prevention, disaster mitigation, maintenance and management of structures, etc. According to the present invention, there is an advantage that the cost of introducing a monitoring system can be significantly reduced compared to conventional monitoring systems. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram illustrating the configuration of a state change detection device according to the present invention. [Diagram 2] An illustration of an optical fiber sensor that can sense changes in condition [Diagram 3] Conceptual diagram of sensing changes in the internal condition of a structure [Figure 4] Conceptual diagram of condition change detection device [Diagram 5] Functional diagram of the condition change detection device [Figure 6] Conceptual diagram of a disaster prevention management system using the condition change detection device of the present invention. [Figure 7] An explanatory diagram of an example of a positioning jig [Figure 8] Example of optical fiber end arrangement and smartphone image [Figure 9] Marker illustration [Figure 10] Processing flow diagram of the condition change detection program [Figure 11] Illustration of monitoring the risk of landslides on dangerous slopes [Figure 12] Diagram of a conventional monitoring system [Figure 13] Image of the use of the condition change detection device of Example 3 [Figure 14] Image of light source image capture [Figure 15] Illustrative diagram of detection area definition [Figure 16] Variation of communication format in the fourth embodiment [Figure 17] Image of the configuration of the condition change detection device DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] The present invention is a condition change detection technology for monitoring the occurrence, progression, and increased danger of condition changes in natural or artificial structures, and an example of an embodiment of the present invention will be described in detail below with reference to the drawings, taking a dangerous slope, which is one type of natural structure, as an example. The measurement object in the present invention is not limited to natural structures, and can also be applied to artificial structures. Here, natural structures refer to parts of natural landforms formed by soil materials and bedrock, such as natural slopes around roads and residential areas, and natural river levees. Snow is also included when considering the risk of avalanches in heavy snowfall areas. Man-made structures are broadly divided into civil engineering structures, architectural structures, and construction machinery used in their construction. Civil engineering structures refer to bridges, power transmission and communication towers, dams, tunnels, embankments, reclaimed land, artificial river levees, artificial slopes, etc., architectural structures refer to ordinary houses, high-rise buildings, public buildings (museums, schools, train stations, gymnasiums, etc.), large-scale leisure facilities (concert halls, sports stadiums, Ferris wheels, roller coaster tracks, etc.), temporary structures at event venues, etc., and construction machinery refers to large cranes and large heavy machinery that require operators and may be approached by residents or workers during construction.

[0028] As shown in Fig. 1, the condition change detection device 1 of the present invention includes a mobile terminal 10 equipped with a camera means, one or more optical fibers 11 connected to an object to be measured, and a positioning jig 12 that aligns the lens position of the camera means with one end position of the optical fiber. In Fig. 1(1), one end of one optical fiber 11 is connected to the camera lens of the mobile terminal 10 via the positioning jig 12, and the other end of the optical fiber 11 functions as an optical fiber sensor 3 and is connected to one object to be measured 2. In Fig. 1(2), one end of a bundle of multiple optical fibers 11 is connected to the camera lens of the mobile terminal 10 via the positioning jig 12, and the other end of each of the multiple optical fibers 11 functions as an optical fiber sensor 3 and is connected to a different location of one object to be measured 2. In FIG. 1(3), one end of a bundle of multiple optical fibers 11 is connected to a camera lens of a mobile terminal 10 via a positioning jig 12, and the other end of each of the multiple optical fibers 11 functions as an optical fiber sensor 3 and is connected to four objects to be measured 2a to 2d. In Figs. 1(1) to (3), the mobile terminal 10 uses an image sensor of a camera means to process light emitted from an optical fiber 11, and detects a change in the state of the object to be measured from color information of the processed image.

[0029] In the present invention, a plastic optical fiber (POF) is used as the optical fiber because of its cost and ease of installation, but the optical fiber is not limited to this, and a glass optical fiber can also be used. In the present invention, an optical fiber is connected to the object to be measured, and there are various connection methods, such as attaching the optical fiber at a distance from the surface of the object to be measured, attaching the optical fiber to the surface, inserting the optical fiber into the object to be measured, or embedding the optical fiber inside the object, and the connection method can be appropriately selected according to the characteristics and state of the object to be measured. In the present invention, a change in the state of the object to be measured is detected by a sensor that can sense the change, and the sensor is required to output light according to the change in state. Outputting light according to the change in state may be a type that emits light when the state changes, or a type that changes the intensity or color of light according to the amount of change in state. Alternatively, the optical fiber may be a type that emits light under normal conditions and does not emit light, such as by blocking the light when the state changes.

[0030] As an example of a sensor that can sense a change in condition using optical fiber, there is the optical fiber sensor shown in Fig. 2 (1) to (4). In Fig. 2 (1), the sensor is made up of a single optical fiber, and functions as a sensor when light emitted from an arbitrary light source enters the optical fiber after being reflected by or transmitted through the object to be measured 2. Events that occur around the tip of the optical fiber are the subject of sensing. Fig. 2 (2) to (4) show an optical fiber sensor made up of two optical fibers. In Fig. 2 (2), light L sent into fiber 1 s exits from the end of the fiber as L2, is reflected by the object to be measured 2, and enters the fiber 2 as L3. r The sensor functions as a type of sensor that measures events occurring around the tips of the two fibers. In Figure 2 (3), the light L sent to fiber 1 is s From that end, L 2b The light L exits the fiber as L1, is reflected by the object under test 2, and travels toward fiber 2 as L2. 2a While passing through the object to be measured 2, the light travels toward fiber 2 and reaches L 2a and L 2b Finally, L r The sensor functions as a sensor that measures events occurring around the gap between the tips of the two fibers. In Figure 2(4), light L sent to fiber 1 is s passes through the end face, is refracted, and exits as L2. The reflected light L3 heads toward fiber 2. A part of L3 leaves fiber 1 and enters fiber 2 from the side as L'3, and again splits into refracted light L4 and reflected light L r It is divided into: rSince the brightness and color of light depend on the refractive index of the surrounding material, this type of sensor senses events that cause a change in the refractive index of the surrounding material. Events that cause a change in the refractive index of the surrounding material include, for example, changes in the surrounding material (from gas to liquid, or vice versa) and phase changes in the surrounding material (from solid to liquid, or freezing and melting in the case of water). By embedding an optical fiber sensor in the soil on a dangerous slope, it is possible to extract optical information about changes in the condition of the soil on the slope. Figures 3(1) and (2) show the optical fiber sensors described in Figures 2(2) and 2(3) embedded in the soil, respectively.

[0031] In the case of the optical fiber sensor described above, if we limit our consideration to the area where sensing by light is performed, no devices or power sources that depend on electricity are required. In the present invention, the optical information of the sensor is transmitted by optical fiber, and the transmitted optical information is image-processed by the camera of the mobile terminal, which allows for cost reduction of the system. Since the acquired data can be analyzed by the mobile terminal, there is no need to introduce expensive data loggers or analytical computers, which allows for significant cost reduction.

[0032] The positioning jig in the present invention can be a socket-type jig that connects the camera lens position of the mobile terminal and the end position of the optical fiber so that they face each other, or a hole-type jig in which a hole for the camera lens is provided in the receiving base on which the mobile terminal itself is installed or mounted, and the end of the optical fiber is fixed to the bottom of the hole.

[0033] The socket-type jig is preferably provided with a housing that covers the periphery of the camera lens of the mobile terminal and is fixed to the mobile terminal. Since the camera lens of a smartphone or tablet terminal is embedded in the terminal surface without any step, the jig is fixed to the mobile terminal itself by a clip method or the like, and the socket-type jig is attached so that the periphery of the camera lens is covered by the housing of the socket. When the jig is attached to the mobile terminal, the housing is designed so that the end position of the optical fiber is located at a position opposite the lens position of the camera of the mobile terminal. For example, if the housing is cylindrical, one opening is directed toward the camera lens, and the other opening is covered with a member in which the end of the optical fiber is arranged. The camera of the mobile terminal is automatically focused by an autofocus function, but in the present invention, focus adjustment is not necessary. That is, light is emitted from the end of the optical fiber, and the state abnormality is detected by the color information of the emitted light, so there is no problem even if the focus is not correct. Even when the ends of multiple optical fibers are aligned, the images of the individual optical fibers do not need to be completely focused as long as they are within a range where they do not overlap. Therefore, the cylindrical housing only needs to block external light, and the distance between the camera lens and the end of the optical fiber does not need to be particularly precise. When the marker for identifying the end of the optical fiber is read using the camera flash, the housing is shaped to cover the periphery of the camera lens and the flash window.

[0034] On the other hand, the hole type jig has a cradle on which the mobile terminal itself is placed or attached, and a hole for the camera lens is provided in the cradle. The cradle is shaped to match the shape of the mobile terminal, so that the user can easily understand how to place the mobile terminal. The cradle may not only receive the mobile terminal on a horizontal surface, but also hold and support the mobile terminal. The hole for the camera lens may be provided at a position corresponding to the position of the camera lens, as long as it is shaped to match the shape of the mobile terminal, or may be provided near the center of the cradle so that the user places the mobile terminal so that the position of the camera lens is at the position of the hole. The end of the optical fiber is disposed at the bottom of the hole. As with the socket type jig, precise focusing is not required, and when the marker identifying the end of the optical fiber is read by the camera flash, the size of the hole is set to include not only the camera lens but also the flash window.

[0035] In the present invention, when multiple optical fibers are used, it is necessary to identify which measurement location of which object the optical information of each optical fiber originates from. An identification number is assigned to the aligned positions of the ends of the optical fibers, and a marker capable of identifying the aligned positions of the ends of the multiple optical fibers is provided in order to identify multiple pieces of optical information within an image. Regarding the markers, three-point markers, two-point markers, and one-point markers will be described later.

[0036] In the present invention, the mobile terminal processes the light emitted from the optical fiber with the image sensor of the camera, and detects a change in the state of the object to be measured from the color information of the processed image. In order to output data according to the color of the image, a color threshold is set in advance and stored as a threshold table in the memory of the mobile terminal. In the threshold table, for example, a color threshold is specified using an RGB value (values ​​of 0 to 255) for specifying a color and an HSV value that can be calculated from the RGB value. Depending on the threshold, a notification such as a warning or caution can be output as data according to the color of the image. The data output includes displaying a message or figure on the screen, sounding an alarm, vibrating, turning on the light, sending an email to the outside via data communication, and the like. Note that the threshold can be set by appropriately selecting one or more of the six values ​​of the RGB value and the HSV value according to the characteristics of the object to be measured, and setting it optimally. For example, the judgment may be made using three values ​​of the RGB value simultaneously, the judgment may be made using the H value (color number of 0 to 360), or the judgment may be made using the V value (simply brightness). Specific examples will be described in detail below. Note that the scope of the present invention is not limited to the following examples and illustrated examples, and many modifications and variations are possible. EXAMPLES

[0037] FIG. 4 shows a configuration diagram of an embodiment of the condition change detection device of the present invention. The condition change detection device 1 shown in FIG. 4 includes a smartphone 10, a plurality of POFs 11 connected to a plurality of points on a dangerous slope, and a positioning jig 12 for overlapping the position of a camera 110 of the smartphone with the position of an end of the POFs 11. The smartphone 10 takes an image of the light emitted from the end of the POFs 11 with the camera 110 through the positioning jig 12. The smartphone 10 is equipped with a condition change detection application (program), which detects various abnormalities such as the displacement of a dangerous slope, changes in the inclination angle, and the movement of soil from the color information of the image processed by the image sensor of the camera, displays a time-series line graph of the amount of change on the condition change detection application screen, and can output an alarm sound and data to an external server as necessary. In the case of FIG. 4, two POFs 11 are prepared, one of which supplies white light from the LED 111 of the smartphone 10 to the object to be measured 2, and the other transmits the reflected light from the object to be measured 2 as optical information. POF11 is placed at multiple points within areas at risk of landslides, and optical information regarding the movement of soil particles and groundwater is observed from the ends of the POF11. In addition to supplying white light from the LED 111 of the smartphone 10 to the POF 11 side, a device having an LED light source can be attached to the smartphone 10 or its periphery to supply white light or light of a specific color from the LED light source to the POF 11 side.

[0038] Fig. 5 shows an image of a disaster prevention management system using the present invention, in which an app installed in a smartphone analyzes the optical information of the POF, displays the danger level of the slope on the screen for the user, and transmits the data to an external server computer. The smartphone 10 acquires optical information emitted from the ends of multiple POFs 11 using the installed camera as an optical fiber end image 101, and the app on the smartphone 10 analyzes the color information of each end, analyzes the analysis results in real time, displays them on a data screen 102 to notify the user of the danger level in the surrounding area, and can also upload the results to the cloud server 4. That is, as shown in Fig. 6, a group of sensors capable of sensing displacement, vibration, tilt, turbidity, groundwater level, material movement, etc. are constructed using optical fiber sensors 3 and connected to the monitoring target, and optical information obtained from the optical fiber sensors 3 is transmitted via multiple POFs 11, and condition abnormality detection device 1 performs image processing to detect abnormalities in condition. The detected data on abnormalities in condition is uploaded to a cloud server 4 (web server), and local governments, etc. can check the safety of the surrounding area from the uploaded analysis data, upload it to a dedicated website, and, if necessary, issue alarms to residents' mobile devices and PCs.

[0039] Next, the positioning jig will be described with reference to Fig. 7. As shown in Fig. 7, the positioning jig 12 has a base 123 on which the smartphone 10 can be placed, and a hole 124 in which the camera lens of the smartphone 10 can be set. The positioning jig 12 receives a cable bundle of multiple POFs 11, and has a POF end 120 fixed to an arrangement panel 122 at the bottom of the hole 124. The smartphone 10 itself is mounted on the cradle 123 so that the position of the camera lens 110 overlaps the position of the hole 124 of the cradle 123. The POF end 120 aligned on the placement panel 122 inside the hole 124 is photographed by the camera function of the smartphone 10.

[0040] An example of the arrangement of POF ends and an image captured by a smartphone will be described with reference to FIG. 8. As shown in FIG. 8, 96 POF ends 120 are arranged on an arrangement panel 122 within a 10×10 grid. The image (optical fiber end image) 101 captured by the smartphone 10 shows 96 POF ends arranged within a 10×10 grid. Three markers 121 are arranged in three of the four corners, similar to a QR code (registered trademark). The smartphone 10 app recognizes the markers 121 to determine the orientation of the image. The app can obtain color information for each grid. If the marker 121 is not reflected in the camera image and cannot be recognized because the housing (not shown) of the positioning jig blocks external light, the smartphone camera's flash function can first recognize the marker 121 to determine the orientation of the image. Alternatively, a light source can be provided on the rear side of the placement panel 122 of the POF end 120 (with the camera position side facing forward) to provide light from behind the marker 121 to clearly identify the marker 121 and determine the orientation of the image.

[0041] In addition, to avoid a situation where the light emitted from the end of the optical fiber is reflected on the surface of the smartphone or the surface of the camera lens and then further reflected on the housing surface on which the end of the optical fiber is located, causing the light emitted from the end and the reflected light to mix and be unable to be detected correctly, a transparent anti-reflection film may be provided between the camera lens and the end of the optical fiber.

[0042] Smartphone cameras are equipped with a function that can display real-time images, and the camera can be turned on at all times to continuously monitor the condition of the slope in real time. Note that instead of real-time images, photographs can be taken at regular sampling intervals (e.g., every 30 seconds) and image processing can be used to detect changes in the condition. If any signs of abnormality are found, the sampling interval can be shortened.

[0043] The smartphone condition anomaly detection app detects the condition anomaly of the object being measured from the color information of the image captured by the camera of the light emitted from the optical fiber. In order to change the alarm sound output according to the color of the image, for example, color thresholds are preset in the threshold table shown in Table 1 below. In the threshold table shown in Table 1, alarm sounds are set for five levels according to the degree of the condition anomaly, such as two types of normal, caution, warning, and danger.

[0044] [Table 1]

[0045] In this embodiment, a three-point marker is used as a marker provided on the arrangement panel of the POF end, but a one-point marker 121a may be used as shown in FIG. 9(1). The one-point marker 121a has a characteristic shape, and the XY direction can be determined by the shape. Also, two-point markers 121b and 121c may be used as shown in FIG. 9(2). The two-point markers 121b and 121c can be used to determine the XY direction of the alignment surface of the POF end. The positions of the markers 121b and 121c are detected in the image, and the direction from marker 121b to 121c is recognized as the X axis with marker 121b as the reference point (origin), and the direction perpendicular to the X axis is recognized as the Y axis with marker 121b as the origin, so that each POF end can be identified by each marker. EXAMPLES

[0046] An example of the process flow of the condition abnormality detection program of the present invention will be described. Fig. 10 shows a process flow diagram of the condition abnormality detection program. The condition abnormality detection program of the present invention is an application program (hereinafter, also simply referred to as an app) installed in the mobile terminal (such as a smartphone) 10 of the condition abnormality detection device 1 of the first embodiment. As shown in the flow of Fig. 10, first, the application is started to execute a step of inputting an identification tag into the computer (step S01). The identification tag can be input by inputting (or selecting) the identification tag number from the application screen, or by reading the barcode tag attached to the positioning jig 12. Next, a step of inputting a camera image into the computer is executed (step S02). The mobile terminal 10 is set in the hole-type positioning jig 12 of the first embodiment, and the POF end 120 at the bottom of the hole in the positioning jig 12 is photographed with a camera to obtain a POF end image 101, which is then input.

[0047] Next, the input POF end image 101 is associated with the identification tag (step S03). From the identification tag, information on the POF connected to the positioning jig 12 can be obtained. The POF information is information on how many POFs 11 there are and what they are measuring, that is, information on which measurement point on which measurement object is being measured.

[0048] Next, color information corresponding to the multiple POF ends 120 is obtained based on the input POF end image 101 (step S04). In the POF end image 101 associated with the identification tag, it is possible to identify which color on the image comes from which POF 11. Then, the color information of each POF end 120 is compared with a set threshold value related to the object to be measured connected to the POF 11, and an alarm screen is displayed on the application as necessary, and an alarm sound is output (step S05). In step S04, a marker 121 as described in the first embodiment is placed on the POF end image 101, so that the arrangement orientation and addresses of the multiple POFs 11 can be identified, and it can be determined which POFs 11 each of the multiple POF ends 120 corresponds to. EXAMPLES

[0049] FIG. 13 is a conceptual diagram of the use of the condition change detection device of the third embodiment, where (1) shows an image of light emitted from a light source, and (2) shows an image of an image captured by a camera. As shown in FIG. 13(1), the ground 6 is a slope that is prone to landslides. Light sources (9a to 9e) are arranged on the ground 6, and these are, for example, lights such as street lamps arranged to illuminate the surroundings. As the light source, a wide variety of light-emitting devices other than lights can be used, but they are not limited to those that emit light themselves, and may be, for example, those that have a structure that receives and reflects light. As shown in Fig. 13(1), light sources (9a to 9e) are placed on the ground 6. Therefore, as shown in Fig. 13(2), a camera 10a is used to detect abnormalities. Specifically, the camera 10a acquires optical information emitted from the multiple light sources (9a to 9e) as an image of the end of an optical fiber, and transmits the acquired image information to an information terminal (not shown). An application on the information terminal analyzes the color information of each end, analyzes the analysis results in real time, and displays them on a data screen to notify the user of the degree of danger in the surrounding area, while also uploading the results to a cloud server. As the information terminal, in addition to a smartphone, a mobile terminal such as a tablet terminal or a PC may be used.

[0050] In the condition alteration detection device of the third embodiment, as in the first embodiment, the input light source image is associated with the identification tag. The identification tag is input by inputting (or selecting) the identification tag number from the application screen. The light source information that can be acquired from the identification tag is information such as the number of light sources and where they are located. However, unlike the case where all POF ends are aligned vertically and horizontally as in the condition alteration detection device 1 of the first embodiment, an image of randomly arranged light sources is acquired, so the light source information acquired from the identification tag is also different. Therefore, the light source information acquired from the identification tag in the condition alteration detection device of the third embodiment will be described.

[0051] FIG. 14 shows an image of a light source image. As shown in FIG. 14, a total of 100 cells (10×10) are displayed as an image in the light source image 101a. Although light sources are not all arranged in each cell, it is assumed that light sources are all arranged in an order in each cell, and some of the cells are recognized as detection areas. That is, in the example shown in FIG. 14, the light source information is linked to the arrangement of light source 9a in cell 5a, light source 9b in cell 5b, light source 9c in cell 5c, light source 9d in cell 5d, and light source 9e in cell 5e.

[0052] If it is possible to limit the area in an image in which abnormalities are to be detected, it will be possible to detect abnormalities with high accuracy using a small amount of data. Fig. 15 is an explanatory diagram of detection area definition, where (1) shows area definition by dividing into 9 parts, and (2) shows area definition by the center. The example shown in Fig. 15(1) is the method used in Example 3, in which the detection area is limited by dividing cell 5 into 9 areas (50a to 50i). In other words, by considering only the area where the light source exists, it is possible to ignore the area where the light source does not exist.

[0053] Specifically, when it is desired to consider only the region in cell 5 where light source 9f exists, it is defined as taking into account region 50a in cell 5. Also, when the position of a light source spans multiple regions, such as light source 9g, it is defined as taking into account regions (50c, 50f) in cell 5. Similarly, when it is desired to consider only the region where light source 9h exists, it is defined as taking into account regions (50d, 50e, 50g, 50h) in cell 5. Also, unlike this embodiment, when it is known that the POF end 120a exists in the center of the cell 5 as in the first embodiment, a central area may be defined in advance, such as the area 50j shown in Fig. 15(2). By limiting to the central area 50j, the position from which data is to be acquired can be effectively limited.

[0054] In addition, in the image processed by the image sensor, the area to be detected for abnormality may be an area within a range of a predetermined pixel value in a predetermined light band. By limiting the light band or brightness to be detected to a certain range, it becomes possible to efficiently obtain data only from the required area. EXAMPLES

[0055] Fig. 16 shows variations of the communication form. The example shown in Fig. 16(1) is a communication form in the condition anomaly detection device 1 of the first embodiment, in which the POF end 120, which is the end of a plurality of optical fibers 11, is photographed using the camera lens of the smartphone 10. In contrast, in the communication mode of the condition abnormality detection device 1a shown in Fig. 16(2), a camera 10b externally connected to a smartphone 10 is used to capture an image of a POF end 120, which is an end of a plurality of optical fibers 11. In this way, the camera means is not necessarily limited to one built into a mobile terminal, and may be one connected externally.

[0056] In addition, in the communication form of the condition abnormality detection device 1b shown in FIG. 16(3), a camera 10b is used to capture an image of a POF end 120b, which is an end of a plurality of optical fibers 11a. The image captured by the camera 10b is transmitted to a PC 8 via the Internet 7, and a POF end image 101b is displayed on the display of the PC 8. The POF end image 101b displayed on the display of the PC 8 is then directly or indirectly connected to one end of the plurality of optical fibers 11b. A smartphone 10 is used to capture an image of a POF end 120c, which is the other end of the plurality of optical fibers 11b. In this way, connecting an optical fiber to an object to be measured does not necessarily mean that it is directly connected, but may mean that the optical fiber is indirectly connected to the object to be measured via another optical fiber, a camera, the Internet, an information terminal such as a PC, etc.

[0057] In the communication mode of the condition abnormality detection device 1c shown in Fig. 16(4), a camera 10b is used to capture an image of a POF end 120, which is an end of a plurality of optical fibers 11. The image captured by the camera 10b is transmitted to a PC 8 via the Internet 7, and a POF end image 101b is displayed on the display of the PC 8. The PC 8 processes the POF end image 101b using a previously installed application. In this way, by using the PC 8 as an information terminal, it is possible to process the POF end image 101b received via the Internet 7 using an application installed in the PC 8 itself.

[0058] Fig. 17 shows a configuration image diagram of a condition anomaly detection device. As shown in Fig. 17, according to the condition anomaly detection device of the present invention, it is possible to place many different types of cameras (10a to 10c) around the world, and process images captured by the cameras (10a to 10c) in a PC 8 via the Internet (not shown). Here, only one PC 8 is shown, but it is also possible to use multiple PCs 8 or other servers according to the scale of the system, etc. In this way, a system can be created that allows the results of monitoring conducted at any location in the world to be analyzed from anywhere. [Industrial Applicability]

[0059] The present invention can be deployed as a monitoring system in a variety of fields, such as disaster prevention, disaster mitigation, maintenance and management of structures, and other monitoring that is required on a daily basis, for example, science education for children and disaster prevention education. [Explanation of symbols]

[0060] 1, 1a to 1c Condition change detection device 2,2a~2d Object to be measured 3. Optical fiber sensor 4. Cloud Server 5,5a~5e Cell 6 Ground 7. Internet 8 PC 9a~9h Light source 10 Mobile devices (smartphones) 10a~10c,110 Camera 11, 11a, 11b Optical fiber (POF) 12 Positioning jig 50a~50j area 101,101b POF end image 101a Light source image 102 Analysis Data Screen 111 LED 120,120a~120c POF end 121 Marker 122 Placement Panel 123 Cradle 124 holes

Claims

1. A camera means; an information terminal for processing images captured by the camera means; one or more optical fibers connected to the device under test; a positioning jig for overlapping a lens position of the camera means with one end position of the optical fiber; Equipped with the positioning jig is provided with a marker capable of determining the alignment position of the ends of the plurality of optical fibers, and the ends and the marker are fixedly disposed so that the ends of the plurality of optical fibers and the marker are reflected in an image captured by an image sensor of the camera means; The information terminal is a condition change detection device characterized in that the image sensor of the camera means processes the emitted light from each end of the optical fiber, identifies the object to be measured or the measurement location from the position information within the processed image, and detects a change in the condition of the object to be measured from the color information of the processed image.

2. The condition abnormality detection device described in claim 1, characterized in that the information terminal detects color information of a plurality of divided image areas corresponding to the emitted light from each end of the plurality of optical fibers, and detects the condition abnormality from the detected plurality of color information.

3. 3. The condition change detection device according to claim 1, wherein the image sensor processes a real-time image of the emitted light and detects the condition change in real time.

4. The condition change detection device according to claim 1 or 2, characterized in that the information terminal is provided with a tag input means for inputting an identification tag pre-assigned to the optical fiber, and stores the image processed by the image sensor in association with the identification tag.

5. The condition abnormality detection device according to claim 1 or 2, characterized in that the information terminal is provided with a threshold setting means for setting a color threshold for outputting data according to the color information, and the color information is compared with the threshold to output the data.

6. A condition anomaly detection program that causes a computer to perform the following steps: an image input step of inputting an image obtained by image processing of light emitted from one or more optical fibers connected to the object to be measured by an image sensor of the camera means in a state where one end position of the optical fibers is superimposed on a lens position of the camera means; a tag input step of inputting a pre-assigned identification tag for the optical fiber; a threshold setting step of setting a color threshold for outputting data on a change in a state of the object to be measured according to color information of the image; a tag association step of associating an input image with the identification tag; a data output step of comparing color information of the input image with the threshold value and outputting the data; a marker detection step of detecting a marker capable of determining an alignment position of the ends of the plurality of optical fibers from the input image; A color information detection step of detecting color information of a plurality of divided image regions corresponding to the light emitted from each end of the plurality of optical fibers based on the positions of the markers.

7. The condition abnormality detection device according to claim 2, characterized in that in the captured image, the area that is the subject of condition abnormality detection is an image area among the divided image areas that is previously associated with the object to be measured.

8. 3. The condition abnormality detection device according to claim 2, wherein an area in the captured image that is a target of condition abnormality detection is a specific area among the divided image areas.

9. The condition abnormality detection device according to claim 1 or 2, characterized in that in the captured image, the area that is the subject of condition abnormality detection is an area that is within a range of a predetermined pixel value in a predetermined light band.

Citation Information

Patent Citations

  • Vision inspecting device

    JP1992349643A

  • Scope unit for portable terminal

    JP2005192583A

  • Deformation in-situ display device of natural and artificial structure by light color

    JP2008309784A

  • Measuring method and device of analyte

    JP2018105886A

  • Imaging method for surface color and liquid contact angle

    JP2021535373A