Information processing device, information processing method, and program

Abstract

To provide an information processing device, an information processing method, and a program suitable for measuring the length of an object having a repeating pattern. [Solution] An information processing device comprising: an image acquisition unit that acquires images of an object having a continuous repeating pattern along a predetermined direction, the first image being taken of the object before a change in the measurement target section, and the second image being taken of the object after a change in the measurement target section; an image analysis unit that measures the number of repeating patterns included in the measurement target section of the first image and the number of repeating patterns included in the measurement target section of the second image by image analysis; and a length calculation unit that calculates the amount of change in the measurement target section from the difference between the number of repeating patterns measured in the first image and the number of repeating patterns measured in the second image.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program that can be used for measuring the length of a predetermined object.

Background Art

[0002] In order to confirm whether work on a predetermined member (for example, manufacturing, processing, installation, fixing, etc. of the member) has been properly performed, the length of the member may be measured. For example, in Patent Document 1, a technique is disclosed in which, for quality inspection of rolled materials and cut materials in a manufacturing line, the end portions of the member are detected using image analysis and it is determined whether the length is within a standard range.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art such as Patent Document 1, a length measurement method is used in which the end portions of the member to be measured are detected from an image and the length of the member is calculated. In such a method, in order to convert the distance between the end portions detected on the image into the physical length in actual dimensions, correction processing such as calibration and scaling correction based on the shooting distance, shooting angle, reference scale information, etc. is required. However, in the actual shooting environment, due to fluctuations in the shooting position, the distance from the subject, the camera angle, etc., and the difficulty of installing the reference scale, accurate and stable conversion processing is not always easy, and an error is likely to occur in the length measurement result. Therefore, a technique for measuring the length of an object more accurately and efficiently than in the past has been demanded.

[0005] An objective of the exemplary embodiments of this disclosure is to provide an information processing device, an information processing method, and a program suitable for measuring the length of an object having a repeating pattern. [Means for solving the problem]

[0006] An information processing device relating to one aspect of this disclosure is: A shooting data acquisition unit that captures images of an object having a continuous repeating pattern along a predetermined direction, and acquires a first image capturing the state of the object before a change in the measurement target section, and a second image capturing the state after a change in the measurement target section. An image analysis unit measures the number of repeating patterns included in the measurement target section of the first image and the number of repeating patterns included in the measurement target section of the second image by image analysis. The system includes a length calculation unit that calculates the amount of change in the measurement target section from the difference between the number of repeating patterns measured in the first image and the number of repeating patterns measured in the second image.

[0007] According to the above-described information processing device, the length of an object with a repeating pattern, and its changes, can be calculated accurately and efficiently. In particular, the object can be efficiently measured in length without having to read a reference scale to convert the distance on the image to actual dimensions, or to perform corrections according to shooting conditions such as shooting distance and shooting angle.

[0008] Further details regarding the problems disclosed in this application and their solutions will be made clear in the section on embodiments and drawings of this disclosure. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram illustrating the hardware configuration of an information processing device according to one embodiment of the present disclosure. [Figure 2] Figure 2 is a block diagram illustrating the functions of the information processing device shown in Figure 1. [Figure 3]Figure 3 is a schematic diagram illustrating a conventional example of verification work performed when attaching a specific component to an overhead line cable. [Figure 4] Figure 4 is a schematic diagram illustrating an example of preprocessing to identify the measurement target section in an image. [Figure 5] Figure 5 is a schematic diagram illustrating an example of image analysis for calculating the number of repeating patterns. [Figure 6] Figure 6 is a schematic diagram illustrating another example of image analysis for calculating the number of repeating patterns. [Figure 7] Figure 7 is a flowchart showing an example of information processing performed by the information processing device shown in Figure 1. [Modes for carrying out the invention]

[0010] An information processing system according to one embodiment of this disclosure will be described with reference to the drawings. In the attached drawings, identical or similar elements are given identical or similar reference numerals and names, and redundant descriptions of identical or similar elements may be omitted in the description of the embodiment. The contents shown in each drawing are merely examples for explaining this embodiment and are only schematic examples to facilitate explanation of this embodiment. The contents of each drawing may be modified or changed to the extent that no technical problems arise.

[0011] <System Overview> The information processing device 2 according to this embodiment is a calculation device used for measuring the length of an object having a repeating pattern that is continuous along a predetermined direction. Here, "repeating pattern" means a structure in which the same or similar shapes and / or patterns are continuously arranged at regular intervals. Examples of objects having such repeating patterns include objects having a twisted structure such as electric wires, overhead ground wires, wire ropes, and metal braided hoses; objects having grooves and / or ribs such as ribbed reinforcing bars, ribbed steel pipes, screws, and bolts; objects having a corrugated structure such as corrugated steel plates (e.g., deck plates, corrugated steel plates, etc.) and stainless steel bellows pipes; objects having a mesh or grid structure such as metal mesh and gratings; objects having a hole or protrusion row structure such as perforated metal and zippers; objects having a repeating surface pattern such as embossed sheets and anti-slip flooring materials; and objects having a link or joint structure in which small components are repeatedly connected, such as chains, cable carriers, and metal bands. As these examples show, "objects having a repeating pattern that is continuous along a predetermined direction" are not limited to linear bodies, but also include planar bodies, cylindrical bodies, and other three-dimensional structures.

[0012] In this embodiment, the process of attaching a predetermined component to an overhead line cable is used as a specific use case to illustrate the information processing performed by the information processing device 2. However, the use case of an overhead line cable is merely illustrative, and the objects and operations to which the length measurement processing by the information processing device 2 can be applied are not necessarily limited. The information processing device 2 and the information processing performed by the device shown in this embodiment can be applied to operations performed on objects other than the overhead line cable exemplified above (for example, manufacturing, processing, installation, and fixing of components).

[0013] Furthermore, "overhead line cables" refers to all cables used in overhead lines. Overhead line cables are a concept that includes not only power lines that carry out power transmission, but also overhead ground wires installed for the purpose of lightning protection and / or potential stabilization. In addition, these cables are not limited to those that have been actually installed as overhead lines, but also include those in the storage, processing, and installation preparation stages before installation. In this specification, "overhead line cables" will be used in this broad sense. Such overhead line cables are constructed by twisting together multiple strands of wire.

[0014] When installing overhead lines in power transmission facilities and railway facilities, various components such as tension clamps, joints, branch connectors, insulators, and spacers are attached to the overhead line cables. When such mounting components are fixed to the overhead line cables using a prescribed method such as crimping, it is necessary to verify that the mounting components have been properly installed. For example, when attaching a mounting component as shown in Figure 3(a) to the end of a wire by crimping, a verification of the installation by measuring the length is performed. Specifically, a mark is made at a predetermined position on the wire (for example, at a predetermined distance from the end or the planned mounting position of the component), and the mounting component is temporarily placed on the wire (Figure 3(b)). Then, before fixing it by crimping, the length from the end of the mounting component to the mark ("D1 before compression" in Figure 3(b)) is measured. When the mounting component is compressed while in a temporary position and then crimped to the electric wire, the mounting component deforms, as shown in Figure 3(c), causing a change in the length between the mounting component and the marker (in this case, the change is due to the deformation of the mounting component, not the deformation of the electric wire itself). The length from the end of the mounting component to the marker after fixing ("D2 after compression" in Figure 3(c)) is measured, and the amount of change (i.e., the difference between D1 before compression and D2 after compression) is used to determine whether the mounting component has been properly fixed.

[0015] In the above-described work, conventionally, the operator measured the length D1 before fixation and the length D2 after fixation using a scale. However, in the case of manual measurement by the operator, there is a risk of mistakes such as measurement omission and reading errors, and there is also a risk of personal measurement errors and variations. Therefore, in the attachment confirmation work based on such length measurement, an accurate and efficient length measurement technique is required.

[0016] As a method for automating measurement, for example, a technique for automatically measuring the length of an object by acquiring the three-dimensional shape of the object using a distance measurement sensor such as LiDAR (Light Detection and Ranging) is known. However, in the case of LiDAR mounted on a portable terminal (for example, a smartphone, a tablet terminal, etc.) carried by an operator, there is a limit to the measurable distance, and for an object located at a certain distance (for example, an electric wire installed at a high place), it may be impossible to measure in the first place. There are also high-performance LiDARs that can support long-distance measurement, but such high-performance LiDARs have a large device scale and are often difficult to carry to the site and use. In addition, the device cost also tends to be high. Thus, the length measurement technique using LiDAR has problems in terms of both on-site applicability and economy.

[0017] As another method, there is also a method of calculating the amount of change in length by overlapping images before and after the attachment work (before and after the change in length) using a technique such as Homography. However, in this method, it is premised that the shape of the member serving as the reference for determining the measurement target range does not deform. Therefore, in work where the attachment part itself deforms due to compression or the like, it is difficult to accurately overlap the images using the same reference. In addition, the accuracy of the overlap is easily affected by variations in shooting conditions such as the camera angle and shooting distance during shooting, and there are large operational constraints. Therefore, the length measurement technique relying on Homography has problems in terms of versatility, reproducibility, and on-site applicability.

[0018] In the length measurement system using the information processing apparatus 2 of the present embodiment, attention is paid to the repeating pattern provided on the object, and the number of repeating patterns is calculated from the images of the object before and after the work (before and after the change in the length of the measurement target section). For example, in the case of the fixing work of the attachment parts for the aerial cable as described above, a repeating pattern due to the stranded strands appears in the image of the aerial cable. The unit length of the repeating pattern (the length per pattern) is known by measuring it in advance, and the unit length of the repeating pattern does not vary before and after the fixing of the attachment parts.

[0019] In the length measurement method using the information processing apparatus 2, by respectively calculating the number of repeating patterns in the image before the fixing of the attachment parts and the number of repeating patterns in the image after the fixing of the attachment parts by a predetermined image analysis, the length of the object before the work, the length of the object after the work, and the change amount thereof are calculated. In this length measurement method, since the length of the object can be calculated by multiplying the calculated number of repeating patterns by the length per pattern, there is no need for a process of converting the distance based on the number of pixels on the image into actual dimensions. Further, even if the shooting conditions before the work and the shooting conditions after the work are different, it is possible to calculate the length of the object and the change amount thereof from the number of repeating patterns with high precision and stability without requiring correction processing according to the shooting conditions (shooting distance, shooting angle, etc.).

[0020] Hereinafter, the information processing apparatus 2 in the present embodiment and the information processing method using the apparatus will be described in detail.

[0021] <Hardware Configuration of Information Processing Apparatus 2> The information processing device 2 is a terminal held by a user, such as a worker, and may be a smartphone, tablet, dedicated camera terminal, or other general-purpose computer such as a personal computer. The information processing device 2 is equipped with a camera 25 that acquires images of the object to be measured, and is used to acquire images before and after work during work such as attaching components to overhead line cables. The information processing device 2 also has a program installed to realize this system (a program for executing processing related to measurement). In this embodiment, the information processing device 2 is exemplified as a single portable terminal that combines the function of a camera 25 and other imaging device with the function of a computing device that executes the information processing described later, but the information processing device 2 is not limited to this example and may be composed of multiple devices. For example, images may be acquired by an imaging device other than the information processing device 2. Individual devices such as unmanned mobile vehicles (unmanned aerial vehicles (UAVs) such as drones, unmanned ground vehicles (UGVs), etc.) may be used as imaging means, and these individual imaging means may be connected to the computing device in a communication-enabled state.

[0022] As shown in Figure 1, the information processing device 2 includes a processor 20, memory 21, storage 22, a transmitting / receiving unit 23, an input / output unit 24, and a camera 25, etc., which are electrically connected to each other via a bus 27. Note that the configuration of the information processing device 2 shown is just one example, and the information processing device 2 may have other configurations.

[0023] The processor 20 is a computing unit that controls the operation of the entire information processing device 2, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication processing. For example, the processor 20 is a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), and executes the program for this system stored in the storage 22 and loaded into the memory 21 to perform the information processing described below.

[0024] Memory 21 includes main memory, which is composed of a volatile storage device such as DRAM (Dynamic Random Access Memory), and auxiliary memory, which is composed of a non-volatile storage device such as flash memory or HDD (Hard Disk Drive). Memory 21 is used as a work area for the processor 20, and also stores the BIOS (Basic Input / Output System) executed when the information processing device 2 starts up, as well as various setting information.

[0025] The storage 22 stores various programs, such as application programs, for executing the information processing shown in this embodiment. A database containing data used for each information processing may be built in the storage 22. For example, the storage unit 220 described later may be provided in a part of the storage area of ​​the memory 21 and / or the storage 22.

[0026] The transmitting / receiving unit 23 is a communication interface for connecting the information processing device 2 to a network. The transmitting / receiving unit 23 may also be equipped with Bluetooth® and BLE (Bluetooth Low Energy) short-range communication interfaces.

[0027] The input / output unit 24 consists of input devices such as keyboards and mice, and output devices such as displays. In the information processing device 2, the input devices and output devices may be mounted on the information processing device 2 as separate devices, or a touch panel display that combines the functions of both input and output devices may be mounted on the information processing device 2.

[0028] As described above, camera 25 is a means for capturing images of objects such as overhead line cables, and may have an image sensor. Bus 27 is connected in common to all of the above elements and transmits, for example, address signals, data signals, and various control signals. In addition to the above elements, information processing device 2 may be equipped with various sensors such as inertial sensors (e.g., IMU (Inertial Measurement Unit), acceleration sensors, gyro sensors, etc.), GPS sensors, and distance sensors (e.g., LiDAR, etc.).

[0029] <Functions (Software Configuration) of Information Processing Device 2> Figure 2 is a block diagram illustrating the functions implemented in the information processing device 2. The information processing device 2 comprises a processing unit 200 and a storage unit 220. The processing unit 200 may also include a shooting data acquisition unit 201, a pre-processing unit 202, an image analysis unit 203, a length calculation unit 204, and a determination unit 205. The various functions of the processing unit 200 are realized, for example, by the processor 20 reading data and programs stored in the storage 22 and executing various programs in the working area of ​​the memory 21.

[0030] The storage unit 220 stores various information necessary for the processing unit 200 to perform each function. For example, the storage unit 220 may store information about the object to be photographed and / or measured, information about the work performed on the object, information about the analysis, various acquired data, information about the user, and history data (past work history).

[0031] Information regarding the object may include, for example, a reference pattern showing the structure and / or pattern of a standard repeating pattern on the object, and reference data such as the length of each repeating pattern. Other information may include information indicating the attributes or characteristics of the object, and information regarding markers, parts, etc. associated with the object. Information regarding the work may include information indicating the content of the work performed on the object, the work procedure, and judgment criteria (thresholds, etc.) and judgment rules for determining whether the work is successful or not based on the measurement results. Information regarding the analysis may include predefined analysis rules, parameters, reference images for segmentation, and machine learning datasets for image analysis. Various acquired data may include, for example, image data of the object, and such image data may include metadata such as the date and time of shooting and shooting conditions.

[0032] The image data acquisition unit 201 acquires image data including the object. Specifically, the image data acquisition unit 201 acquires images of an object having a continuous repeating pattern along a predetermined direction, which include a first image capturing the state of the object before the change in the measurement target section and a second image capturing the state after the change in the measurement target section.

[0033] Here, the "measurement target section" in an object refers to an area that is at least a part of the object to be measured in length and is demarcated by one or more of the following: parts attached to the object, markers, the ends of the object itself, or predetermined parts possessed by the object itself. The measurement target section and the ends that demarcate the section are set as appropriate according to predetermined work such as manufacturing, processing, installation, and fixing, and are not particularly limited. For example, if the object is an overhead line cable in which multiple strands are twisted together, the ends that demarcate the measurement target area may be parts attached to the overhead line cable and / or markers. In the case of a work example where an attachment part 502, such as a tension clamp, is crimped onto the end of an overhead line cable 501, as shown in image 50 of Figure 4(a), the area between the mark 503, which is marked on the overhead line cable 501 with tape or ink, and the end of the attachment part 502 becomes the measurement target section 504 (that is, one end of the measurement target section 504 is the attachment part 502, and the other end is the mark 503).

[0034] This system takes images of the object, including at least the measurement target section, before and after performing a predetermined operation. The image taken before the operation, which captures the state of the measurement target section before any change, will be referred to as the "first image." The image taken after the operation, which captures the state of the measurement target section after any change, will be referred to as the "second image." In the case of overhead line cable work, the first image is an image taken before the mounting part 502 is fixed, and the second image is an image taken after the mounting part 502 is fixed. "Change in the measurement target section" refers to a change in the apparent length of the measurement target section before and after the operation. In some cases, the apparent length of the measurement target section may change due to deformation of the object itself, while in other cases, the apparent length of the measurement target section may change due to a change in the position of the part that divides the section, or a change in the position of the end dividing the section due to deformation of the part, without deformation of the object itself. The case of overhead line cable work falls under the latter category, where the length of the measurement target section 504 changes as the mounting component 502 compresses and deforms, without the overhead line cable 501 deforming.

[0035] The first image capture, the installation of the mounting component 502, and the second image capture may be performed at a high location where the overhead line cable 501 is installed (for example, between supports such as transmission towers and utility poles), or at a location other than the installation site of the overhead line cable 501, such as on the ground. The first and second images may be still images or images of predetermined frames extracted from moving images. Each image may be captured by the camera 25 of the information processing device 2, or by a means of capture other than the information processing device 2. In the latter case, the image data acquisition unit 201 acquires each image data by any communication means or by an image upload operation by the user.

[0036] When capturing an image, the image data acquisition unit 201 may output guide or instruction information to the display unit of the information processing device 2 to prompt the capture of an object including the measurement target section. The image data acquisition unit 201 may also acquire accompanying information corresponding to the acquired image (for example, metadata such as the time of shooting, photographer, ID of the terminal used, and shooting conditions (focal length, exposure, resolution, etc.)) together with the image, and store the image and its accompanying information in the storage unit 220 in association.

[0037] The preprocessing unit 202 performs predetermined preprocessing on the first and second images. The predetermined preprocessing is not necessarily limited. For example, the preprocessing unit 202 may perform preprocessing to correct each image to a state suitable for subsequent image analysis processing by the image analysis unit 203, such as preprocessing for noise reduction, such as image reduction; image smoothing using a predetermined filter such as a smoothing filter (e.g., a Gaussian filter); resolution correction; brightness correction; and color space conversion (e.g., image binarization).

[0038] Furthermore, the preprocessing unit 202 may perform processing to identify the measurement target section in each image. In this case, the preprocessing unit 202 identifies the region located between the partition sections as the measurement target section by identifying partition sections in the image that correspond to both ends of a section, such as markers, parts, or parts or ends of the object itself. As a method for identifying the partition sections, segmentation processing may be used, or a machine learning model that has been previously trained on images of partition sections may be used. For example, the preprocessing unit 202 identifies the measurement target section 504 in each image by segmenting the first and second images, etc., to identify the end that partitions the measurement target section and is attached to the overhead line cable 501, namely the part 502 and / or marker 503, from the first and second images (Figure 4(c)). The preprocessing unit 202 may extract the identified measurement target section 504 as the analysis region AD.

[0039] When performing multiple preprocessing steps, the order in which these steps are performed is not particularly limited. For example, the preprocessing unit 202 may first perform a process to identify the measurement target section, extract the measurement target section as the analysis region AD, and then perform image correction processes on the analysis region AD, such as noise reduction by image reduction and smoothing using a smoothing filter. Alternatively, the process may be performed in the reverse order, with image correction processes being performed first, followed by the extraction of the measurement target section.

[0040] The image analysis unit 203 measures the number of repeating patterns in the measurement target section of the first image and the number of repeating patterns in the measurement target section of the second image by performing image analysis. In the case of the overhead line cable 501, as shown in Figure 4(c), the repeating pattern is the pattern of twisted strands 5010. The image analysis unit 203 measures the number of strands 5010 in the measurement target section 504 of the first image and the number of strands 5010 in the measurement target section 504 of the second image. The image analysis method for measuring the number of repeating patterns is not necessarily limited. For example, a method of analyzing the brightness distribution or an analysis method using pattern matching may be employed.

[0041] In the case of the luminance distribution analysis method, the image analysis unit 203 extracts the luminance distribution on a linear vector set along a predetermined direction in the measurement target section of the first and second images, and measures the number of waves in the luminance distribution as the number of repeating patterns. A "linear vector" is a straight analysis line along the direction in which the repeating pattern continues. The image analysis unit 203 may set the linear vector based on user input operations specifying the analysis line on the measurement target section of each image. Alternatively, the image analysis unit 203 may automatically set the linear vector by detecting the direction in which the repeating pattern continues or by identifying the longitudinal direction of the measurement target section extracted by the preprocessing unit 202. At least one linear vector needs to be set, but from the viewpoint of improving measurement accuracy, it is preferable to set multiple linear vectors in the measurement target section of each image.

[0042] Figure 5(a) is a schematic diagram illustrating how multiple linear vectors are set in the measurement target section 504 of the image 50 (first image and second image) of the overhead line cable 501. In Figure 5(a), five linear vectors are set, indicated by dashed lines L1-L5. When setting only a single linear vector, the linear vector may be set to pass approximately through the center in a direction perpendicular to the longitudinal direction of the overhead line cable 501 (dashed line L3 in Figure 5(a)). The image analysis unit 203 extracts the luminance distribution on the linear vector based on the luminance value of each pixel on the linear vector.

[0043] In image 50 of the overhead cable 501, the luminance value is relatively low at the boundaries between the strands 5010, and relatively high near the center of each strand 5010 in the radial direction (left-right direction in the drawing). Therefore, the luminance distribution of the linear vector appears as a waveform graph as shown in Figure 5(b). The luminance distribution shown by the thick black solid line in Figure 5(b) is the luminance distribution on the dashed line L3 in Figure 5(a). This luminance distribution has eight local maximums indicated by white circles and nine local minimums indicated by gray circles. The local maximums correspond to the vicinity of the radial center of the strands 5010, and the local minimums correspond to the boundaries between the strands 5010.

[0044] The image analysis unit 203 measures the number of sign change points (i.e., the number of extreme value points, which is the sum of the maximum and minimum points) by differentiating the luminance distribution. Half the number of sign change points obtained by differentiation (the number of extreme value points divided by 2) corresponds to the number of waveforms in the luminance distribution, and the image analysis unit 203 calculates this number of waveforms in the luminance distribution as the number of repeating patterns in the measurement target section. In the case of the luminance distribution of the dashed line L3 in Figure 5(b), there are 17 sign change points, so the number of repeating patterns is calculated to be 8.5.

[0045] If multiple linear vectors are set for each image, the image analysis unit 203 calculates the average number of waves in the brightness distribution extracted on each linear vector as the number of repeating patterns included in the measurement target section of each image. Other statistical values ​​such as the maximum, minimum, or median may be used instead of the average. The number of repeating patterns corresponding to the number of waveforms may be expressed as a floating-point number, or as a natural number rounded to the first decimal place. The output format for the number of repeating patterns may be set according to the length of each pattern; for example, if the unit length is greater than a predetermined threshold, a floating-point number may be used, and if it is less than the predetermined threshold, a natural number calculated by rounding may be used.

[0046] Note that while Figure 5(b) shows a conceptualized luminance distribution with a well-defined waveform for clarity, the actual extracted luminance distribution may be affected by the shooting environment, such as light source reflection, and / or shape errors of the object, resulting in disturbances and noise in the waveform. To further improve measurement accuracy, the image analysis unit 203 may apply a Fast Fourier Transform to the luminance distribution to smooth the graph and extract the luminance distribution as a linear vector.

[0047] In the pattern matching analysis method, the image analysis unit 203 performs pattern matching to calculate the similarity between the measurement target section captured in the first and second images and a reference pattern, continuously along a predetermined direction. The reference pattern is data that shows the structure and / or pattern of a standard repeating pattern in the object, for example, Figure 6(a) is an example of a reference pattern RP for an overhead line cable 501. In pattern matching, for example, as shown in Figure 6(b), the reference pattern RP is superimposed on the measurement target section of the image and sequentially slid along a predetermined direction, and the degree of agreement between the reference pattern RP and the image at each position is quantified. This quantified degree of agreement is calculated as the similarity. For quantifying the similarity, known similarity evaluation indices such as normalized cross-correlation (NCC), mean squared error (MSE), and structural similarity index (SSIM) may be used. In the case of pattern matching, as in the analysis for extracting the luminance distribution described above, one or more linear vectors aligned in a predetermined direction may be set for the measurement target interval. Then, the similarity on each linear vector may be calculated.

[0048] The image analysis unit 203 performs pattern matching as described above to calculate a graph showing the change in similarity to a reference pattern in the measurement target section, and measures the number of waves in the graph as the number of repeating patterns. For example, Figure 6(c) is an example of a graph showing the change in similarity in the measurement target section of an image (the change in similarity along the longitudinal direction of the measurement target section). The graph in Figure 6(c), like the luminance distribution in Figure 5(b), has eight local maximums indicated by white circles and nine local minimums indicated by gray circles.

[0049] In pattern matching analysis, the number of sign change points (i.e., the number of extreme value points, which is the sum of the maximum and minimum points) is measured by differentiating the similarity graph. Half of the number of sign change points obtained by differentiation corresponds to the number of waveforms in the similarity graph, and the image analysis unit 203 calculates this number of waveforms as the number of repeating patterns in the measurement target interval. In pattern matching, the number of repeating patterns may also be calculated on multiple linear vectors, and statistical values ​​such as the average value may be calculated. Alternatively, the image analysis unit 203 may apply a Fast Fourier Transform to the similarity graph calculated by pattern matching, smooth the graph, and then measure the number of waves in the graph.

[0050] The length calculation unit 204 calculates the length of the measurement target section based on the number of repeating patterns in the measurement target section of each image identified by the image analysis unit 203. Specifically, by multiplying the number of repeating patterns in each image by the unit length (length per pattern) which has been measured in advance and stored in the storage unit 220, the actual length of the measurement target section in each image can be calculated. This process does not require converting the length in pixels to actual dimensions, and robust length measurement can be achieved against changes in shooting conditions such as shooting distance and shooting angle.

[0051] Furthermore, the length calculation unit 204 calculates the change in the measurement target interval from the difference between the number of repeating patterns measured in the first image and the number of repeating patterns measured in the second image. In this case as well, the change in the measurement target interval can be calculated by multiplying the difference between the number of repeating patterns in the first image and the number of repeating patterns in the second image by the unit length (length per pattern). Note that for the change before and after the operation, the change may be calculated by multiplying the difference between the length of the measurement target interval calculated in the first image and the length of the measurement target interval calculated in the second image by the unit length. If the number of repeating patterns is statistically calculated based on multiple linear vectors, the length and change can be calculated by multiplying the statistical value (e.g., the mean value) by the unit length.

[0052] In the case of the overhead line cable 501, the length calculation unit 204 calculates the change in the length of the overhead line cable visible in the measurement target section from the difference between the number of strands measured in the first image and the number of strands measured in the second image. Here, the length of the overhead line cable visible in the measurement target section refers to the apparent length of the cable that is visible in the measurement target section, and the change in this case refers to the change due to the deformation of the mounting component 502, not the change due to the deformation of the overhead line cable 501. That is, the length calculation unit 204 calculates the change in the length of the overhead line cable 501 visible in the measurement target section, which occurs when the position of at least one end of the measurement target section changes due to the fixing of the mounting component 502, from the difference between the number of strands measured in the first image and the number of strands measured in the second image.

[0053] In the above example, the length of the measurement target section in each image can be calculated by multiplying the number of strands identified by image analysis by the unit length of the repeating pattern formed by the strand twisting structure. Similarly, the amount of change in the length of the measurement target section can be calculated by multiplying the difference between the number of strands in the first image and the number of strands in the second image by the unit length of the repeating pattern. In this example, since the structure and length of the overhead line cable 501 itself hardly change, the unit length of the repeating pattern does not change before and after fixing the component and can be used as a single reference data value.

[0054] In cases where the object itself deforms during the process, multiple repeating patterns with different unit lengths may be prepared in advance, depending on the degree and / or direction of deformation. In this case, the unit length used to calculate the amount of change may be selected to an appropriate value according to the working conditions.

[0055] The determination unit 205 determines whether the work performed on the object is appropriate based on the amount of change in the length of the measurement target section calculated by the length calculation unit 204. The appropriateness of the work is determined according to the determination rules stored in the storage unit 220 in advance. The determination rules may include conditions such as whether the amount of change in length before and after the work has reached a predetermined threshold, or whether it is within a predetermined threshold range. The determination rules may also define a judgment criterion for determining the appropriateness of the work based on whether or not there is a change in length before and after the work. For example, in the case of attaching parts to overhead line cables, a judgment criterion may be defined that determines the work is acceptable if the length after the work is shorter than the length of the measurement target section before the work.

[0056] For example, in the crimping work of attachment parts 502 to overhead line cables 501, the fact that the length of the measurement target section 504 is shortened within a predetermined range as a result of the crimping serves as an indicator of the proper execution of the work. For such work, the judgment unit 205 may determine whether the amount of change in length calculated from images before and after crimping falls within a set standard range, and generate a judgment result such as "normal" if it is within the standard range, or "improper" if it is outside the range.

[0057] The judgment unit 205 may output the judgment result to the display unit 2 of the information processing device 2 so that the user (worker or verifier) ​​can understand the work result. Alternatively, the judgment unit 205 may register the judgment result as record data in the storage unit 220 so that the record data of the judgment result can be used for work performance reporting, quality control, etc. The judgment unit 205 may also perform a process to notify the judgment result to an administrator terminal, a quality control system, or an external device on the cloud. As a means of notification, transmission via a wired or wireless communication network may be used, and various notification formats such as email, API integration, and automatic reflection to a dashboard may be adopted.

[0058] <An example of an information processing method> Next, an example of an information processing method using the information processing device 2 of this embodiment will be explained based on the flowchart shown in Figure 7.

[0059] First, the worker takes a photograph of the object's condition before performing the prescribed work on it (Step SQ101). Examples of objects include overhead line cables, and the photograph must include the section to be measured. Next, the worker performs the prescribed work on the overhead line cable, such as crimping and securing attachments like tension clamps (Step SQ102). After the work is completed, the worker takes another photograph of the object's condition (Step SQ103).

[0060] The two captured images are acquired by the image data acquisition unit 201 as the first image (before work) and the second image (after work) (step SQ104). At this time, the shooting conditions and other information such as the shooting time corresponding to each image are also acquired and recorded in the storage unit 220 as necessary. The preprocessing unit 202 performs predetermined preprocessing such as smoothing, filtering, and segmentation on the first and second images to identify the measurement target section in each image (step SQ105). The measurement target section is, for example, the section located between a mark provided on the cable and the end of the mounting part.

[0061] Next, the image analysis unit 203 measures the number of repeating patterns included in each identified measurement target section by image analysis (step SQ106). Repeating patterns are shapes and / or patterns that appear periodically on the object, such as the structure of twisted strands. As image analysis methods, for example, waveform analysis based on brightness distribution (Figure 5) or pattern matching based on similarity with a reference pattern (Figure 6) can be employed.

[0062] The length calculation unit 204 calculates the change in the length of the measurement target section by multiplying the difference in the number of repeating patterns included in the measurement target section of the first image and the second image by the unit length (length per pattern) stored in the storage unit 220. This makes it possible to efficiently and accurately determine the change in the measurement target section before and after the work, without requiring calibration or scaling processing to convert to the actual dimensions of the object. Alternatively, the length of the measurement target section may be calculated individually by multiplying the number of repeating patterns in each image by the unit length, and then the change is derived from the difference between them. In either case, more stable measurement is possible by calculating the length and its change by multiplying the unit length by the statistical processing result (e.g., average value) based on multiple linear vectors.

[0063] Finally, the determination unit 205 uses the calculated change amount to determine whether the work was performed correctly (step SQ108). The determination is made by referring to predetermined determination rules, thresholds, etc., stored in the storage unit 220. The determination unit 205 may output the determination result to the display unit of the information processing device 2, save it as recorded data, or notify a management terminal or external system.

[0064] Note that the information processing method flow shown in Figure 7 is merely an example, and steps may be added, modified, or rearranged as appropriate.

[0065] According to the information processing device 2 of this embodiment, the number of repeating patterns in the measurement target section of an object can be calculated by image analysis from images of the object before and after the work, and the amount of change in the measurement target section can be calculated based on the difference. This eliminates the need for workers to manually measure the length using scales or the like, as in the past, and reduces the risk of human errors such as forgetting to measure or reading errors. It also makes it possible to suppress human error and measurement variability.

[0066] Furthermore, in the system using the information processing device 2 of this embodiment, the length of the object and the amount of change in its length are determined by multiplying the length per pattern (unit length), which has been measured in advance, by the number of repeating patterns calculated by image analysis. With this configuration, calibration processing to convert the distance in pixels on the image to actual dimensions is unnecessary. In addition, even if the shooting conditions (shooting distance, shooting angle, etc.) differ before and after the work, the amount of change in length can be measured accurately and stably without performing error correction processing due to the shooting distance and shooting angle. In particular, in work in which mounting parts are deformed by compression, etc., the change in length caused by the change in the position of the end of the part can be accurately captured through the change in the number of repeating patterns.

[0067] Furthermore, the information processing device 2 can automatically determine whether the work was performed appropriately according to a predetermined standard based on the measured change in length. For example, in the crimping work of attachment parts to overhead line cables, the pass / fail status of the work can be evaluated based on whether the measured section of the cable has been shortened by a predetermined range. In this way, the information processing device 2 of this embodiment contributes to the efficiency of quality evaluation and recording of work content, and realizes advanced and labor-saving work management at the site.

[0068] The embodiments described above are merely illustrative to facilitate understanding of this disclosure and are not intended to limit it. This disclosure may be modified and improved without departing from its intent, and its equivalents are included.

[0069] <Variation> For example, the above embodiment mainly illustrates the attachment work (crimping work) of overhead line cables and components to those cables, but the application examples of this disclosure are not limited to these cases. For example, this disclosure can be applied to butt welding work of ribbed reinforcing bars. The ribs (projections) provided on the reinforcing bars have a repeating structure at predetermined intervals, and this can be treated as a repeating pattern. By performing the image analysis processing of the present invention on images before and after welding, the amount of change in the length of the reinforcing bar near the weld can be measured. In particular, if the length changes due to the amount of indentation, etc., as a result of welding the reinforcing bars at both butt ends, the amount of change can be quantitatively grasped. Based on such measurement results, it is possible to automatically evaluate whether appropriate welding has been performed. Furthermore, this disclosure may also be applied to the work of inspecting bolt looseness. In this application example, for example, the helical structure of the threaded portion provided on the bolt (helical pattern due to the projections of the threads) may be treated as a repeating pattern, and the number of threads protruding from the nut may be measured by the system of this disclosure to determine whether or not the bolt is loose.

[0070] Furthermore, while the above embodiments exemplified image analysis methods such as waveform analysis of luminance distribution and pattern matching, the image analysis methods are not limited to these examples. For example, machine learning models (such as image classifiers or object detection models using CNNs (Convolutional Neural Networks)) may be used.

[0071] The information processing device 2 described in the above embodiment may be implemented as a single device, or it may be implemented by multiple devices (e.g., cloud servers) that are partially or entirely connected by a network. For example, the functions of the processor 20 and storage 22 of the information processing device 2 may be implemented by different computing devices that are connected to each other by a network.

[0072] Furthermore, the series of processes performed by the information processing device 2 described herein may be implemented using software, hardware, or a combination of software and hardware. Each function of the information processing device 2 may be configured to be executed by a server device such as a web server or cloud server. A computer-readable recording medium containing computer programs for implementing the above functions can also be provided. Examples of recording media include magnetic disks, optical disks, magneto-optical disks, and flash memory. The above computer programs may also be distributed without using a recording medium, for example, via a network.

[0073] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that are obvious to those skilled in the art from the description herein, in addition to or in lieu of the effects described herein.

[0074] The information processing apparatus, information processing method, and program disclosed herein have, for example, the following configuration. [Item 1] A shooting data acquisition unit that captures images of an object having a continuous repeating pattern along a predetermined direction, and acquires a first image capturing the state of the object before a change in the measurement target section, and a second image capturing the state after a change in the measurement target section. An image analysis unit measures the number of repeating patterns included in the measurement target section of the first image and the number of repeating patterns included in the measurement target section of the second image by image analysis. An information processing device comprising: a length calculation unit that calculates the amount of change in the measurement target section from the difference between the number of repeating patterns measured in the first image and the number of repeating patterns measured in the second image. [Item 2] The information processing device according to item 1, wherein the image analysis unit extracts a luminance distribution on a linear vector set along a predetermined direction in the measurement target section of the first image and the second image, and measures the number of waves in the luminance distribution as the number of repeating patterns. [Item 3] The aforementioned image analysis unit, In the measurement target intervals of the first and second images, a plurality of the linear vectors are set, The information processing device according to item 2, which calculates the average number of waves in the luminance distribution extracted on each linear vector as the number of repeating patterns included in the measurement target section of each image. [Item 4] The image analysis unit is an information processing device according to item 2, which applies a fast Fourier transform to extract the brightness distribution on the linear vector. [Item 5] The aforementioned image analysis unit, The information processing device according to item 1, wherein pattern matching is performed on the measurement target section captured in the first image and the second image, calculating the similarity to a reference pattern continuously along the predetermined direction, thereby calculating a graph showing the change in similarity to a reference pattern in the measurement target section, and the number of waves in the graph is measured as the number of repeating patterns. [Item 6] The image analysis unit applies a Fast Fourier Transform to the graph calculated by the pattern matching and measures the number of waves in the graph, as described in item 5. [Item 7] The aforementioned object is an overhead line cable in which multiple strands are twisted together. The repeating pattern is the pattern of the twisted strands, The image analysis unit measures the number of strands included in the measurement target section of the first image and the number of strands included in the measurement target section of the second image. The length calculation unit calculates the amount of change in the length of the overhead line cable visualized in the measurement target section from the difference between the number of strands measured in the first image and the number of strands measured in the second image, according to any one of items 1 to 6. [Item 8] The information processing device according to item 7, further comprising a preprocessing unit that identifies the measurement target section in each image by segmenting the first image and the second image to identify the end portion that divides the measurement target section and is attached to the overhead line cable from among the first image and the second image. [Item 9] At least one end of the measurement target section is a mounting component attached to the overhead line cable, The first image above is a photograph taken before the mounting part was fixed in place. The second image above is a photograph taken after the mounting part has been fixed in place. The length calculation unit calculates the amount of change in the length of the overhead line cable visualized in the measurement target section, which occurs when the position of at least one end of the measurement target section changes due to the fixing of the mounting component, based on the difference between the number of strands measured in the first image and the number of strands measured in the second image. This information processing device is as described in item 7. [Item 10] The method involves capturing images of an object having a continuous repeating pattern along a predetermined direction, and obtaining a first image capturing the state of the object before a change in the measurement target section, and a second image capturing the state after a change in the measurement target section. By image analysis, the number of repeating patterns included in the measurement target section of the first image and the number of repeating patterns included in the measurement target section of the second image are measured. An information processing method in which a computer performs the following steps: calculating the amount of change in the measurement target interval from the difference between the number of repeating patterns measured in the first image and the number of repeating patterns measured in the second image. [Item 11] The method involves capturing images of an object having a continuous repeating pattern along a predetermined direction, and obtaining a first image capturing the state of the object before a change in the measurement target section, and a second image capturing the state after a change in the measurement target section. By image analysis, the number of repeating patterns included in the measurement target section of the first image and the number of repeating patterns included in the measurement target section of the second image are measured. A program to cause a computer to perform the following: calculate the amount of change in the measurement target interval from the difference between the number of repeating patterns measured in the first image and the number of repeating patterns measured in the second image. [Explanation of Symbols]

[0075] 2. Information Processing Device 201 Shooting Data Acquisition Unit 203 Image Analysis Department 204 Length Calculation Unit

Claims

1. A shooting data acquisition unit that captures images of an object having a continuous repeating pattern along a predetermined direction, and acquires a first image capturing the state of the object before a change in the measurement target section, and a second image capturing the state after a change in the measurement target section. An image analysis unit measures the number of repeating patterns included in the measurement target section of the first image and the number of repeating patterns included in the measurement target section of the second image by image analysis. The system includes a length calculation unit that calculates the amount of change in the measurement target section from the difference between the number of repeating patterns measured in the first image and the number of repeating patterns measured in the second image, The object in question is an electric wire in which multiple strands are twisted together. The repeating pattern is the pattern of the twisted strands, The image analysis unit measures the number of strands included in the measurement target section of the first image and the number of strands included in the measurement target section of the second image. The length calculation unit is an information processing device that calculates the amount of change in the length of the electric wire visualized in the measurement target section from the difference between the number of strands measured in the first image and the number of strands measured in the second image.

2. The information processing apparatus according to claim 1, wherein the image analysis unit extracts a luminance distribution on a linear vector set along a predetermined direction in the measurement target section of the first image and the second image, and measures the number of waves in the luminance distribution as the number of repeating patterns.

3. The aforementioned image analysis unit, In the measurement target intervals of the first and second images, a plurality of the linear vectors are set, The information processing apparatus according to claim 2, wherein the average value of the number of waves in the luminance distribution extracted on each linear vector is calculated as the number of repeating patterns included in the measurement target section of each image.

4. The information processing apparatus according to claim 2, wherein the image analysis unit applies a fast Fourier transform to extract the brightness distribution on the linear vector.

5. The aforementioned image analysis unit, The information processing apparatus according to claim 1, wherein pattern matching is performed on the measurement target section captured in the first image and the second image, calculating the similarity to a reference pattern continuously along the predetermined direction, thereby calculating a graph showing the change in similarity to a reference pattern in the measurement target section, and the number of waves in the graph is measured as the number of repeating patterns.

6. The information processing apparatus according to claim 5, wherein the image analysis unit applies a Fast Fourier Transform to the graph calculated by the pattern matching and measures the number of waves in the graph.

7. The information processing apparatus according to claim 1, further comprising a preprocessing unit that identifies the measurement target section in each image by segmenting the first image and the second image to identify the end portion that divides the measurement target section and which is attached to the electric wire, from among the first image and the second image.

8. At least one end of the measurement target section is a mounting component attached to the electric wire, The first image above is a photograph taken before the mounting part was fixed in place. The second image above is an image taken after the mounting part has been fixed in place. The information processing apparatus according to claim 1, wherein the length calculation unit calculates the amount of change in the length of the electric wire visualized in the measurement target section, which occurs when the position of at least one end of the measurement target section changes due to the fixing of the mounting component, based on the difference between the number of strands measured in the first image and the number of strands measured in the second image.

9. The method involves capturing images of an object having a continuous repeating pattern along a predetermined direction, and obtaining a first image capturing the state of the object before a change in the measurement target section, and a second image capturing the state after a change in the measurement target section. By image analysis, the number of repeating patterns included in the measurement target section of the first image and the number of repeating patterns included in the measurement target section of the second image are measured. This is an information processing method in which a computer performs the following steps: calculate the amount of change in the measurement target interval from the difference between the number of repeating patterns measured in the first image and the number of repeating patterns measured in the second image. The object in question is an electric wire in which multiple strands are twisted together. The repeating pattern is the pattern of the twisted strands, In the aforementioned image analysis, the number of strands included in the measurement target section of the first image and the number of strands included in the measurement target section of the second image are measured. An information processing method in which the computer performs the process of calculating the amount of change in the measurement target section, wherein the computer calculates the amount of change in the length of the electric wire visualized in the measurement target section from the difference between the number of strands measured in the first image and the number of strands measured in the second image.

10. The method involves capturing images of an object having a continuous repeating pattern along a predetermined direction, and obtaining a first image capturing the state of the object before a change in the measurement target section, and a second image capturing the state after a change in the measurement target section. By image analysis, the number of repeating patterns included in the measurement target section of the first image and the number of repeating patterns included in the measurement target section of the second image are measured. This is a program that causes a computer to perform the following steps: calculate the amount of change in the measurement target interval from the difference between the number of repeating patterns measured in the first image and the number of repeating patterns measured in the second image. The object in question is an electric wire in which multiple strands are twisted together. The repeating pattern is the pattern of the twisted strands, In the aforementioned image analysis, the number of strands included in the measurement target section of the first image and the number of strands included in the measurement target section of the second image are measured. A program that causes the computer to perform the process of calculating the amount of change in the measurement target section, which involves calculating the amount of change in the length of the electric wire visualized in the measurement target section from the difference between the number of strands measured in the first image and the number of strands measured in the second image.

Images