Condition determination system and condition determination method.

Abstract

To improve the efficiency of abnormality determination of an imaged object by an infrared image.SOLUTION: A state determination system according to an embodiment identifies a preset set area including an imaging target from an infrared image. Also, the distribution of temperature information based on the set area is obtained. Further, in the distribution, a first average value of temperature information included in a first setting range including the maximum temperature is obtained. Further, in the distribution, a second average value of temperature information included in a second set range which is lower than the first set range and does not overlap with the range is obtained. Further, a reference value which is a difference between the first average value and the second average value is obtained, a plurality of setting areas are grouped, and a determination target value based on at least one of a value indicating the variation of the reference value of each setting area and the average value for each group is obtained. In addition, an imaging target that may have a failure is specified on the basis of the determination target value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a system and a method for determining the state of an object using infrared images taken from the air. [Background technology]

[0002] Conventionally, aerial photography using aircraft (unmanned aerial vehicles (drones, etc.) has been widely practiced. Such aerial images are not only used by individuals to appreciate the scenery, but are also used for business purposes such as disaster relief, agriculture, ecological surveys, news reports, infrastructure inspections, and inspections of abnormalities.

[0003] For example, as an example of infrastructure inspection, aerial images can be used to inspect solar power generation equipment (such as solar panels) at solar power plants. Solar power generation equipment generates electricity mainly through the photovoltaic effect, so power generation is hindered by dirt, damage, etc. on the surface that receives sunlight. Aerial images make it possible to grasp the dirt, damage, etc. of the solar power generation equipment. Therefore, photographing and inspecting solar power generation equipment with aerial images is effective in improving the power generation efficiency at solar power plants.

[0004] There is also an inspection method in which a solar panel is photographed with an infrared camera, the infrared image is viewed on a video monitor, and the image is saved or printed out to identify defective areas where abnormal heat is occurring and to provide objective information and materials for repair estimates (see Patent Document 1).

[0005] However, there are photovoltaic power generation facilities that are configured by arranging photovoltaic power generation panels (for example, arranged in a matrix). When inspecting such photovoltaic power generation facilities, it may be difficult to check each panel one by one depending on the scale. However, even in such cases, a system has been proposed that acquires the temperature distribution on the front side of a group of photovoltaic power generation panels from infrared images and determines abnormalities based on the temperature information (see Non-Patent Document 1). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2015-146371 A [Non-patent literature]

[0007] [Non-Patent Document 1] Guide and Technical Information for DC Electrical Safety of Solar Power Generation (2nd Edition) National Research and Development Agency, National Institute of Advanced Industrial Science and Technology, April 2019, P.207-209 Summary of the Invention [Problem to be solved by the invention]

[0008] The following is a specific example of a method for acquiring the temperature distribution on the surface side of a group of photovoltaic panels by taking infrared images from the air and determining whether there is an abnormality based on the temperature information. First, a photovoltaic panel that includes a hotspot is identified. Then, the temperature of the hotspot of that panel is identified. Next, the average temperature around the hotspot is calculated, and the difference from the temperature of the previous hotspot is calculated. In this method, an abnormality in the panel is determined by determining whether the difference thus calculated exceeds a threshold value.

[0009] However, the difference between the average temperature around the hotspot and the temperature at the hotspot may vary greatly depending on various factors including the season, weather conditions, and the progress of the fault. For example, it is very difficult to classify the above temperature difference by threshold value alone among the three patterns of high progress of the fault in winter and cloudy weather, low progress in the same season and weather, and low progress in summer and sunny weather.

[0010] In other words, in order to extract all faulty panels using the above technology, it is desirable to include panels that are unlikely to be faulty in the judgment target and make the final judgment manually. However, this makes it difficult to reduce the burden of abnormality judgment using infrared images. In addition, there are various causes of failure of solar power generation panels, and unless the person is experienced, there is a risk of failing to judge abnormality of solar power generation panels.

[0011] The present invention has been made to solve the above-mentioned problems, and has an object to improve efficiency in determining an abnormality in an imaged object using an infrared image. [Means for solving the problem]

[0012] A status determination system according to one embodiment includes an image receiving unit that receives an infrared image from an airborne vehicle, an identification unit that identifies a pre-set set area including the image capture target from the infrared image, a temperature information acquisition unit that calculates a distribution of temperature information based on infrared information for each identified set area, a high temperature information acquisition unit that calculates a first average value of temperature information included in a first set range including the highest temperature in the distribution, a mid-temperature range acquisition unit that calculates a second average value of temperature information included in a second set range including a temperature below the first set range and not overlapping with the first set range in the distribution, a reference value acquisition unit that calculates a reference value that is the difference between the first average value and the second average value, a judgment target value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas, and calculates a judgment target value for each group based on at least one of a value indicating the variation in the reference values ​​of each set area and the average value, and a judgment unit that identifies an image capture target that may be defective based on the judgment target value. Moreover, by varying the setting range for the first average value, it may be possible to determine the type of failure. A status determination system according to another embodiment includes an image receiving unit that receives an infrared image from the flying object, an identification unit that identifies a pre-set set area including the image capture target from the infrared image, a temperature information acquisition unit that calculates a distribution of temperature information based on infrared information for each identified set area, a mid-temperature range acquisition unit that calculates a second average value of the temperature information included in a second set range that is less than the maximum temperature within the distribution, a reference value acquisition unit that calculates a reference value that is the difference between the maximum temperature and the second average value, a judgment target value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas, and calculates a judgment target value for each group based on at least one of a value indicating the variation in the reference values ​​of each set area and the average value, and a judgment unit that judges whether the image capture target is abnormal based on the judgment target value. In addition, the judgment target value acquisition unit may be configured to calculate a standard deviation of the reference value as the variation in the reference value, and further calculate a deviation value for each set area, and the judgment unit may be configured to make an abnormality judgment by comparing the deviation value with a first threshold value. In addition, when the judgment unit judges that the variation in the reference value or the average value is greater than the first threshold value, it may be configured to identify the suspected fault area causing the variation, calculate the reference value again so as not to include the suspected fault area, and identify the imaging target that may be faulty based on the reference value. In addition, the determination unit may be configured to repeat the process of recalculating the reference value and identifying the suspected failure area until the suspected failure area is no longer identified or until the area or ratio of the suspected failure area falls below a predetermined value. Another embodiment of the state determination system includes an image receiving unit that receives an infrared image from the flying object, a specifying unit that specifies a preset set area including the imaged subject from the infrared image, a temperature information acquiring unit that obtains a distribution of temperature information based on infrared information for each specified set area, a high temperature information acquiring unit that obtains a first average value of temperature information included in a first set range including the highest temperature in the distribution, a medium temperature range acquiring unit that obtains a second average value of temperature information included in a second set range that is a temperature below the first set range and does not overlap with the first set range in the distribution, and a reference value that is the difference between the first average value and the second average value. the set area and other adjacent set areas, and for each group, obtains a judgment target value based on at least one of a value indicating the variation in the reference value of each set area and an average value; a suspect target identification unit that identifies a suspect target that is an imaged object that may have a malfunction based on the judgment target value and a first threshold value; and a judgment unit that classifies the malfunction of the imaged object based on a pattern of high temperature parts in the identified suspect target, and in the case of a specific classification, re-identifies the suspect target based on the judgment target value of the suspect target and a second threshold value. The fault classification of the judgment unit may include at least two of cluster faults, cracks, stains, and cell faults. In addition, a second threshold value is used when the judgment unit judges the fault classification of the suspected object to be a cluster fault, and the second threshold value may be set so that fewer suspected objects are included in the set area compared to the first threshold value. In addition, the judgment unit may perform the judgment by inferring the classification of the suspected fault based on deep learning technology. The identification unit may also infer and identify the imaging target or a group of a predetermined number of imaging targets from the infrared image based on the characteristics of their arrangement. The state determination system may further include a setting unit for setting characteristics of the arrangement of the imaging targets, and the identification unit may identify the imaging target or a group of a predetermined number of imaging targets by utilizing the characteristics of the arrangement set by the setting unit. In addition, in the condition determination system, the identification unit may identify the imaging target or the group of imaging targets by inferring them based on deep learning technology. The condition determination system may further include a first correction unit that determines a tilt of the infrared image based on the arrangement of the image capture target or the group of image capture targets identified by the identification unit, and performs tilt correction. In addition, the status determination system may calculate the size of the identified imaging target or group of imaging targets, compare the variation in the calculated value with a determination value indicating an acceptable range of the variation, and determine an identification error if the variation is in a predetermined relationship with the determination value. In addition, in the status determination system, it may be determined whether any of the imaging targets or imaging target groups are out of alignment based on the positional relationship of the imaging targets or imaging target groups identified by the identification unit, and an imaging target or imaging target group that is out of alignment may be determined to be an identification error. Furthermore, to determine whether any of the imaging targets or groups of imaging targets are out of alignment, a variance value of the alignment position may be calculated and compared with a judgment value in the Y-axis direction in the infrared image, and if the calculated variance value has a predetermined relationship with the judgment value, it may be determined that there is a specific error. In addition, in the state determination system, the identification unit may sequentially identify the imaging objects in the arrangement direction, and determine an identification error depending on the degree of overlap between the imaging object or group of imaging objects to be determined and other identified objects. Furthermore, the identification unit may sequentially perform identification processing in the X-axis direction, and may determine, as a degree of overlap, that an identification error has occurred when a predetermined position in the X-axis direction of an identified object is between one end and the other end of the X coordinate of an adjacent object in the X-axis direction. In the condition determination system, the specified position in the X-axis direction of the imaging target that is identified may be the center position in the X-axis direction of the infrared image. In addition, in the status determination system, for each row or column of the imaging objects or imaging object groups identified by the identification unit, the imaging objects or imaging object groups belonging to that row or column may be grouped into one cluster, and if the number of imaging objects or imaging object groups belonging to each cluster is equal to or less than a specified number, the imaging objects or imaging object groups belonging to that cluster may be determined to be an identification error. In addition, in the condition determination system, the fourth determination unit may collect, as a cluster, imaging targets or groups of imaging targets having similar Y-axis positions or X-axis positions based on the Y-axis positions of the imaging targets or groups of imaging targets identified by the identification unit. In addition, in the state determination system, for each row or column of the imaging target or the imaging target group identified by the identification unit, the imaging target or the imaging target group belonging to that row or column are grouped into one cluster; The fifth judgment unit may calculate the distance between each cluster and the nearest other cluster, compare this calculated value with a judgment value, and if the calculated value has a predetermined relationship with the judgment value, judge it to be a specific error. The state determination system may further include a setting unit that allows a user to set at least one of the designated number in a cluster and the number of cluster columns. Also, The state determination system may have a sixth determination unit that, for each row or column of the imaging targets or group of imaging targets identified by the identification unit, groups the imaging targets or group of imaging targets belonging to that row or column into a single cluster so that they form a specified rectangle, and corrects the positions of the imaging targets or group of imaging targets belonging to each cluster that fall outside the rectangle so that they are included within the rectangular range. Effect of the Invention

[0013] According to the embodiment, it is possible to improve the efficiency of determining an abnormality in an image capture target using an infrared image. [Brief description of the drawings]

[0014] [Figure 1]1 is a schematic block diagram showing a state determination system according to a first embodiment; [Figure 2A] Schematic diagram of an infrared image. [Figure 2B] Schematic diagram of an infrared image. [Diagram 3] Schematic of infrared image [Figure 4] 1 is a schematic flowchart of a first embodiment. [Diagram 5] FIG. 4 is a schematic block diagram showing a state determination system according to a modified example of the first embodiment. [Figure 6] 6 is a schematic flowchart of a modified example of the first embodiment. [Figure 7] FIG. 11 is a schematic block diagram showing a state determination system according to a second embodiment. [Figure 8] 8A to 8C are schematic diagrams of infrared images illustrating the concept of processing according to the second embodiment. [Figure 9] 11 is a schematic flowchart of a second embodiment. [Figure 10] FIG. 11 is a schematic block diagram showing a state determination system according to a third embodiment. [Figure 11] 13A to 13C are schematic diagrams of infrared images illustrating the concept of processing according to a third embodiment. [Figure 12] 13A to 13C are schematic diagrams of infrared images illustrating the concept of processing according to a third embodiment. [Figure 13] 13A to 13C are schematic diagrams of infrared images illustrating the concept of processing according to a third embodiment. [Figure 14] 13 is a schematic flowchart of a third embodiment. [Figure 15] FIG. 13 is a schematic block diagram showing a determination object detection system according to a fourth embodiment. [Figure 16] 13 is a schematic flowchart of a fourth embodiment. [Figure 17] FIG. 13 is a schematic block diagram showing a state determination system according to a modified example of the fourth embodiment. [Figure 18] 13 is a schematic flowchart of a modification of the fourth embodiment. [Figure 19] FIG. 13 is a schematic block diagram showing a determination object detection system according to a fifth embodiment. [Figure 20] FIG. 13 is a conceptual diagram showing an outline of a correction process according to a fifth embodiment. [Figure 21]13 is a schematic flowchart of a fifth embodiment. [Figure 22] FIG. 13 is a schematic block diagram showing a determination object detection system according to a sixth embodiment. [Figure 23] FIG. 23 is a conceptual diagram showing an outline of a determination process according to the sixth embodiment. [Figure 24] 13 is a schematic flowchart of a sixth embodiment. [Diagram 25] FIG. 13 is a schematic block diagram showing a determination object detection system according to a seventh embodiment. [Figure 26] FIG. 23 is a conceptual diagram showing an outline of a determination process according to the seventh embodiment. [Figure 27] 13 is a schematic flowchart of a seventh embodiment. [Figure 28] FIG. 13 is a schematic block diagram showing a determination object detection system according to an eighth embodiment. [Figure 29] FIG. 23 is a conceptual diagram showing an outline of a determination process according to the eighth embodiment. [Diagram 30] 13 is a schematic flowchart of an eighth embodiment. [Diagram 31] FIG. 13 is a schematic block diagram showing a determination object detection system according to a ninth embodiment. [Diagram 32] FIG. 23 is a conceptual diagram showing an outline of a determination process according to the ninth embodiment. [Diagram 33] FIG. 23 is a conceptual diagram showing an overview of a determination process according to a modified example of the ninth embodiment. [Diagram 34] 13 is a schematic flowchart of a ninth embodiment. [Diagram 35] FIG. 23 is a schematic block diagram showing a determination object detection system according to a tenth embodiment. [Diagram 36] FIG. 23 is a conceptual diagram showing an overview of the process according to the tenth embodiment. [Figure 37] FIG. 23 is a conceptual diagram showing an overview of the process according to the tenth embodiment. [Figure 38] 23 is a schematic flowchart of a tenth embodiment. [Figure 39] FIG. 13 is a conceptual diagram showing an outline of an image indicating the location of a suspected failure point. [Diagram 40] FIG. 13 is a conceptual diagram showing an outline of an image indicating the location of a suspected failure point. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] With reference to Figs. 1 to 40, state determination systems according to first to third embodiments and determination object detection systems according to fourth to tenth embodiments will be described.

[0016] [First embodiment] The overall configuration of a condition determination system 100 according to the first embodiment will be described with reference to Figures 1 to 4. Note that the condition determination system in Figure 1 is just an example, and does not exclude the inclusion of other configurations, and can be embodied in various forms.

[0017] (Overview of Aircraft D) As shown in FIG. 1, the flying object D includes a photographing unit D1 and a position acquiring unit D2. The photographing unit D1 includes a visible light camera and an infrared camera, and can simultaneously or synchronously photograph visible light images (FIGS. 39 and 40) and infrared images (FIGS. 2A and 2B). The position receiving unit D2 receives navigation signals from positioning satellites such as the Global Navigation Satellite System (GNSS), such as GPS, GLONASS, Galileo, and Quasi-Zenith Satellite System (QZSS), measures the three-dimensional position of the flying object D, and adds the position information to the visible light image and the infrared image (for example, EXIF ​​information). The position receiving unit D2 may also be equipped with, for example, a water pressure gauge and a barometer, and convert the results into depth and altitude. As another example, an ultrasonic sensor may be equipped to measure the distance from the surface of the earth, the bottom of the sea, etc. to the flying object D. The received one-dimensional position information is added to the visible light image and the infrared image. In this way, the three-dimensional position of the visible light image and the infrared image can be identified. These are transmitted as additional information together with the visible light image and the infrared image to the condition determination system 100. Note that the imaged objects in Figures 39 and 40 and 2A and 2B are merely examples, and the objects depicted in each image may be other types of imaged objects.

[0018] (Overview of condition determination system 100) The condition determination system 100 includes a position information receiving unit 110, an image receiving unit 120, an identifying unit 130, a temperature information acquiring unit 140, a high temperature information acquiring unit 150, a medium temperature range acquiring unit 160, a reference value acquiring unit 170, a determination target value acquiring unit 180, a determining unit 190, a control unit C, and a storage unit S. The position information receiving unit 110 receives, from the flying object D, positioning information (such as a three-dimensional position) of the flying object D based on a navigation signal, and the like. For example, this may be received as information added to an image as EXIF ​​information.

[0019] The image receiving unit 120 receives an infrared image from the photographing unit D1 of the flying object D. The identifying unit 130 identifies a predetermined imaging target or a group of imaging targets from the visible light image. This is hereinafter referred to as a "set area". The imaging target is, for example, a solar power generation panel and a group of multiple solar power generation panels. Other imaging targets can include a large number of automobiles, structures of buildings (glass, etc.), various sensors arranged in large numbers, mass-produced products that are inspected, etc. In the following, a solar power generation panel will be taken as an example of the set area. In addition, for the convenience of explanation, the "solar power generation panel or a group of a predetermined number of solar power generation panels" as the "set area" will be simply referred to as a "solar power generation panel" in principle. However, in this embodiment and other embodiments and their modified examples described later, the setting area is merely an example and is not limited to a solar power generation panel.

[0020] (Specific Section 130) The identification unit 130 will be further described. The identification unit 130 identifies a set area in a visible light image received from the imaging unit D1 of the flying object D. As an example, the identification is performed based on the shape of the outer edge of the imaging target. When the imaging target is a solar power generation panel or a group of solar power generation panels, the solar power generation panel is identified based on the shape of the panel. Identifying a solar power generation panel here means, as an example, identifying the coordinates of the outer edge of a solar power generation panel or the coordinates belonging to a solar power generation panel in an infrared image. For example, the photovoltaic power generation panel group may be a group of photovoltaic power generation panels arranged in two vertical columns and two horizontal rows, or a group of photovoltaic power generation panels arranged in two vertical columns and one horizontal row. This detection unit may be determined for each state determination system and not be changed, or may be set by a user's operation via an input unit (not shown). Below, a configuration for identifying "each photovoltaic power generation panel" will be described, but this embodiment includes a modified example in which the identifying unit 130 detects "two or more photovoltaic power generation panels as one unit," so the processing of the temperature information acquisition unit 140 described below and each processing in the second embodiment and thereafter (including modified examples) also take into consideration the modified example in which such a photovoltaic power generation panel group is detected.

[0021] Next, the coordinates of the identified region of the imaging target (set region) in the visible light image are used directly in the coordinates of the infrared image by the identification unit 130. This is based on the premise that the angle of view, angle, scale, etc. of the visible light image and the infrared image are the same or substantially the same (within the allowable range of the difference in panel positions between the two images).

[0022] Alternatively, when there is a difference in the angle of view, angle, scale, etc. between the visible light image and the infrared image, and it is not acceptable to simply replace the coordinates of one image with the coordinates of the other (the difference is large), the identification unit 130 may be configured to align the scale and position by using some landmark, marker, or morphological characteristics and positional characteristics of the subject to be imaged.

[0023] The above-described configuration in which the detection of the imaging target is performed in a visible light image and the coordinates of the identified area are applied to an infrared image can also be applied to the following modified examples and other embodiments.

[0024] (Temperature information acquisition unit 140) The temperature information acquisition unit 140 acquires temperature information within a set area (e.g., each panel, a group of panels, etc.) by any known technique for each solar power generation panel in the infrared image identified by the identification unit 130. For example, the temperature information acquisition unit 140 obtains temperature information from the color temperature of each smallest unit constituting the image in the set area of ​​the infrared image. The smallest unit may be a pixel or a block of predefined pixel units. Coordinate information in the image may be associated with each piece of temperature information for the smallest unit. The temperature information acquisition unit 140 also obtains the temperature distribution for each set area from the temperature information.

[0025] (High temperature information acquisition unit 150) The high temperature information acquisition unit 150 identifies temperature information included in a first set range based on temperature information for each minimum unit of the set area, for example, a temperature distribution. The first set range is a set range below the maximum temperature in the temperature distribution or a set range based on the maximum temperature. The high temperature information acquisition unit 150 also obtains an average value of each temperature based on the temperature information included in the first set range. This average value is set as a first average value. As a specific example, the high temperature acquisition unit 150 obtains temperature information of each pixel in the set area obtained by the temperature information acquisition unit 140. In this example, the high temperature acquisition unit 150 obtains an average value of each temperature information of the top 20% including the maximum temperature in the set area. This average value does not have to be the average value of all temperatures in the set area. For example, it may be a sample average value such as an average of the maximum and minimum temperatures of the top 20% or a median. In the example of a photovoltaic panel being imaged, if there is a cluster fault, high temperature areas will appear in clusters that span multiple cells of the photovoltaic panel in rows or columns. Therefore, in order to respond to cluster faults in each photovoltaic panel, high temperature information in the top 20% of the temperature distribution is used as the high temperature area when determining suspected objects. This high temperature information is used in comparison with the medium temperature area described below. In contrast, if the fault is one in which the area of ​​the high temperature part tends to be small, such as dirt or cell faults, the top 3% of the temperature distribution is used as the high temperature area to determine the suspected fault. This high temperature information is used in comparison with the medium temperature area described below.

[0026] (Medium temperature range acquisition part 160) The mid-temperature range acquisition unit 160 identifies temperature information of a second set range that is less than the first set range based on temperature information for each minimum unit of the set range, for example, temperature distribution. The lower limit of the second set range is a temperature that does not reach the minimum temperature. In the case of an imaging target where errors or defects appear in the high temperature parts of an infrared image, such as a solar power generation panel, excluding low temperature parts that are likely to become noise helps prevent erroneous detection and improve the efficiency of abnormality determination of the imaging target. In other words, by setting a temperature higher than the minimum temperature and lower than the mid-temperature as the lower limit value of the second set range, a predetermined range that becomes a low temperature part is excluded. In other words, a temperature zone between high and low temperatures is identified. Furthermore, the mid-temperature range acquisition unit 160 obtains the average value of each temperature based on the temperature information included in the second set range. This average value is set as the second average value. As a specific example, the mid-temperature range acquisition unit 160 obtains the temperature information of each pixel in the set range obtained by the temperature information acquisition unit 140. In this example, the mid-temperature range acquisition unit 160 obtains the average value of each temperature information in the mid-temperature range, with any temperature exceeding the bottom 20% as the lower limit and any temperature below the top 20% as the upper limit. Alternatively, the mid-temperature range acquisition unit 160 obtains the average value of each temperature information of the top 40 to 80%. This average value does not have to be the average value of all temperatures in the set range. For example, it may be a sample average value such as the average of the maximum and minimum temperatures in the second set range, or a median. Also, it may be a predetermined temperature (for example, a median) instead of a range such as the second set range.

[0027] (Reference value acquisition unit 170) The reference value acquisition unit 170 acquires a reference value X , which is the difference between the first average value acquired by the high temperature acquisition unit 150 and the second average value acquired by the medium temperature range acquisition unit 160. i Request.

[0028] (Determination target value acquisition unit 180) The determination target value acquisition unit 180 groups a plurality of set regions, including the set region and other adjacent set regions, into one group. This grouping is performed in a preset arrangement. When the imaging targets are arranged in a matrix, grouping is performed by the number of rows and columns.

[0029] The judgment target value acquisition unit 180 obtains a judgment target value based on at least one of a value indicating the variation of the reference value of each set region and an average value for each group. For example, the judgment target value acquisition unit 180 obtains a judgment target value based on the reference value obtained by the reference value acquisition unit 170 (e.g., reference value = first average value - second average value) X i And, X i The average value μ x The judgment target value acquisition unit 180 calculates the difference "X i -μ x " with standard deviation σ x The value divided by is the deviation value SS, which indicates the deviation amount of the temperature change difference in the set area. i Obtain as. Reference temperature (reference value) of each panel X i Standard deviation SS i is defined by the following equation 1.

number

[0030] When the total number of panels is N, the reference temperature is X i The average value μ x is defined by the following equation 2.

number

[0031] When the total number of panels is N, the reference temperature is X i Standard deviation of σ x is defined by the following equation 3.

number

[0032] In addition, the reference temperature for each setting area is X iIf are all equal, Standard deviation σ x =0 and SS i Because it is not possible to obtain SS i =1.

[0033] (Judgment section 190) The judgment unit 190 judges the deviation value SS i The target value for judgment, such as temperature, is compared with the threshold value to identify the target image that may be faulty. Specifically, the target value for judgment is compared with the threshold value or the range of the appropriate range to determine whether the target image contains an abnormality. For example, in each solar panel, the reference temperature X i Standard deviation SS i If the temperature exceeds an allowable range or value (for example, if it exceeds a preset threshold value), it is identified as a suspected fault location. In addition, the determination unit 190 may store in advance the shape of a group (pattern) of pixels exhibiting an abnormal temperature, and identify a specific abnormality by comparing with the pattern.

[0034] Furthermore, if an abnormality is found in each photovoltaic power generation panel as post-processing of the determination process, the determination unit 190 performs processing to at least identify the position of the panel and present it to the user in a manner of their choice (see the "DIFF" panel in the fourth row and sixth column of the upper block in Fig. 3, or Figs. 39 and 40, etc.). In addition to the position of the imaged subject, processing may be performed to create and display the maximum temperature, average temperature, or number of abnormal pixels for each imaged subject as image data.

[0035] (Control section C configuration) For convenience of explanation, the processing of each unit in the above description has been described as being executed under the control of the control unit C shown in FIG. 1. The control unit C may be configured to include a single or multiple circuits, such as a central processing unit (CPU), a graphics processing unit (GPU), or an application specific integrated circuit (ASIC). The control unit C realizes its function by reading and executing, for example, an image position identification program stored in a memory. Note that instead of storing a program in a memory, the program may be directly built into the circuit. In this case, the function is realized by reading and executing the program built into the circuit as the control unit C. Note that the control unit C is not limited to being configured as a single circuit, and may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, the functions of the embodiment may be realized by integrating multiple components (e.g., at least two of the location information receiving unit 110, image receiving unit 120, identification unit 130, temperature information acquiring unit 140, high temperature information acquiring unit 150, medium temperature range acquiring unit 160, reference value acquiring unit 170, judgment target value acquiring unit 180, and judgment unit 190) into a single processor.

[0036] [Operation] Fig. 4 is a diagram showing the flow of processing in this embodiment. Hereinafter, with reference to Fig. 4, the flow of processing in the state determination system 100 will be described along with step numbers (S001 to S009).

[0037] (S001) The position information receiving unit 110 in the status determination system 100 receives position information from the flying object D. Furthermore, the image receiving unit 120 receives visible light images and infrared images.

[0038] (S002) The identification unit 130 identifies a set area (such as a solar power generation panel) in the visible light image. Next, the identification unit 130 uses the coordinates of the identified area of ​​the imaging target (set area) in the visible light image as they are in the coordinates of the infrared image. Note that the identification unit 130 may identify the set area from the infrared image (see the fourth embodiment, etc.). In this case, acquisition of the visible light image and related processing may be omitted.

[0039] (S003) The temperature information acquisition unit 140 acquires temperature information within each solar panel in the infrared image identified by the identification unit 130. This may be performed for each pixel, or a plurality of pixels may be treated as one unit and an average value for each pixel group may be used as temperature information. The temperature information acquisition unit 140 also obtains a temperature distribution for each set region from the temperature information.

[0040] (S004) The high temperature information acquiring unit 150 identifies temperature information included in a first set range. The first set range is a set range below the maximum temperature in the temperature distribution or a set range based on the maximum temperature, for example, the top 20% including the maximum temperature in the set area.

[0041] (S005) Moreover, the high temperature information acquiring section 150 obtains a first average value which is an average value of each temperature based on the temperature information included in the first set range.

[0042] (S006) The intermediate temperature range acquisition unit 160 identifies temperature information in the second set range. The second set range is, so to speak, a temperature zone between high and low temperatures.

[0043] (S007) Furthermore, the mid-temperature range acquisition section 160 obtains a second average value, which is the average value of each temperature based on the temperature information included in the second set range.

[0044] (S008) The reference value acquiring section 170 acquires a reference value which is the difference between the first average value acquired by the high temperature acquiring section 150 and the second average value acquired by the medium temperature range acquiring section 160.

[0045] (S009) The judgment target value acquisition unit 180 groups a plurality of set regions into one group. The judgment target value acquisition unit 180 also obtains a judgment target value for each group based on at least one of a value indicating the variation in the reference values ​​of the set regions and an average value. The judgment target value is, for example, a standard deviation value SS i It is.

[0046] (S010) The determination unit 190 compares the determination target value with the first threshold value to identify an imaging target that may be at risk of failure. Furthermore, if an abnormality is found in any photovoltaic power generation panel as a post-processing of the determination process, the determination unit 190 performs a process of at least identifying the position of that panel and notifying the user by any method (see FIGS. 39 and 40).

[0047] [effect] According to this embodiment, the difference between the average value of the high temperature range and the average value of the medium temperature range is obtained from the temperature distribution in the set area of ​​the infrared image. In addition, a judgment target value is obtained for each group of multiple set areas based on at least one of the value indicating the variation in the reference value of each set area and the average value. Furthermore, a judgment is made about an abnormality of the imaged object based on the judgment target value. With this configuration, mistakes in the abnormality judgment of the imaged object in each set area are reduced. In other words, since the variation in the difference between the average value of the high temperature range and the average value of the medium temperature range is obtained within a group of photovoltaic power generation panels exposed to similar conditions (weather conditions, etc.), it is possible to distinguish between the influence of the season and weather and the influence of the progress of the disorder of the imaged object.

[0048] Another reason why errors in abnormality determination can be reduced is that the imaging target in the infrared image is identified from other objects, and temperature information of other objects is excluded from the information used for abnormality determination. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controllers and cables), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panels. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder abnormality determination. According to this embodiment, the solar power generation panel is first identified, and abnormality determination is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to shadows, and in this respect, it is also possible to reduce errors in abnormality determination.

[0049] According to the above embodiment, since the occurrence of misjudgment is reduced, panels that are unlikely to be faulty are more likely to be excluded from suspected faulty targets, which makes it possible to improve the efficiency of abnormality judgment of the imaging target using the infrared image.

[0050] [Variation A] A state determination system 100 according to a modified example of the first embodiment will be described with reference to Fig. 5 and Fig. 6. In the following description, descriptions that overlap with the first embodiment will be omitted.

[0051] (overview) As in the first embodiment, the present modification also sets the imaging target in the infrared image as the set region. In other words, it is configured to exclude anything other than the imaging target from the judgment of suspected faults. That is, the photovoltaic power generation panels are first identified, and the abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, the abnormality judgment error due to the arrangement of the photovoltaic power generation panels is also reduced.

[0052] In the state determination system 100 of this modification, the high temperature information acquisition unit 150A identifies the maximum temperature based on temperature information for each minimum unit of the set region, for example, a temperature distribution. The mid-temperature range acquisition unit 160 identifies temperature information for a second set range that is less than the maximum temperature and includes the mid-temperature, based on temperature information for each minimum unit of the set region, for example, a temperature distribution. Furthermore, the mid-temperature range acquisition section 160 obtains an average value of each temperature based on the temperature information included in the second set range, and sets this average value as the second average value.

[0053] The reference value acquiring section 170 acquires a reference value which is the difference between the maximum temperature (high temperature information) identified by the high temperature acquiring section 150 and the second average value acquired by the mid temperature range acquiring section 160.

[0054] The judgment target value acquisition unit 180 obtains a judgment target value for each group based on at least one of a value indicating the variation in the reference values ​​of the set regions and an average value. i By comparing the judgment target values ​​such as those mentioned above with a threshold value, imaging targets that may be at risk of being damaged are identified.

[0055] [Operation] Fig. 6 is a diagram showing the flow of processing in this modified example. Hereinafter, with reference to Fig. 6, the flow of processing in the state determination system 100 will be described along with step numbers (S011 to S019).

[0056] (S011~S013) These are similar to S001 to S003 in the state determination system 100 of the first embodiment.

[0057] (S014) The high-temperature information acquiring unit 150A identifies the maximum temperature based on, for example, the temperature distribution.

[0058] (S015) The mid-temperature range acquisition section 160 identifies temperature information in the second set range.

[0059] (S016) Furthermore, the mid-temperature range acquisition section 160 obtains a second average value, which is the average value of each temperature based on the temperature information included in the second set range.

[0060] (S017) The reference value acquiring section 170 obtains a reference value which is the difference between the maximum temperature and the second average value.

[0061] (S018) The judgment target value acquisition unit 180 obtains a judgment target value for each group based on at least one of a value indicating the variation of the reference values ​​of the set regions and an average value. The judgment target value is, for example, a standard deviation value SS i It is.

[0062] (S019) The determination unit 190 identifies an imaging target that may be a failure by comparing the determination target value with the first threshold value. Furthermore, if an abnormality is found for each photovoltaic power generation panel as a post-processing of the determination process, the determination unit 190 performs a process of at least identifying the position of the panel and presenting it to the user in a method of their choice (see FIGS. 39 and 40).

[0063] [effect] According to this modification, the difference between the maximum temperature and the average value of the medium temperature range is obtained from the temperature distribution in the set area of ​​the infrared image. In addition, a judgment target value is obtained for each group of multiple set areas based on at least one of a value indicating the variation in the reference value of each set area and the average value. Furthermore, a judgment is made regarding an abnormality of the imaged object based on the judgment target value. With this configuration, mistakes in judging an abnormality of the imaged object in each set area are reduced. In other words, since the variation in the difference between the maximum temperature and the average value of the medium temperature range is obtained within a group of photovoltaic power generation panels exposed to similar conditions (weather conditions, etc.), it is possible to distinguish between the influence of the season and weather and the influence of the progress of the disorder of the imaged object.

[0064] Another reason why errors in abnormality determination can be reduced is that the imaging target in the infrared image is identified from other objects, and temperature information of other objects is excluded from the information used for abnormality determination. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controllers and cables), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panels. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder abnormality determination. According to this embodiment, the solar power generation panel is first identified, and abnormality determination is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to shadows, and in this respect, it is also possible to reduce errors in abnormality determination.

[0065] According to the above embodiment, since the occurrence of misjudgment is reduced, panels that are unlikely to be faulty are more likely to be excluded from suspected faulty targets, which makes it possible to improve the efficiency of abnormality judgment of the imaging target using the infrared image.

[0066] [Variation B] A description will be given of a state determination system 100 according to the first embodiment and Modification B of Modification A thereof. In the following description, descriptions that overlap with the first embodiment and the like will be omitted.

[0067] (overview) As in the first embodiment, the present modification also sets the imaging target in the infrared image as the set region. In other words, it is configured to exclude anything other than the imaging target from the judgment of suspected faults. That is, the photovoltaic power generation panels are first identified, and the abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, the abnormality judgment error due to the arrangement of the photovoltaic power generation panels is also reduced.

[0068] In addition, in the state determination system 100 in this modification, the high temperature information acquisition unit 150A identifies temperature information or the maximum temperature included in the first set range based on temperature information for each minimum unit of the set area, for example, temperature distribution. In this modification B, the high temperature information acquisition unit A acquires multiple types of high temperature information. An example of the multiple types is multiple ranges in which at least one of the upper limit value and the lower limit value of the first set range is different. Also, it may be a predetermined temperature such as the maximum temperature without being limited to a range. For example, the high temperature information acquisition unit 150A identifies three types: the top 20%, the top 3%, and the maximum temperature.

[0069] The mid-temperature range acquisition unit 160 identifies temperature information of a second set range that is less than the maximum temperature and includes the intermediate temperature, based on temperature information for each minimum unit of the set range, for example, a temperature distribution. For this mid-temperature range, the unit may be configured to determine a plurality of set ranges in accordance with the high temperature range. Furthermore, the mid-temperature range acquisition unit 160 determines the average value of each temperature based on the temperature information included in the second set range. This average value is set as the second average value. This average value does not have to be the average value of all temperatures within the set range. For example, it may be a sample average value such as the average of the maximum and minimum temperatures in the second set range, or a median value. Furthermore, it may be a predetermined temperature (for example, a median value) instead of a range such as the second set range.

[0070] The reference value acquiring section 170 acquires a reference value which is the difference between the multiple types of high temperature information (maximum temperature, first average value, etc.) specified by the high temperature acquiring section 150 and the second average value acquired by the mid temperature range acquiring section 160.

[0071] The judgment target value acquisition unit 180 obtains a judgment target value for each group based on at least one of a value indicating the variation in the reference values ​​of the set regions and an average value. i The image capturing target that may be damaged is identified by comparing the judgment target value such as the above with the threshold value. The threshold value may be set to a unique threshold value for at least one type of high temperature information according to the type of high temperature information.

[0072] The determination unit 190 identifies an imaging target that may be impaired by comparing the determination target value with a threshold value. Specifically, the determination unit 190 identifies the presence or absence of an impairment and the type of impairment based on the result of the comparison. For example, if the determination target value for any of the above types of high temperature information is equal to or less than the threshold value, the determination unit 190 determines that there is no impairment. On the other hand, if the threshold value is exceeded for any of the above types of high temperature information, the determination unit 190 identifies the type of impairment corresponding to the type of high temperature information on which the exceeded determination target value was calculated.

[0073] It is also possible to handle cases where two or more types of faults can be identified by the determination unit 190. As an example, it is assumed that a prediction pattern of the comparison result between the determination target value and the threshold value is stored in advance. In this case, the determination unit 190 determines which of the prescribed prediction patterns the comparison result with the threshold value is closest to, and identifies the presence or absence of a fault and the type of the fault.

[0074] According to the above embodiment, since the occurrence of misjudgment is reduced, panels that are unlikely to be faulty are more likely to be excluded from suspected faulty targets, which makes it possible to improve the efficiency of abnormality judgment of the imaging target using the infrared image.

[0075] [Second embodiment] A state determination system 200 according to the second embodiment will be described with reference to Figures 7 to 9. In the following description, descriptions that overlap with those of the first embodiment will be omitted.

[0076] As in the first embodiment, in the second embodiment, the imaging target in the infrared image is identified, and the reference value and the judgment target value are obtained. Also, as in the first embodiment and its modified example, the point that the object suspected of failure (panel, etc.) is identified is also the same.

[0077] Furthermore, in the second embodiment, when a location that may be a fault (hereinafter, in principle, referred to as a "suspected fault location") is detected, the status determination for that group is performed again without including that location. The more likely that a number of likely fault locations are detected within the same group during the determination, the more likely the calculated deviation amount will be slowed down. Therefore, in the second embodiment, the suspected fault locations are identified in detail by excluding the likely fault locations and performing a status determination again. Furthermore, in the second embodiment, the process is repeated until no suspected fault locations are identified for each group, or until the area or ratio of the suspected fault locations falls below a set value.

[0078] (Condition Determination System 200) As shown in Figure 7, the condition determination system 200 of the second embodiment is configured to include a location information receiving unit 210, an image receiving unit 220, an identification unit 230, a temperature information acquiring unit 240, a high temperature information acquiring unit 250, a medium temperature range acquiring unit 260, a reference value acquiring unit 270, a determination target value acquiring unit 280, a determination unit 290, as well as a control unit C and a memory unit S.

[0079] Detection of solar power generation panels by the identification unit 230, acquisition of high temperature information by the high temperature information acquisition unit 250, calculation of the second average value by the medium temperature range acquisition unit 260, calculation of the reference value by the reference value acquisition unit 270, acquisition of the judgment target value by the judgment target value acquisition unit 280, and identification of an imaging target that may be at risk of failure by the judgment unit 290 are the same as in the first embodiment and its modified examples.

[0080] In the second embodiment, after the determination by the determination unit 290, if there is a suspected fault location, the suspected fault location is excluded. For example, the determination unit 290 identifies the coordinates of the suspected fault location in the image and excludes the coordinates. While keeping the initial grouping of set areas, the temperature information acquisition unit 240 obtains the temperature distribution for each set area without including the temperature information at the excluded coordinates. The high temperature information acquisition unit 250 obtains high temperature information based on the temperature distribution after this exclusion.

[0081] The mid-temperature range acquisition unit 260 identifies temperature information of the second set range based on the temperature distribution after this exclusion. Alternatively, in the second embodiment, it is also possible to use temperature information of the second set range in the first determination. In this case, the process of identifying the second set range from the second time onwards may be omitted. Even in this case where the process of identifying is omitted, the process of identifying the second set range may be configured to be performed again when the third determination process is performed after the second determination process.

[0082] The calculation of the reference value and the judgment target value thereafter is the same as in the first embodiment and its modified examples. As in the first embodiment, the judgment unit 290 calculates, for example, the reference temperature X i Standard deviation SS i exceeds an allowable range or value (for example, exceeds a preset threshold), the suspected fault location is identified (see FIG. 8). If a suspected location still exists after the identification process, the determination unit 290 repeats the above process. However, the second embodiment is not limited to this. For example, the determination unit 290 may be configured to repeat the determination process from excluding the suspected location until the area or ratio of the suspected fault area falls below a preset value. Alternatively, instead of this configuration, the determination unit 290 may be configured to repeat the determination process from excluding the suspected location while updating the threshold.

[0083] [Operation] Fig. 9 is a diagram showing a process flow in the second embodiment. Hereinafter, with reference to Fig. 9, the process flow of the state determination system 200 will be described along with step numbers (S021 to S030-1).

[0084] (S021~S028) These are similar to S001 to S008 or S011 to S019 in the state determination system 100 of the first embodiment.

[0085] (S029) The determination section 290 compares the determination target value with the first threshold value to identify an imaging target that may be at risk of failure.

[0086] (S030) The determining unit 290 determines whether or not there is a suspected fault location for each set area by comparing the determination target value with the first threshold value.

[0087] (S030-1) As a result of the determination in S030, in the set area where it is determined that there is a suspected fault location (S030; Yes), the coordinates in the image of the suspected fault location are identified, and the coordinates are excluded from the set area.

[0088] After S030-1, the process returns to S023 and the temperature information acquisition unit 240 obtains the temperature distribution for each set area without including the temperature information at the excluded coordinates. These processes are repeated until it is no longer determined that there is a suspected fault location as a result of the determination in S030 (S030; No). Furthermore, if there is an abnormality in each photovoltaic power generation panel as post-processing of the determination process, the determination unit 290 performs a process of at least identifying the position of that panel and presenting it to the user by any method (see Figures 39 and 40).

[0089] [effect] According to the second embodiment, the difference between the maximum temperature or the average value of the high temperature range and the average value of the medium temperature range is obtained from the temperature distribution in the set area of ​​the infrared image. In addition, a judgment target value is obtained for each group of multiple set areas based on at least one of the value indicating the variation in the reference value of each set area and the average value. Furthermore, a judgment is made about an abnormality of the imaged object based on the judgment target value. With this configuration, mistakes in the abnormality judgment of the imaged object in each set area are reduced. In other words, since the variation in the difference between the average value of the high temperature range and the average value of the medium temperature range is obtained within a group of photovoltaic power generation panels exposed to similar conditions (weather conditions, etc.), it is possible to distinguish between the influence of the season and weather and the influence of the progress of the disorder of the imaged object.

[0090] Another reason why errors in abnormality determination can be reduced is that the imaging target in the infrared image is identified from other objects, and temperature information of other objects is excluded from the information used for abnormality determination. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controllers and cables), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panels. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder abnormality determination. According to this embodiment, the solar power generation panel is first identified, and abnormality determination is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to shadows, and in this respect, it is also possible to reduce errors in abnormality determination.

[0091] Furthermore, in the second embodiment, the suspected fault location is excluded and the state judgment is performed again to identify the suspected fault location in detail. This makes it possible to reduce the omission of identification of the suspected fault location even when many locations that may be faulty are detected within the same group in the judgment.

[0092] According to the above embodiment, since the occurrence of misjudgment is reduced, panels that are unlikely to be faulty are more likely to be excluded from suspected faulty targets, which makes it possible to improve the efficiency of abnormality judgment of the imaging target using the infrared image.

[0093] The determination object detection system understood from the above-described second embodiment can have the following features. [Feature A] an image receiving unit for receiving an infrared image from the flying object; An identification unit that identifies a preset region including an imaging target from an infrared image; Identifying a suspected fault area for each of the identified setting areas, A determination unit that performs identification of another suspected fault area again so as not to include the suspected fault area. Status determination system. The second embodiment can be configured independently instead of being combined with the first embodiment.

[0094] [Third embodiment] A state determination system 300 according to the third embodiment will be described with reference to Figures 10 to 14. In the following description, descriptions that overlap with the first and second embodiments will be omitted.

[0095] In the third embodiment, the imaging target in the infrared image is identified and the reference value and the judgment target value are calculated, as in the first embodiment. Also, the suspected fault target (panel, etc.) is identified, as in the first embodiment.

[0096] In the third embodiment, the type of failure is further classified for the set area including the suspected failure area. Depending on the classification of the failure, there may be cases where the amount of heat generated is small due to the influence of the external environment such as weather conditions or the progress of the failure. That is, it is expected that the deviation amount will be small when identifying the suspected failure area by the first embodiment or the like. Therefore, in order to deal with such a situation, the third embodiment classifies the type of failure of the photographed object such as a solar power generation panel, and is configured to identify more suspected failure areas in a specific classification than in other classifications, thereby reducing the omission of identification of the suspected failure area in a specific pattern such as a small amount of heat generated, and reducing excessive identification of the suspected failure area.

[0097] (Condition Determination System 300) As shown in FIG. 10, the condition determination system 300 of the third embodiment is configured to include a location information receiving unit 310, an image receiving unit 320, an identification unit 330, a temperature information acquiring unit 340, a high temperature information acquiring unit 350, a medium temperature range acquiring unit 360, a reference value acquiring unit 370, a determination target value acquiring unit 380, a determination unit 390, a suspected fault identifying unit 391, as well as a control unit C and a memory unit S.

[0098] Detection of the solar power generation panel by the identifying unit 330, acquisition of high temperature information by the high temperature information acquiring unit 350, calculation of the second average value by the medium temperature range acquiring unit 360, calculation of the reference value by the reference value acquiring unit 370, and acquisition of the judgment target value by the judgment target value acquiring unit 380 are the same as those in the first embodiment etc. In the third embodiment, a suspect target identifying unit 391 identifies an imaging target that may have a problem. The identification method is the same as that in the first embodiment etc.

[0099] (Judgment unit 390) In the third embodiment, after the suspected object identifying unit 391 identifies a suspected fault location, if there is a suspected fault location, the determining unit 390 classifies the type of fault in the set area. As an example, the determining unit 390 classifies the type of fault in the set area in the image by deep learning.

[0100] This uses an inference model created by deep learning using so-called artificial intelligence (AI). The inference model contains a group of teacher data consisting of models of suspected areas that appear for each type of failure of the object being imaged in the infrared image (e.g., a solar power generation panel). The models of suspected areas include information such as shape, size, and temperature. In other words, the shape can be said to be a feature quantity of the failure pattern. In other words, the size can be said to be the input size. The shapes that appear in an infrared image (for example, a shape formed by a collection of relatively high temperature points (Figure 8: "above threshold") are based on temperature information. Therefore, the temperature information can be used as a model of the suspected area to supplement the above shapes.

[0101] Deep learning is performed by artificial intelligence using the teacher data group, and an inference model is generated. In the determination unit 390, inference is performed using the inference model for a set area including a suspected fault location, and the type of fault is classified. Note that in this embodiment, the classification method is not limited to deep learning. Other methods such as pattern matching can also be used.

[0102] Classification examples of the determination unit 390 will be described with reference to Figs. 11 to 13. Figs. 11 to 13 show infrared images in which the imaging target is a solar power generation panel. Fig. 11 is an example of a cluster failure. Fig. 12 is an example of a crack. Fig. 13 is an example of a stain or cell failure.

[0103] <Classification of disabilities> In a solar panel, for example, if clusters each consisting of two rows of cells are arranged vertically in three rows, a cluster failure will occur in a horizontally elongated shape that roughly follows the shape of the cell cluster. In an infrared image, as shown in Fig. 11, an abnormal heat generation area can be seen along the horizontally elongated cluster. Therefore, the determination unit 390 classifies the area as a cluster failure based on the shape of the heat generation area on the infrared image that is similar to the cluster shape.

[0104] The determination unit 390 may classify a heat generating spot as a "failure other than a cluster failure" by determining that the shape of the heat generating spot on the infrared image is not similar to the cluster shape. Alternatively, when a heat generating spot is observed over a relatively wide area like a crack but has a different form from the cluster model (see FIG. 12), the determination unit 390 may be configured to classify the heat generating spot as a "crack" failure.

[0105] The determination unit 390 may be configured to classify the failure as a type of dirt or cell failure as another classification. In the case of dirt or cell failure, the heat generating portion is seen in a relatively narrow area compared to clusters or cracks. Therefore, when the type is different from the cluster or crack model (see FIG. 13), the determination unit 390 may be configured to classify the heat generating portion as a failure of "dirt or cell failure."

[0106] <Re-identification process overview> There are several classifications of faults, and the above is just one example, but by classifying the specific fault type by the determination unit 390 as described above, it becomes possible to execute the following re-identification process of suspected targets. As will be described later, this re-identification process can further reduce the failure to identify suspected targets. As a result, it becomes possible to more reliably determine the presence or absence of a fault in the set area according to the fault type.

[0107] In the above three-tiered vertical cluster arrangement, if a cluster fault occurs in one tier, abnormal heat generation tends to be observed throughout the cluster arrangement, as shown in Figure 11. The judgment target value used to identify suspected faults is a value that indicates the variation of the reference value of each set area calculated for each group, so if a relatively widespread fault occurs, such as a cluster fault, the judgment target value may not show a significant value or the variation may be small, which may cause the suspected fault location to be missed. In addition, in cluster faults, it is expected that the deviation in heat generation may be small depending on the progression of the fault (reduced power generation, poor current, etc.). Weather conditions such as season, weather, and geographical conditions may also have a significant effect on the deviation in heat generation. That is, in order to identify a cluster fault as a suspected fault location, it is necessary to identify it as a fault even if the temperature deviation is relatively smaller than other faults. When a suspected fault is identified due to a cluster fault, it is possible to prevent or reduce the oversight of identifying the suspected fault location by making a more lenient fault determination, so to speak, compared to the case of cracks as shown in FIG. 12, dirt as shown in FIG. 13, and cell faults. However, since there is a tendency for the deviation to be large in the case of cracks, dirt, and cell faults, there is a risk that the criteria for identifying a cluster fault will excessively identify (detect) non-faulty areas. Therefore, in order not to reduce the efficiency of anomaly determination, excessive identification of suspected infringement locations is reduced. Therefore, in this embodiment, the faults are classified as described above, and the suspected target identification unit 391 described below performs fault determination corresponding to the aspect (contour shape, etc.) of the suspected fault location for each fault classification.

[0108] <Re-identification process_suspect target identification unit 391> In this embodiment, a re-identification process for a specific suspect target is performed in accordance with the classification of the failure mode. The re-identification process will be described using a cluster failure as an example. Among the set areas, set areas other than the set areas classified by the determination unit 390 as including a cluster failure (for example, a photovoltaic power generation panel or a group thereof) (hereinafter referred to as "set areas of other classifications") are the targets of the re-identification process.

[0109] The suspected target identification unit 391 re-identifies suspected failure locations in other classification setting areas using a second threshold value different from the threshold value (first threshold value) for the setting area suspected of cluster failure or the setting area without suspected failure locations. The second threshold value is a value that suppresses excessive identification of suspected failure locations in this setting area. For example, the standard deviation value SS in the above embodiment i If the suspected disability is identified by the standard deviation SS i By increasing the threshold, the number of locations identified as suspected fault locations is further reduced.

[0110] In the case of a cluster fault where the amount of heat deviation may be small due to weather conditions or the progression of the fault as described above, excessive identification of suspected locations in other classification setting areas is reduced by re-identification processing using the second threshold value.

[0111] This embodiment is not limited to the above. For example, a threshold value corresponding to cracks may be set by distinguishing them from other classifications. Also, a threshold value corresponding to dirt or cell failure may be set by distinguishing them from other classifications. Also, the classification of failures may be divided into at least two or more classifications.

[0112] In the third embodiment, the exclusion of suspected fault locations and the identification process after the exclusion in the second embodiment may be combined.

[0113] [Operation] Fig. 14 is a diagram showing the flow of processing in this embodiment. Hereinafter, with reference to Fig. 14, the flow of processing of the state determination system 300 will be described along with step numbers (S031 to S03). Note that the following processing is executed after the processing of identifying a suspected fault location in the first embodiment and the like. Also, the flow of an example of a form in which a re-identification process is performed after classifying cluster fault areas will be described.

[0114] (S031) After the suspected object identifying unit 391 identifies a suspected fault location, if there is a suspected fault location, the determining unit 390 classifies the type of fault in the set area. As an example, the determining unit 390 classifies the type of fault in the set area in the image by deep learning.

[0115] (S032) As a result of the classification, the determining unit 390 determines whether the set region includes a cluster failure.

[0116] (S033) If the set area is classified as not including a cluster failure according to the classification result of the judgment unit 390 (S032; No), the judgment unit 390 judges whether it includes a suspected area classified as an other failure. If the set area does not include a failure location (S033; No), there is no cluster failure and no failure of another classification, so the area is judged to be normal.

[0117] (S034) If it is determined that the classification result by the determination unit 390 includes a suspected area classified as a different type of failure, the suspected object identification unit 391 performs re-identification processing.

[0118] (S035) If it is determined that the set area includes a cluster failure (S033; Yes), or after the re-identification process (S034), a process is performed to at least identify the position of the set area in its entirety and to present this to the user in a method of their choice.

[0119] [effect] According to the third embodiment, the difference between the maximum temperature or the average value of the high temperature range and the average value of the medium temperature range is obtained from the temperature distribution in the set area of ​​the infrared image. In addition, a judgment target value is obtained based on at least one of the value indicating the variation in the reference value of each set area and the average value for each set area for each group of multiple set areas. Furthermore, an abnormality of the imaged object is judged based on the judgment target value. With this configuration, mistakes in the abnormality judgment of the imaged object in each set area are reduced. In other words, since the variation in the difference between the average value of the high temperature range and the average value of the medium temperature range is obtained within a group of photovoltaic power generation panels exposed to similar conditions (weather conditions, etc.), it is possible to distinguish between the influence of the season and weather and the influence of the progress of the disorder of the imaged object.

[0120] Another reason why errors in abnormality determination can be reduced is that the imaging target in the infrared image is identified from other objects, and temperature information of other objects is excluded from the information used for abnormality determination. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controllers and cables), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panels. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder abnormality determination. According to this embodiment, the solar power generation panel is first identified, and abnormality determination is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to shadows, and in this respect, it is also possible to reduce errors in abnormality determination.

[0121] Furthermore, in the third embodiment, in the case of a specific failure classification, by performing a process of re-identifying a suspected failure location, it is possible to reduce excessive identification of a suspected failure location.

[0122] According to the above embodiment, since the occurrence of misjudgment is reduced, panels that are unlikely to be faulty are more likely to be excluded from suspected faulty targets, which makes it possible to improve the efficiency of abnormality judgment of the imaging target using the infrared image.

[0123] The determination object detection system understood from the above-described second embodiment can have the following features. [Feature B] an image receiving unit for receiving an infrared image from the flying object; a temperature information acquisition unit for obtaining a distribution of temperature information based on infrared information; a suspect target identification unit that identifies a suspect target that is an imaging target that may be at risk of failure based on a distribution of temperature information and a determination target value that indicates a variation between a high temperature range and a medium temperature range and a first threshold value; a determination unit that classifies the disorder of the imaging target based on the pattern of high temperature parts in the identified suspect target, and in the case of a specific classification, identifies the suspect target again based on the judgment target value of the suspect target and a second threshold value; Equipped Status determination system. The third embodiment can be configured independently instead of being combined with the first or second embodiment.

[0124] [Target detection system] Next, a description will be given of a determination object detection system that can be combined with the above-described embodiments and their modified examples, as well as the technical ideas shown in the corresponding claims. The determination object detection system will be described in four to ten embodiments.

[0125] [Fourth embodiment] The object detection system 400 of this embodiment receives infrared images, positioning information, shooting direction information, etc. from the flying object D, similarly to the above embodiment, and identifies the shooting object. As shown in Fig. 15, the object detection system 400 includes a position information receiving unit 410, an image receiving unit 420, an identifying unit 430, a temperature information acquiring unit 440, a control unit C, and a storage unit S. The position information receiving unit 410 receives, from the flying object D, positioning information (such as a three-dimensional position) of the flying object D based on the navigation signal, etc. For example, this may be received as information added to the image as EXIF ​​information.

[0126] The image receiving unit 420 receives an infrared image from the photographing unit D1 of the flying object D. The identifying unit 430 identifies a predetermined imaging target or a group of imaging targets from the infrared image. The imaging target is, for example, a solar power generation panel. Other imaging targets may include a large number of automobiles, the structure of a building (glass, etc.), a large number of various sensors, mass-produced products that are inspected, etc. In the following, a solar power generation panel will be given as an example of an imaging target or a group of imaging targets.

[0127] For the infrared image received by the image receiving unit 420, the identifying unit 430 detects each solar power generation panel in the image by inferring it using deep learning, for example. This uses an inference model created by deep learning using so-called artificial intelligence (AI). The inference model has a teacher data group consisting of a plurality of data that identifies each solar power generation panel from the infrared image in advance.

[0128] The artificial intelligence performs deep learning using the group of teacher data, and generates an inference model. In the identification unit 430, inference is performed on the received image using the inference model, and the coordinates of, for example, the outer edge of each photovoltaic power generation panel are identified from the infrared image (the coordinates may be the four corners of the photovoltaic power generation panel instead of the outer edge).

[0129] In this embodiment, the method of detecting a photovoltaic power generation panel from an infrared image is not limited to deep learning. Other methods such as pattern matching can also be used. In addition, the detection of a photovoltaic power generation panel does not have to be performed for each photovoltaic power generation panel, but may be performed for a group of photovoltaic power generation panels arranged in two vertical columns and two horizontal rows, or a group of photovoltaic power generation panels arranged in two vertical columns and one horizontal row. This detection unit may be determined for each determination target detection system and may not be changed, or may be set by a user's operation via an input unit (not shown).

[0130] In addition, the above configuration detects the imaging target in an infrared image. However, this embodiment is not limited to this. For example, the imaging target can be specified by the following modified example. That is, in this system, visible light images and infrared images are received from the flying object D, and both can be used. Furthermore, the visible light images and infrared images are taken at the same angle. Therefore, the position of the solar power generation panel can be specified not only in the infrared image, but also in the visible light image. In this case, the specification unit 430 specifies the position of the imaging target in the visible light image by any of the above inference techniques in the visible light image.

[0131] Next, the coordinates of the identified region of the imaging target in the visible light image are used directly in the coordinates of the infrared image by the identification unit 430. This is based on the premise that the angle of view, angle, scale, etc. of the visible light image and the infrared image are the same or substantially the same (within the allowable range of the difference in the panel positions in both images).

[0132] Alternatively, when the angle of view, angle, scale, etc. are different between the visible light image and the infrared image, and it is not permissible to replace the coordinates of one image with the coordinates of the other image as they are (the difference is large), the identification unit 430 may be configured to perform scale and position alignment using some landmark, marker, or morphological features and positional features of the imaging target. Furthermore, the position alignment is not limited to this.

[0133] For example, the identification unit 430 acquires the signal transmission time from a navigation signal from a global positioning satellite system as additional information for the visible light image. It also acquires the horizontal movement speed from a speed sensor or the like of the flying object D. It also compares the time attached to the visible light image with the time attached to the infrared image to determine the time difference and the time difference between the images.

[0134] Furthermore, the identifying unit 430 determines the amount of positional shift between both images (the distance between the photographing positions) based on the moving speed and the time difference, etc., and determines in what direction and by how much the two images are shifted based on the composite pitch angle θ corresponding to each of the visible light image and the infrared image and the amount of shift. Furthermore, when there is a difference in the angle of view, for example when the visible light image has a wider area, the identifying unit 430 can reduce the processing load or improve the accuracy of the processing result when performing processing to identify the range indicated by the infrared image in the visible light image based on the difference information thus identified.

[0135] Furthermore, since it is possible to determine the positional correspondence between the visible light image and the infrared image, it is also possible to unify the angles of view of both images or change the ratio of the angles of view of both images to any desired ratio.

[0136] The above-described configuration in which the detection of the imaging target is performed in a visible light image and the coordinates of the identified area are applied to an infrared image can also be applied to the following modified examples and other embodiments.

[0137] The temperature information acquisition unit 440 acquires temperature information within the panel for each photovoltaic power generation panel in the infrared image identified by the identification unit 430 using any known technique.

[0138] (Control section C configuration) For convenience of explanation, the processing of each unit in the above explanation has been explained as being executed under the control of the control unit C shown in Fig. 1. The explanation of this control unit C is omitted as it overlaps with the explanation of the above embodiment. Note that the multiple components in this embodiment (for example, at least two or more of the location information receiving unit 410, image receiving unit 420, identification unit 430, and temperature information acquisition unit 440) may be integrated into one processor to realize the functions.

[0139] (Control unit C - Judgment process) The control unit C also executes the judgment process in the judgment target detection system. For example, the control unit C can judge a faulty panel from an infrared image based on the temperature difference and pattern. As a specific example, the control unit C determines whether or not the panel contains an abnormality by calculating the average temperature of all pixels or the temperature distribution of all pixels based on the temperature information of each pixel contained therein, and comparing the average temperature of all pixels or the temperature distribution of all pixels with a threshold value or an appropriate range based on the number of pixels showing a high temperature. The control unit C may also store in advance the shape of a group (pattern) of pixels exhibiting abnormal temperatures, and identify the specific type of abnormality by comparing with the pattern.

[0140] Furthermore, if an abnormality is found in each photovoltaic panel as a post-processing of the judgment process, the control unit C performs a process of at least identifying the position of that panel and presenting it to the user in a manner of their choice (see "DIFF" for the panel in the fourth row vertically and sixth column horizontally in the upper block of Fig. 3, or Figs. 39 and 40, etc.). In addition to the panel position, the control unit C may also execute a process of creating and displaying the maximum temperature, average temperature, or number of abnormal pixels for each photovoltaic panel as image data. Note that the control unit C corresponds to an example of a "judgment unit" in this process.

[0141] [Operation] Fig. 16 is a diagram showing the flow of processing in this embodiment. Below, with reference to Fig. 16, the flow of processing in the determination object detection system 400 will be explained along with step numbers (S041 to S045).

[0142] (S041) The position information receiving unit 410 in the determination object detection system 400 receives position information from the flying object D. Furthermore, the image receiving unit 420 receives an infrared image.

[0143] (S042) The identification unit 430 identifies the photovoltaic power generation panel in the infrared image.

[0144] (S043) The temperature information acquisition unit 440 acquires temperature information within each photovoltaic panel in the infrared image identified by the identification unit 430. This may be performed for each pixel, or a plurality of pixels may be treated as one unit, and an average value for each pixel group unit may be used as the temperature information.

[0145] (S044) The control unit C detects obstructed panels from the infrared images based on temperature differences and patterns.

[0146] (S045) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0147] [effect] According to this embodiment, the error in the abnormality judgment of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for the abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error in the abnormality judgment.

[0148] [Variations] A determination object detection system 400 according to a modified example of the fourth embodiment will be described with reference to Fig. 17 and Fig. 18. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

[0149] (overview) As in the fourth embodiment, in this modification, the imaging target in the infrared image is identified from other objects, and the temperature information of the imaging target is excluded from the information used for anomaly determination. That is, the photovoltaic power generation panels are first identified, and anomaly determination is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, errors in anomaly determination due to the arrangement of the photovoltaic power generation panels are also reduced.

[0150] The determination target detection system 400 in this modified example detects the positions of the solar power generation panels by, for example, detecting the solar power generation panels using an AI (deep learning) inference model or pattern matching, and by using arrangement information of the solar power generation panels.

[0151] (Object Detection System 400) 17, a determination target detection system 400 of this modification includes a position information receiving unit 410, an image receiving unit 420, an identifying unit 430, a temperature information acquiring unit 440, a control unit C, a storage unit S, and a setting unit 410a. The identifying unit 430 detects a solar power generation panel from an infrared image, similar to the fourth embodiment.

[0152] (Setting unit 410a) As shown in FIG. 2A, FIG. 2B and FIG. 3, the imaging target such as a photovoltaic power generation panel may have a specific arrangement for each installation site. Alternatively, the imaging target may have a specific arrangement tendency. In either case, by setting in advance arrangement information indicating the manner or state of the arrangement, it is possible to reduce identification errors by using the arrangement information for the position of the imaging target detected from the infrared image. For example, it is possible to detect a photovoltaic power generation panel from an infrared image based on the arrangement information and infrared information (or temperature information) indicated by the imaging target in the infrared image. In this example, as shown in FIG. 3, one or more blocks of photovoltaic power generation panels arranged two-dimensionally with four panel groups arranged vertically in seven columns horizontally and seven panel groups arranged vertically in four rows horizontally are included in the infrared image. When the user inputs the arrangement information of the infrared image by an input unit (not shown), the setting unit 410a stores the arrangement information in the storage unit S. In the case of a solar panel, the temperature information presented in the image differs between the panel (e.g., silicon) and the outer frame (e.g., aluminum), so one example of detection is to detect the frame using infrared information or the like and identify each panel. However, this modified example is not limited to a configuration in which the user inputs the array information. For example, it is also possible to dynamically derive the number of arrays. In this case, the derived number of arrays is used to reduce identification errors.

[0153] As another example, in deep learning, in addition to the above-mentioned teacher data group, it is also possible to use this sequence information in the form of multimodal learning.

[0154] The identification unit 430 detects the photovoltaic power generation panel from the infrared image in the same manner as in the first embodiment, and further identifies the photovoltaic power generation panel by using the arrangement information in the storage unit S. A first example will be given. When the arrangement information is set to have a photovoltaic power generation panel group of 4 rows and 7 columns as shown in FIG. 3, the photovoltaic power generation panel may detect the photovoltaic power generation panel in a manner different from the arrangement information. For example, when the arrangement information is set to 4 rows, the number of rows is less than 4 rows, or the number of rows is more than 4 rows, or when the arrangement information is set to 7 columns, the number of rows is less than 7 columns, or the number of columns is more than 7 columns. In this case, the identification unit 430 may perform detection again and repeat the detection until there is no difference from the arrangement information. In addition, the identification unit 430 may be configured to notify the detection error in any manner, such as by outputting an error message to a display unit (not shown) or by outputting a warning sound to an audio output unit (not shown).

[0155] Next, a second example will be given. When the arrangement information includes a matrix-shaped photovoltaic panel group as shown in FIG. 3, the outline shape of the outer edge of the panel group is, for example, a rectangle. Therefore, after detecting a photovoltaic panel, the identification unit 430 further identifies the outer edge of the photovoltaic panel group based on the arrangement information. Furthermore, the identification unit 130 identifies a photovoltaic panel detected at a position outside the outer edge as a detection error. When a detection error occurs, the processing of the identification unit 430 is the same as in the first example. Other examples of processes that the determination object detection system 400 may take after determining that a detection error has occurred will be described in detail after the description of the tenth embodiment (Modification 1 and onwards).

[0156] [Operation] Fig. 18 is a diagram showing a process flow in this modified example. Hereinafter, with reference to Fig. 18, the process flow of the determination object detection system 400 will be described along with step numbers (S041a to S045a).

[0157] (S041a) A position information receiving unit 410 in the determination object detection system 400 receives position information from the flying object D. In addition, an image receiving unit 420 receives a visible light image and an infrared image.

[0158] (S042a) The identification unit 430 identifies the photovoltaic power generation panel in the infrared image.

[0159] (S013a) The identifying unit 430 determines whether there is a detection error for the multiple photovoltaic panels identified in S042a based on the arrangement information set in the setting unit 410a. If there is a detection error (S043a; Yes), the identifying unit 430 repeats the detection until the detection error is eliminated. However, the present invention is not limited to this, and may be configured to notify the user of the detection error and end the inspection of the photovoltaic panels.

[0160] (S044a) If there is no detection error or the detection error has disappeared as determined in S043a (S043a; No), the temperature information acquisition unit 140 acquires temperature information within the panel for each photovoltaic power generation panel in the infrared image identified by the identification unit 430. This may be performed for each pixel, or multiple pixels may be treated as one unit and an average value for each pixel group unit may be used as temperature information.

[0161] (S045a) The control unit C detects obstructed panels from the infrared images based on temperature differences and patterns.

[0162] (S046a) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0163] [effect] According to this embodiment, the error in the abnormality judgment of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for the abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error in the abnormality judgment.

[0164] Furthermore, according to this embodiment, since a detection error of a photovoltaic power generation panel is determined based on the arrangement information, it is possible to reduce the occurrence of missed detection of a fault in a photovoltaic power generation panel.

[0165] [Fifth embodiment] A determination object detection system 500 according to the fifth embodiment will be described with reference to Fig. 19 to Fig. 21. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

[0166] (overview) As in the fourth embodiment, the fifth embodiment also performs a process of identifying the imaging target and other objects in the infrared image, and excluding temperature information of objects other than the imaging target from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, errors in abnormality judgment due to the arrangement of photovoltaic power generation panels are also reduced.

[0167] In the fifth embodiment, the inclination of the image is corrected in order to detect the imaged object such as the solar power generation panel in the fourth embodiment. In one example of the fourth embodiment, object detection is performed based on the physical characteristics of the imaged object by using an inference model using deep learning technology, pattern matching, and other inference technology. In the case of a solar power generation panel, for example, the area of ​​the imaged object is inferred based on its morphological characteristics (such as a rectangular outer edge). However, when the imaging direction is inclined with respect to the object to be photographed, even if a rectangular area is inferred as shown in the left diagram of FIG. 20, there is a risk that other adjacent objects to be photographed or other objects that are not the object to be photographed may be included in the detected rectangular area. Therefore, in the fifth embodiment, a predetermined position is specified for each of the multiple objects to be photographed, and the inclination of the image is calculated from the positional relationship of each of the positions. Furthermore, the calculated inclination is corrected to reduce detection errors of the object to be photographed.

[0168] (Object detection system 500) 19, the determination target detection system 500 of the fifth embodiment includes a position information receiving unit 510, an image receiving unit 520, an identifying unit 530, a temperature information acquiring unit 540, a control unit C, a storage unit S, and a first correction unit 510a. The identifying unit 530 detects a solar power generation panel from an infrared image, similar to the first embodiment.

[0169] (First correction unit 510a) As in the first embodiment, an imaging target, for example, a solar power generation panel, is identified from an infrared image. The first correction unit 510a further identifies predetermined positions of the identified multiple solar power generation panels. For example, as shown in the left diagram of FIG. 20, the position of the upper left corner of each identified imaging target area (coordinates within the image, hereinafter referred to as the "specific position") is identified. Next, the first correction unit 510a calculates a regression line that minimizes the distance to each specific position by the least squares method.

[0170] Next, the first correction unit 510a calculates the inclination of this regression line for the infrared image. The first correction unit 510a also performs correction by rotating the image so that this inclination is eliminated. The identification unit 530 executes the identification process of the imaging target again after the rotation correction. Note that the first correction unit 510a may be configured not to execute the correction process if the inclination is within a predetermined range (e.g., ±5°) or less.

[0171] Furthermore, the first correction unit 510a may determine that the image is an imaging error if the inclination is equal to or greater than a predetermined range (e.g., ±5°). As a result of the error determination, a process may be executed to exclude the image from the target of panel abnormality determination (obstructed panel determination). Even if the image is excluded from the target of abnormality determination, the following method can be used to deal with the problem. For example, at least one of azimuth information and position information is associated with the infrared image. Therefore, it is possible to identify the imaging target group associated with the excluded image. In other words, it is possible to take an aerial photograph of the position again and acquire the image again. It is also possible for the user to directly inspect only that part.

[0172] In addition, other countermeasures when an image is excluded from the target of abnormality judgment are listed. In aerial photography by an aircraft, there is a case where a deviation occurs between the imaging direction to be controlled and the actual imaging direction due to the influence of the environment at that time (wind, air pressure, and other factors). To solve this problem, the aircraft D may be controlled to perform aerial photography of one imaging target multiple times, that is, to perform overlapping photography. In the judgment target detection system 500, it is possible to configure it so that two or more images are acquired for one imaging target in response to the overlapping photography. With such a configuration, even if one image is excluded according to the degree of tilt, it is possible to complement it with other images. That is, if the tilt in the other images is within a predetermined range, it is possible to perform an abnormality (obstructed panel) judgment in that image. If the tilt is large, it is assumed that other problems other than the tilt, such as image blurring and crushing (reduced resolution, etc.), are occurring. Therefore, the judgment of tilt by the first correction unit 510a can also be performed to reduce abnormality judgment errors due to other factors.

[0173] The degree of overlap in the overlapping photography in the above other examples is, for example, anywhere in the range of 10% to 95%. Also, if the time interval between aerial photography is short, the influence of the aerial photography environment (strong winds, etc.) does not change, and the image immediately following the image determined to be excluded may also exceed the allowable range of tilt. Furthermore, if the overlapping range is too wide, this may lead to an increase in the aerial photography time, as well as an increase in the processing time and processing load on the system. Taking these factors into consideration, the degree of overlap in overlapping photography may be, for example, 90% or less.

[0174] On the other hand, if the overlapping range is too narrow, there is a possibility that some images will not be included in the judgment of the defective panel when one image is excluded. Taking this into consideration, the degree of overlapping may be set to 50% or more.

[0175] Alternatively, the appropriate degree of overlap may be calculated using the wind speed above the shooting location, and the degree of overlap may be set each time according to the calculation result. Depending on the degree of overlap, an image immediately following an image that has been removed due to a large tilt may be removed without going through tilt determination.

[0176] [Operation] Fig. 21 is a diagram showing the flow of processing in this embodiment. Below, with reference to Fig. 21, the flow of processing in the determination object detection system 500 will be explained along with step numbers (S051 to S056).

[0177] (S051) A position information receiving unit 510 in the determination object detection system 500 receives position information from the flying object D. Furthermore, an image receiving unit 520 receives a visible light image and an infrared image.

[0178] (S052) The identification unit 530 identifies the photovoltaic power generation panel in the infrared image.

[0179] (S053) The first correction unit 510a identifies a predetermined position in each of the multiple imaging targets, and calculates a regression line based on the least squares method from the positional relationship of each position. The first correction unit 510a also calculates the inclination of the regression line with respect to the image. The first correction unit 510a further corrects the calculated inclination. The identification unit 530 again identifies the photovoltaic power generation panel.

[0180] (S054) The temperature information acquisition unit 540 acquires temperature information within each photovoltaic panel in the infrared image identified by the identification unit 530. This may be performed for each pixel, or a plurality of pixels may be treated as one unit, and an average value for each pixel group unit may be used as temperature information.

[0181] (S055) The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) from the infrared images based on the temperature difference and pattern.

[0182] (S056) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0183] [effect] According to this embodiment, the error of abnormality judgment (obstructed panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included in a large amount, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, there may be positions that are exposed to direct sunlight and positions that are not due to shadows, and for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array. In this respect, too, the present embodiment can reduce the error of abnormality judgment.

[0184] Furthermore, according to this embodiment, since the inclination that may be a cause of a detection error of a photovoltaic power generation panel once it has been detected is corrected, it is possible to reduce the number of errors in detecting a fault in the photovoltaic power generation panel.

[0185] [Sixth embodiment] A determination object detection system 600 according to the sixth embodiment will be described with reference to Fig. 22 to Fig. 24. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

[0186] (overview) Similarly, in the sixth embodiment, the imaging target in the infrared image is identified from other objects, and temperature information of objects other than the imaging target is excluded from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, errors in abnormality judgment due to the arrangement of the photovoltaic power generation panels are also reduced.

[0187] The target detection system 600 of the sixth embodiment further determines whether there is a deviation in the detected size of the target after detecting the target, in order to reduce errors in the detection process of the target such as a solar power generation panel in the fourth embodiment.

[0188] In one example of the fourth embodiment, object detection is performed based on the physical characteristics of the imaging target using an inference model using deep learning technology, pattern matching, and other inference technology. In the case of a solar power generation panel, for example, the area of ​​the imaging target is inferred based on its morphological characteristics (such as a rectangular outer edge). However, the outer edge of the imaging target may not be clearly depicted in the infrared image. As a result, even if a rectangular area is inferred, there is a risk that other imaging targets adjacent to the detected rectangular area or other objects that are not imaging targets may be included as shown in FIG. 23. Therefore, in the sixth embodiment, the size of each of the multiple imaging targets is calculated, and a detection error is determined based on the calculated size. This aims to reduce detection errors of the imaging target.

[0189] That is, depending on the type of imaging target, the size may be uniform (such as a solar power generation panel). Therefore, taking advantage of the uniform size, if there are imaging targets detected as large (e.g., L6 in FIG. 23) and small, the determination target detection system 600 calculates this as a variation in size, and as a result, determines whether there has been a false detection.

[0190] For example, in the case of solar panels, within the same power plant, the areas and shapes of the light receiving surface, the surface irradiated with sunlight, etc. are configured to be roughly the same for construction reasons, operational reasons, etc. Construction reasons include consideration of land use and the surface irradiated with sunlight, while operational reasons include consideration of status monitoring, replacement in the event of a breakdown, etc.

[0191] Therefore, the vertical length, horizontal length, circumference, area, diagonal length, etc. are formed so that there is no large difference between each panel. According to this premise, if photovoltaic power generation panels of obviously different sizes are detected from the infrared image, there is a possibility that it is a detection error. Since this detection error interferes with the abnormality judgment result of the temperature information in the judgment object detection system, the judgment object detection system 600 has a configuration that can judge a detection error when there is an imaging object of different size. Note that even if it is another imaging object, if it is detected as being out of the predetermined regularity specific to the imaging object, it can be judged as a detection error, and thus even if the imaging object is not a photovoltaic power generation panel, it is included in one example of this embodiment.

[0192] (Object Detection System 600) 22, the determination object detection system 600 of the sixth embodiment includes a position information receiving unit 610, an image receiving unit 620, an identifying unit 630, a temperature information acquiring unit 640, a control unit C, a storage unit S, and a first determination unit 610a. The identifying unit 630 detects a solar power generation panel from an infrared image, similar to the first embodiment.

[0193] (First determination unit 610a) As in the first embodiment, an imaging target, for example, a solar power generation panel, is identified from the infrared image. The first determination unit 610a further identifies a predetermined position in the identified multiple solar power generation panels. For example, as shown in Fig. 23, any position on the upper edge of the area of ​​each identified imaging target (Y coordinate in the image, hereinafter referred to as "specific position") is identified. Next, the first determination unit 610a calculates the length in the Y axis direction at the coordinate on the image in the infrared image.

[0194] The length of the object in the Y-axis direction may be calculated at the center position in the X-axis direction, or at one or the other end of the X-axis direction. That is, the length of each object in the Y-axis direction at a predetermined position in the X-axis direction is calculated. For example, the first determination unit 610a calculates the difference between the coordinates of the top end (top end in the image) position and the coordinates of the bottom end as the length in the Y-axis direction. Also, for each object, multiple (predetermined number) lengths in the Y-axis direction may be calculated and averaged. In this case, the predetermined number is calculated at most as the number of pixels in the X-axis direction in the image.

[0195] Next, the first determination unit 610a compares the calculated feature amount or a value calculated based on the feature amount with the determination value. In the above example, the determination value is compared with a value indicating the size of the imaging target itself or a value calculated based on the size. An example will be described below.

[0196] After calculating the length of each photovoltaic panel in the Y-axis direction as described above, the first determination unit 610a calculates the variance value of the lengths of the photovoltaic panels. The variance value is calculated, for example, by the following formula. JPEG0007629289000004.jpg30119The first determination unit 610a subtracts the average value from the length value of each detection area to obtain the distance (deviation) between each length and the average value.The first determination unit 610a also obtains the variance by taking the mean square of the deviation.The larger the variance value, the greater the dispersion of the lengths of each detection area from the average value.

number

[0197] The first determination unit 610a also compares the variance value with the determination value, and if they are in a predetermined relationship, determines that there is a detection error. For example, the variance value of the lengths of all the photovoltaic panels included in one infrared image is calculated, and the variance value is compared with the determination value (e.g., the difference is calculated). In this case, if the difference between the variance value and the determination value is not within a predetermined range, the first determination unit 610a may determine that there is a detection error. In other words, both the case where a panel with the above-mentioned length being too long and the case where a panel with the above-mentioned length being too short are detected are determined to be an error.

[0198] It is also possible to use the length in the X-axis direction and the area of ​​the detected imaging target for the judgment, instead of the length in the Y-axis direction. In a solar power generation facility where solar power generation panels are arranged, the solar power generation panels tend to be arranged facing south in consideration of the power generation efficiency. In this case, the power generation equipment (power controller, cable) is often not arranged on the north-south side of the solar power generation panel group. Similarly, it is assumed that plants such as trees adjacent to the north-south direction are excluded as much as possible. In other words, consideration is given to not casting a shadow on the light receiving surface. Therefore, the temperature variation is suppressed in the north-south direction, that is, in the Y-axis direction in the infrared image. From the above, in the infrared image showing the temperature information of the solar power generation panel, the edge of the imaging target in the Y-axis direction is easily detected with high accuracy. In this embodiment, the size variation is large, so that the appropriateness of detection of the imaging target is judged, and therefore an example using the length in the Y-axis direction has been described. If the imaging target is not a rectangular solar power generation panel, the size of the imaging target to be used may be set according to the characteristics of the imaging target itself or the location of the imaging target.

[0199] The first determination unit 610a may be configured to obtain the variance from the population mean of the entire power plant, instead of the sample mean in the captured image. Alternatively, the first determination unit 610a may be configured to obtain the standard deviation, instead of the variance. In that case, the judgment value is a judgment value corresponding to the standard deviation. The first determination unit 610a may be configured to compare the average value with the judgment value. In that case, the judgment value is a judgment value corresponding to the average value. The first determination unit 610a may be configured to compare the size of each captured object (e.g., the length in the Y-axis direction) itself with the judgment value to determine the detection error.

[0200] As an example, when the first determination unit 610a determines a detection error, it excludes the image from the temperature abnormality determination target (target for faulty panel determination). As another example, when the first determination unit 610a determines a detection error, it may be configured to exclude only imaging targets with large size variations and perform temperature abnormality determination. The magnitude of size variation can be determined, for example, by whether the difference between the calculated value of the length in the Y-axis direction of one imaging target and the average length of each imaging target is within a predetermined range. It is also possible to determine whether the magnitude of size variation of each imaging target is within an allowable range by standard deviation or variance.

[0201] [Operation] Fig. 24 is a diagram showing a process flow in this embodiment. The process flow of the determination object detection system 600 will be described below along with step numbers (S061 to S067-2) with reference to Fig. 24.

[0202] (S061) A position information receiving unit 610 in the determination object detection system 600 receives position information from the flying object D. In addition, an image receiving unit 620 receives a visible light image and an infrared image.

[0203] (S062) The identification unit 630 identifies the photovoltaic power generation panel in the infrared image.

[0204] (S063) The first determination unit 610a determines a predetermined size (for example, the length in the Y-axis direction) of each of a plurality of imaging targets. The first determination unit 610a also determines the variation in size of the imaging targets.

[0205] (S064) The first determination unit 610a determines the magnitude of the variation in size calculated in S063 based on the determination value. If it is determined that the variation is large, the process proceeds to S065. If it is determined that the variation is not large, the process proceeds to S065-2.

[0206] (S065) If there is an image determined to have a large variation in size in S064 (S064; Yes), the first determination unit 610a determines that a detection error has occurred. Furthermore, in this case, the image including the object determined to have a large variation is excluded from the object to be determined for temperature abnormality. As another example, the first determination unit 610a may exclude only the imaging object having a large variation in size and include the image in the object to be determined for temperature abnormality in S066. The temperature information acquisition unit 640 acquires temperature information for each imaging object that is not excluded.

[0207] (S066) The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on infrared images other than the excluded images (or detection areas other than the imaging target of the detection error) by temperature difference and pattern.

[0208] (S067) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0209] (S065-2) If it is determined in S064 that the size variation in all images is not large (for example, within a predetermined range) (S064; No), the first determination unit 610a determines that there is no detection error. In this case, all images that were determined in S064 as being suitable for detection are included as targets for temperature abnormality determination. The temperature information acquisition unit 640 acquires temperature information for each image capture target.

[0210] (S066-2) The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on the temperature difference and pattern of all infrared images (all detected detection areas) that have been judged for temperature abnormalities by the first judgment unit 610a.

[0211] (S067-2) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0212] [effect] According to this embodiment, the error of abnormality judgment (obstructed panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

[0213] Furthermore, according to this embodiment, the detection error of the imaging target is judged based on the variation in size, and if the variation is large, the image or the imaging target judged as an error is appropriately excluded from the target for temperature anomaly judgment, thereby reducing the number of errors in identifying the defective panel.

[0214] [Seventh embodiment] A determination object detection system 700 according to the seventh embodiment will be described with reference to Fig. 25 to Fig. 27. In the following description, descriptions that overlap with the seventh embodiment will be omitted.

[0215] (overview) As in the fourth embodiment, the seventh embodiment also performs a process of identifying the imaging target and other objects in the infrared image, and excluding temperature information of objects other than the imaging target from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, errors in abnormality judgment due to the arrangement of photovoltaic power generation panels are also reduced.

[0216] In order to reduce errors in the detection process of the imaging target such as a solar power generation panel in the fourth embodiment, the determination target detection system 700 of the seventh embodiment further detects whether there is a deviation in the alignment position of the imaging target after detecting the imaging target. For example, it is determined whether there is a false detection in the detection of the boundary part of the outer edge of the imaging target or the area of ​​each imaging target. In one example of the fourth embodiment, object detection based on the physical characteristics of the imaging target is performed using an inference model using deep learning technology, an inference technology such as pattern matching. In the case of a solar power generation panel, for example, the area of ​​the imaging target is inferred based on its morphological characteristics (such as a rectangular outer edge). However, the outer edge of the imaging target may not be clearly drawn in the infrared image. For example, as shown in FIG. 26, even if a rectangular area is inferred, the detected rectangular area may deviate from the position of the actual imaging target, or the detected area may partially protrude (in the X-axis direction, Y-axis direction, etc.; see FIG. 23) from the range of the area of ​​the actual imaging target. Therefore, in the seventh embodiment, the regularity of a predetermined arrangement specific to the imaging target is used to determine whether there is any deviation from the regularity. For example, a predetermined position in each of the multiple imaging targets identified from the image is further identified, and the amount of deviation of the identified position from the arrangement rule is calculated. If the amount of deviation is large, a detection error is determined. This aims to reduce detection errors of the imaging targets.

[0217] For example, in the case of solar panels, the areas and shapes (e.g., rectangles) of the light receiving surface and the sunlight irradiating surface are generally the same, and they tend to be regularly arranged two-dimensionally in rows and columns. Therefore, it is assumed that the outer edges of each of the multiple solar panels are aligned in both the row and column directions.

[0218] Using this premise, it is possible to determine detection errors by determining from the infrared image whether characteristic positions such as both ends in the Y-axis direction, both ends in the X-axis direction, corners, and the center position of the light receiving surface are aligned or misaligned in a predetermined direction. Note that the predetermined direction is, for example, the Y-axis direction or the X-axis direction, or a direction inclined at a predetermined angle with respect to the Y-axis direction and the X-axis direction.

[0219] Note that even with other imaging targets, if an object is detected that deviates from a predetermined arrangement shape specific to the imaging target, such as an arrangement in concentric circles or a regular wave-like pattern, it can be determined that this is a detection error. Thus, even if the imaging target is not a solar power generation panel, this embodiment also includes such an example.

[0220] (Object Detection System 700) 25, the determination object detection system 700 of the seventh embodiment is configured to include a second determination unit 710a in addition to a position information receiving unit 710, an image receiving unit 720, an identifying unit 730, a temperature information acquiring unit 740, a control unit C, and a storage unit S. The identifying unit 730 detects a solar power generation panel from an infrared image, similar to the fourth embodiment.

[0221] (Second Determination Unit 710a) As in the first embodiment, an imaging target, for example, a solar power generation panel, is identified from an infrared image. The second determination unit 710a further identifies a predetermined position in the identified multiple solar power generation panels. For example, as shown in FIG. 26, the second determination unit 710a determines a position of one of the upper edges of the region of each imaging target identified by the identification unit 730 (Y coordinate in the image, hereinafter referred to as “Ymin” or “Ymin n Next, the second determination unit 710a identifies the position of the bottom edge of the region (Y coordinate in the image, hereinafter referred to as "Ymax" or "Ymax n ").

[0222] Next, the second determination unit 710a determines the Ymin i Variation and Ymax i The sum of these is the alignment variance. The variance value is calculated, for example, using the following formula: JPEG0007629289000006.jpg51114

number

[0223] Not limited to the above, the second determination unit 710a determines whether each imaging target Y′min n Variation of Y´max n It is also possible to obtain the variance of Y´min n and Y´max n (See FIG. 26 for details.) The variance value is calculated, for example, by the following formula: JPEG0007629289000008.jpg42130

number

[0224] The second judgment unit 710a also compares this variance value with the judgment value, and if there is a predetermined relationship between them, judges it as a detection error. For example, it calculates the variance value of the Y coordinates of all the photovoltaic power generation panels included in a predetermined arrangement direction (row direction, etc.), and compares the variance value with the judgment value (for example, calculates the difference). In this case, if the difference between the variance value and the judgment value is not within a predetermined range, the second judgment unit 710a may judge it as a detection error. In other words, the above Ymin1 to Ymin n (Top edge), Ymax1~Ymax n When a panel is detected whose alignment position exceeds the tolerance (bottom edge), it is determined to be an error.

[0225] It is also possible to use the X coordinate for the determination, instead of the Y coordinate. The reason why an example of the variation in the alignment position using the Y coordinate is described in this embodiment is that, as described in the sixth embodiment, in an infrared image showing the temperature information of a photovoltaic power generation panel, the edge (upper end / lower end, Y coordinate) of the imaging target in the north-south direction is easily detected with high accuracy. However, if the imaging target is another imaging target other than a rectangular photovoltaic power generation panel, it is sufficient to set which position (characteristic position (position that is easy to identify)) of the imaging target to be used according to the characteristics of the imaging target itself or the location where it is placed.

[0226] The second determination unit 710a may be configured to determine a standard deviation instead of a variance, in which case the determination value corresponds to the standard deviation.

[0227] As an example, when the second determination unit 710a determines that there is a detection error, it excludes the image from the target for temperature abnormality determination. As another example, when the second determination unit 710a determines that there is a detection error, it may be configured to exclude only the imaging targets with a large variation in the amount of deviation from the arrangement rule and perform the temperature abnormality determination. The magnitude of the variation in the amount of deviation from the arrangement rule can be determined, for example, by whether the difference between the Y coordinate value of one imaging target and the average value of the Y coordinates of each imaging target is within a predetermined range. It is also possible to determine whether the magnitude of the variation in the amount of deviation from the arrangement rule in each imaging target is within an allowable range by using standard deviation or variance.

[0228] [Operation] Fig. 27 is a diagram showing a process flow in this embodiment. Below, with reference to Fig. 27, the process flow of the determination object detection system 700 will be explained along with step numbers (S071 to S077-2).

[0229] (S071) A position information receiving unit 710 in the determination object detection system 700 receives position information from the flying object D. Furthermore, an image receiving unit 720 receives a visible light image and an infrared image.

[0230] (S072) The identification unit 730 identifies the photovoltaic power generation panel in the infrared image.

[0231] (S073) The second determination unit 710a identifies the Y coordinates of the upper and lower edges of each of the multiple imaging targets, and also finds the variation in these Y coordinates of the multiple imaging targets.

[0232] (S074) The second determination unit 710a determines the magnitude of the variation in the Y coordinate calculated in S073 based on the determination value. If it is determined that the variation is large, the process proceeds to S075. If it is determined that the variation is not large, the process proceeds to S075-2.

[0233] (S075) If there is an image determined to have a large Y coordinate variation (e.g., alignment variance) in S074 (S074; Yes), the second determination unit 710a determines that a detection error has occurred. Furthermore, in this case, the image of the object determined to have a large variation is excluded from the object of temperature abnormality determination. As another example, the second determination unit 710a may exclude only the imaging object with a large Y coordinate variation and include the image in the object of temperature abnormality determination in S076. The temperature information acquisition unit 740 acquires temperature information for each imaging object that is not excluded.

[0234] (S076) The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on infrared images other than the excluded images (or detection areas other than the imaging target of the detection error) by temperature difference and pattern.

[0235] (S077) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0236] (S075-2) If it is determined in S074 that the variation in the Y coordinate is not large (for example, within a predetermined range) in all images (S074; No), the second determination unit 710a determines that there is no detection error. In this case, all images that were used as the object of the determination of the appropriateness of detection in S074 are included as the object of the determination of the temperature abnormality. The temperature information acquisition unit 740 acquires temperature information for each imaging object.

[0237] (S076-2) The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on the temperature difference and pattern of all infrared images (all detected detection areas) that have been judged for temperature abnormalities by the second judgment unit 710a.

[0238] (S077-2) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0239] [effect] According to this embodiment, the error of abnormality judgment (faulty panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

[0240] Furthermore, according to this embodiment, the detection error of the imaging target is judged based on the deviation of the imaging target from the regularity of the predetermined arrangement, and if the deviation is large, the image or the imaging target judged as an error is appropriately excluded from the temperature anomaly judgment target, thereby reducing the number of errors in identifying the obstructed panel.

[0241] [Eighth embodiment] A determination object detection system 800 according to the eighth embodiment will be described with reference to Fig. 28 to Fig. 30. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

[0242] (overview) As in the fourth embodiment, the eighth embodiment also performs a process of identifying the imaging target and other objects in the infrared image, and excluding temperature information of objects other than the imaging target from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, errors in abnormality judgment due to the arrangement of photovoltaic power generation panels are also reduced.

[0243] In order to reduce errors in the detection process of the imaging target such as a photovoltaic power generation panel in the fourth embodiment, the determination target detection system 800 of the eighth embodiment detects the imaging target and then sequentially identifies the predetermined position of the imaging target according to the arrangement direction of the detected imaging target. Furthermore, the determination target detection system 800 judges a detection error according to the degree of overlap between the imaging target or group of imaging targets to be judged and the target whose predetermined position was identified immediately before. For example, if the center position of the imaging target is located within the area of ​​an adjacent imaging target, it is judged as detection noise. Furthermore, the imaging target (detection area) judged as detection noise is excluded from the obstruction panel judgment.

[0244] (Object detection system 800) 28, the determination object detection system 800 of the eighth embodiment is configured to include a position information receiving unit 810, an image receiving unit 820, an identifying unit 830, a temperature information acquiring unit 840, a control unit C, a storage unit S, and a third determination unit 810a. The identifying unit 830 detects a solar power generation panel from an infrared image, similar to the first embodiment.

[0245] (Third Determination Unit 810a) As in the fourth embodiment, an imaging target, for example, a solar power generation panel, is identified from an infrared image. The third determination unit 810a further identifies a predetermined position in the identified multiple solar power generation panels. For example, as shown in FIG. 29, the third determination unit 810a identifies any position of the left edge of the area of ​​each imaging target identified by the identification unit 830 (X coordinate in the image, hereinafter referred to as "Xmin"). Next, the third determination unit 810a identifies any position of the right edge of the area (X coordinate in the image, hereinafter referred to as "Xmax"). Furthermore, the third determination unit 810a identifies the center position of the area of ​​each imaging target (X coordinate in the image; hereinafter referred to as "CenterX").

[0246] Next, the third determination unit 810a determines whether the center position CenterX in the X-axis direction of each identified imaging target is included in the region of the adjacent imaging target. For example, as shown in Fig. 29, when CenterX is located between one end (e.g., the left end position in the X-axis direction; "Prev Xmin") and the other end (e.g., the right end position in the X-axis direction; "Prev Xmax") of the adjacent region in the X-axis direction, the third determination unit 810a may be configured to determine that the region of the determination target is included in the adjacent region.

[0247] In the description of this embodiment, the center position of the region of the imaging target to be judged is used because the solar power generation panel is taken as an example as described in the sixth embodiment. That is, in a solar power generation facility, the possibility of a shadow being cast on the solar power generation panel at the end side in the X-axis direction is relatively high compared to all panels. Therefore, it can be said that the technical difficulty is relatively higher in detecting the edge in the X-axis direction than in the Y-axis direction. Therefore, if it is judged as an error when the edge in the X-axis direction of the region of the imaging target to be judged is located between "Prev Xmin" and "Prev Xmax", the probability of error judgment will be high, and there is a possibility that error judgment will occur frequently depending on the infrared image. Therefore, frequent error judgment can be prevented by using, for example, the degree of overlap between the adjacent region and the judgment region being 30% or more as an index, or by using the center position as a reference as in this embodiment. For this purpose, the present embodiment is not limited to a configuration in which the reference position of the region of the imaging target to be determined is the center position. For example, the reference position may be any position that is at least 30% of the total length of the target region from both ends of the target region in the X-axis direction.

[0248] When calculating the degree of overlap as a ratio or percentage (e.g., "overlap of 30% or more") as described above, one example is to compare the area belonging to both with the area of ​​each judgment target region. Alternatively, it is also possible to determine the degree of overlap based on the ratio of the length from "Prev Xmin" to "Xmin" of the judgment target region to the length in the X-axis direction of the judgment target region.

[0249] It is also possible to use the Y-axis direction as the reference, instead of the degree of overlap in the X-axis direction. For example, when combining the sixth or seventh embodiment with this embodiment, if the sixth or seventh embodiment uses the displacement or protrusion of the region of each imaging target in the X-axis direction as the reference, it is possible to use the Y-axis direction as the reference for the positional displacement in this embodiment. By determining the detection error in both the X-axis and Y-axis directions, it is possible to prevent misjudgment of a faulty panel due to a detection error.

[0250] As an example, when the third determination unit 810a determines a detection error, it excludes the target image from the temperature abnormality determination target (target for faulty panel determination). As another example, when the third determination unit 810a determines a detection error, it may be configured to exclude only the image capture target with a large positional deviation and perform the temperature abnormality determination. The criterion for the magnitude of the positional deviation may be the same as the above-mentioned determination criterion, or may be changed.

[0251] [Operation] Fig. 30 is a diagram showing the flow of processing in this embodiment. Below, with reference to Fig. 30, the flow of processing in the determination object detection system 800 will be explained along with step numbers (S081 to S087-2).

[0252] (S081) A position information receiving unit 810 in the determination object detection system 800 receives position information from the flying object D. In addition, an image receiving unit 820 receives a visible light image and an infrared image.

[0253] (S082) The identification unit 830 identifies the photovoltaic power generation panel in the infrared image.

[0254] (S083) The third determination unit 810a identifies the X coordinate of a predetermined position in each of the multiple imaging targets, and determines the degree of overlap with adjacent regions based on the X coordinate. n and Xmax1~Xmax n and CenterX1~CenterX n Identify:

[0255] (S084) Next, the third determination unit 810a determines whether the CenterX of the determination target area is included in the area of ​​the adjacent imaging target. For example, as shown in FIG. n It is determined whether "Prev Xmin" and "Prev Xmax" of the adjacent region are located between each other.

[0256] (S085) If it is determined in S084 that the CenterX of the region to be determined is included in the region of the adjacent imaging target (S084; Yes), the third determination unit 810a determines that region as a detection error. Furthermore, in this case, the third determination unit 810a excludes the image of the target determined to be included in the adjacent region from the target for temperature abnormality determination (obstructed panel determination). As another example, the third determination unit 810a may exclude only the corresponding imaging target region and include the image in the target for temperature abnormality determination (obstructed panel determination) in S086. The temperature information acquisition unit 840 acquires temperature information for each imaging target that is not excluded.

[0257] (S086) The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on infrared images other than the excluded images (or detection areas other than the imaging target of the detection error) by temperature difference and pattern.

[0258] (S087) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0259] (S085-2) If it is determined in S084 that the CenterX of the determination area is not included in the adjacent area in all images (S084; No), the third determination unit 810a determines that there is no detection error. In this case, all images (all determined detection areas) that were the targets for determining whether the detection was correct in S084 are included as targets for determining temperature abnormality. The temperature information acquisition unit 840 acquires temperature information for each imaging target.

[0260] (S086-2) The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) by temperature difference and pattern, based on all infrared images that have been subject to temperature abnormality determination by the third determining unit 810a.

[0261] (S087-2) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0262] [effect] According to this embodiment, the error of abnormality judgment (faulty panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

[0263] Furthermore, according to this embodiment, the detection error of the imaging target is judged based on the degree of overlap between the imaging target area and the adjacent area, and if the degree of overlap is large, the image or the imaging target judged as an error is appropriately excluded from the temperature abnormality judgment target, so that it is possible to reduce the number of errors in identifying the obstruction panel.

[0264] [Ninth embodiment] A determination object detection system 900 according to the ninth embodiment will be described with reference to Fig. 31 to Fig. 34. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

[0265] (overview) As in the fourth embodiment, the ninth embodiment also performs a process of identifying the imaging target and other objects in the infrared image, and excluding temperature information of objects other than the imaging target from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, errors in abnormality judgment due to the arrangement of photovoltaic power generation panels are also reduced.

[0266] In order to reduce errors in the detection process of the imaging target such as a solar power generation panel in the fourth embodiment, the determination target detection system 900 of the ninth embodiment detects the imaging target and then sequentially identifies the predetermined position of the imaging target according to the arrangement direction of the detected imaging target. Furthermore, the determination target detection system 900 gathers together a plurality of imaging targets identified in the infrared image according to the regularity of the arrangement of the imaging targets (hereinafter, the collection is referred to as a "cluster"). In this embodiment, a predetermined number of imaging targets that should be included in a cluster at a predetermined position is stored. The determination target detection system 900 judges a detection error based on whether the number of detections in one cluster satisfies a predetermined number.

[0267] Furthermore, the judgment object detection system 900 may be configured to additionally execute a judgment of a detection error based on the distance between adjacent clusters (variation example described below). Furthermore, the judgment object detection system 900 excludes an imaging object (detection area) judged as a detection error from the judgment of an obstruction panel.

[0268] (Object Detection System 900) 31, the determination object detection system 900 of the ninth embodiment includes a position information receiving unit 910, an image receiving unit 920, an identifying unit 930, a temperature information acquiring unit 940, a control unit C, a storage unit S, and a fourth determination unit 910a. The identifying unit 930 detects a solar power generation panel from an infrared image, similar to the first embodiment.

[0269] (Fourth Determination Unit 910a) As in the first embodiment, an imaging target, for example, a solar power generation panel, is identified from the infrared image. The fourth determination unit 910a further identifies a predetermined position in the identified multiple solar power generation panels. For example, the imaging targets may be arranged two-dimensionally. In this case, it is possible to group imaging targets aligned in the X-axis direction in the infrared image as one cluster (see Cluster C to Cluster F) as shown in FIG. 32.

[0270] As an example, the fourth determination unit 910a identifies the Ymin and Ymax of the imaging target, such as the upper and lower edges in the Y-axis direction, or the corners on the upper and lower edges. Furthermore, the fourth determination unit 910a searches for imaging targets whose Ymin and Ymax are close to each other among the detected imaging targets, so that imaging targets aligned in the X-axis direction are included in one cluster. As an example, the fourth determination unit 910a uses the coordinate of Ymin of one detection area as a reference and compares it with the Ymin of other detection areas to identify those detection areas whose Y coordinates are within a predetermined distance from each other.

[0271] Similarly, the fourth determination unit 910a uses the coordinate of Ymax of one detection area as a reference and compares it with the Ymax of other detection areas to identify those other detection areas that are within a predetermined distance in terms of Y coordinate. In this way, the fourth determination unit 910a associates the one detection area with the identified other detection areas as belonging to one cluster. In this way, the fourth determination unit 910a identifies clusters and detection areas within the clusters in the infrared image.

[0272] The above cluster formation is performed by determining which Y coordinates are close in Ymin and Ymax, but this is only an example and the present embodiment is not limited to this. For example, the fourth determination unit 910a may form a cluster by determining which Y coordinates are close in at least one of Ymin and Ymax, or only in either one of them. As another example, a cluster may be formed by determining which Y coordinates are close based on the center point of the imaging target. Similarly, in the case of a polygonal imaging target, any one of the corners or any number of the corners may be used.

[0273] Next, the fourth determination unit 910a obtains the number of detection areas included in each cluster (see FIG. 32; Cluster A to Cluster F). The number of detected areas in this cluster is compared with a stored predetermined number. That is, the determination object detection system 900 compares the stored predetermined number that should be included in one cluster with the number of detected imaging objects actually included in the cluster. This predetermined number is derived from the characteristics of the imaging objects and their locations, and the regularity of the arrangement, and may be set by the user, for example.

[0274] If the number of detections in a cluster is less than a predetermined number, the fourth determination unit 910a determines that all detection areas belonging to the cluster are detection errors. In this case, as an example, the fourth determination unit 910a excludes the target image from the temperature abnormality determination target. As another example, the fourth determination unit 910a may be configured to exclude the detection area belonging to the cluster to be determined and perform the temperature abnormality determination. Note that, without being limited to the above, the configuration may be such that the imaging targets arranged in the Y-axis direction are grouped into one cluster.

[0275] [Variations] Next, a modified example of the ninth embodiment will be described. In this modified example, a process of grouping a plurality of imaging targets into one cluster is executed as in the ninth embodiment. However, in this modified example, the fourth determination unit 910a obtains the average value of Ymin and the average value of Ymax for each imaging target in a cluster, and sets these as the Ymin and Ymax of the cluster. The Ymin and Ymax of a cluster may be the median values ​​of Ymin and Ymax of each imaging target to which it belongs. Furthermore, the fourth determination unit 910a obtains the variance value of the distance to the nearest cluster in the Y-axis direction. In addition, the obtained variance value is compared with the determination value, and if the two have a predetermined relationship, it is determined as a detection error. The variance value is calculated, for example, by the following formula. JPEG0007629289000010.jpg37158

number

[0276] The fourth judgment unit 910a calculates the difference between the average distances of all clusters from the nearest cluster to the cluster to be judged and the distances of the nearest cluster to each cluster, and calculates the variance by the mean square of the deviation. The larger the variance, the greater the dispersion of each data from the average value.

[0277] In this modification, a judgment value for judging the variation in the interval between clusters is stored in a storage means (not shown). The fourth judgment unit 910a compares the variance value and the judgment value, and judges it as a detection error if they have a predetermined relationship. For example, the variance value of the interval between Ymin of the cluster to be judged and Ymax of the cluster above is calculated, and the variance value is compared with the judgment value (for example, the difference is calculated). Similarly, the variance value of the interval between Ymax of the cluster to be judged and Ymin of the cluster above is calculated, and the variance value is compared with the judgment value. In this case, if the difference between the variance value and the judgment value is not within a predetermined range, the fourth judgment unit 910a may judge it as a detection error. In the example of FIG. 33, the imaging targets are arranged two-dimensionally, and large intervals are not provided between the clusters in terms of the arrangement efficiency of the imaging targets. Therefore, if there is an interval longer than a predetermined interval on both the upper side and the lower side of the cluster to be judged, all the detection areas belonging to that cluster may be considered to be detection noise.

[0278] The above determination of the variation in the interval between clusters was made with respect to the interval between the Ymin of the determination target and the Ymax of the upper cluster. However, this is only an example, and the present embodiment is not limited to this. For example, the fourth determination unit 910a may determine the variation in the interval between the Ymin of the determination target and the Ymin of the upper (or lower) cluster. Also, for example, the fourth determination unit 910a may determine the variation in the interval between the Ymax of the determination target and the Ymax of the upper (or lower) cluster.

[0279] In the example of a photovoltaic panel, if the clusters are aligned in the X-axis direction as described above, they tend to be arranged adjacently in the Y-axis direction to form an array configuration. In this case, the theoretical optimum value of the difference between the Ymin of the cluster to be judged and the Ymax of the cluster located above is 0. In the example of the above embodiment in which the variance is compared, both the upward and downward deviations of the reference average value are likely to be judged as detection noise. Therefore, it can be said that the magnitude of such detection noise is more likely to be detected if it is in a single direction.

[0280] That is, the case where the average value is below the negative value can be said to be a state where clusters overlap, but this case can be removed by a technique such as that of the seventh embodiment. That is, if detection noise is aggregated to the above deviation (when a gap occurs between clusters), it becomes easier to reduce the occurrence of noise. Therefore, the explanation of this embodiment has been centered on the gap between the Ymin of the judgment target and the Ymax of the upper cluster.

[0281] As an example, when the fourth determination unit 910a determines that there is a detection error, the fourth determination unit 910a excludes the target image from the target for temperature abnormality determination. As another example, when the fourth determination unit 910a determines that there is a detection error, the fourth determination unit 910a may be configured to exclude the detection area belonging to the cluster to be determined and perform the temperature abnormality determination. Note that, without being limited to the above, the configuration may be such that the imaging targets arranged in the Y-axis direction are grouped into one cluster.

[0282] [Operation] Fig. 34 is a diagram showing a process flow in this embodiment (determination based on the number of detections in a cluster). The process flow of the determination target detection system 900 will be described below along with step numbers (S091 to S097-2) with reference to Fig. 34.

[0283] (S091) A position information receiving unit 910 in the determination object detection system 900 receives position information from the flying object D. An image receiving unit 920 receives a visible light image and an infrared image.

[0284] (S092) The identification unit 930 identifies the photovoltaic power generation panel in the infrared image.

[0285] (S093) The fourth determination unit 910a, based on the arrangement rule of the imaging objects, groups together multiple imaging objects included in a part of the arrangement that follows the rule into one cluster. When multiple imaging objects are arranged two-dimensionally, imaging objects that are aligned in the X-axis direction in the infrared image as shown in Figs. 32 and 33 are grouped together into one cluster. For example, a predetermined position (e.g. Ymin and Ymax) is identified for each imaging object in the infrared image. The fourth determination unit 910a also groups together multiple areas within a predetermined distance based on the identified predetermined position, for example the Y coordinate, into one cluster.

[0286] (S094) The fourth determination unit 910a obtains the number of detection areas included in each cluster (see FIG. 32; Cluster A to Cluster F) and compares it with a stored predetermined number. The fourth determination unit 910a determines whether the number of detection areas satisfies the predetermined number as a result of the comparison.

[0287] (S095) As a result of the comparison in S94, if the number of detection areas in the cluster does not reach a predetermined number (S094; Yes), the fourth judgment unit 910a judges the area belonging to the cluster as a detection error. Furthermore, in this case, the fourth judgment unit 910a excludes the target image from the temperature abnormality judgment target. As another example, the fourth judgment unit 910a may exclude only the detection areas belonging to the judgment cluster from the temperature abnormality judgment target (obstructed panel judgment) and include the image in the temperature abnormality judgment target in S096. As another example, the fourth judgment unit 910a may exclude only the corresponding imaging target area and include the image in the temperature abnormality judgment target in S096. The temperature information acquisition unit 940 acquires temperature information for each imaging target that is not excluded.

[0288] (S096) The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) based on infrared images other than the excluded images (or detection areas other than the imaging target of the detection error) by temperature difference and pattern.

[0289] (S097) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0290] (S095-2) If it is determined in S094 that the number of detection regions in the cluster meets the predetermined number (S094; No), the fourth determination unit 910a determines that there is no detection error. In this case, all images (all determined detection regions) that were the targets for determining whether the detection was successful in S094 are included as targets for determining temperature abnormality. The temperature information acquisition unit 940 acquires temperature information for each imaging target.

[0291] (S096-2) The control unit C detects faulty panels for each panel (each group of a predetermined number of panels) by temperature difference and pattern, based on all infrared images that have been subject to temperature abnormality determination by the fourth determining unit 910a.

[0292] (S097-2) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0293] [Modification Operation] In the case of the above-described modification of this embodiment, in S094 above, the fourth determination section 910a identifies predetermined positions (for example, Ymin and Ymax) in each of the multiple imaging targets, and thereby obtains the interval between each of the targets and other immediately adjacent regions.

[0294] In the case of the above modification, if the comparison result of S94 shows that the upper and lower sides of the cluster to be judged are spaced apart from each other by a distance longer than the predetermined distance (S094; Yes), the fourth judgment unit 910a judges the detection area belonging to the cluster as a detection error. The processing after the error judgment is the same as that of the ninth embodiment.

[0295] In the case of the above modification, in S095-2, if the comparison result in S094 shows that the interval is within a predetermined length on either the upper side or the lower side of the cluster to be judged (S094; No), the fourth judgment unit 910a judges that there is no detection error. In this case, all images (all judged detection areas) that were judged in S094 as to whether the detection was successful or not are included as targets for temperature abnormality judgment.

[0296] [effect] According to this embodiment, the error of abnormality judgment (faulty panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

[0297] Furthermore, according to this embodiment, the detection error of the imaging target is judged based on the presence or absence of deviation from the regularity of the imaging arrangement or the degree of deviation. Also, based on the judgment result, the image judged as an error or the imaging target judged as an error is appropriately excluded from the target of temperature anomaly judgment. Therefore, it is possible to reduce the number of errors in identifying the obstructed panel.

[0298] [Tenth embodiment] A determination object detection system 1000 according to the tenth embodiment will be described with reference to Fig. 35 to Fig. 38. In the following description, descriptions that overlap with the fourth embodiment will be omitted.

[0299] (overview) As in the fourth embodiment, the tenth embodiment also performs a process of identifying the imaging target and other objects in the infrared image, and excluding temperature information of objects other than the imaging target from information used for abnormality judgment (faulty panel judgment). That is, the photovoltaic power generation panels are first identified, and abnormality judgment is performed for each photovoltaic power generation panel or for each group of photovoltaic power generation panels. As a result, the possibility of noise (asphalt pavement, water, plants, power generation equipment (power controller, cable), etc.) being included is reduced. Furthermore, errors in abnormality judgment due to the arrangement of photovoltaic power generation panels are also reduced.

[0300] In order to reduce errors in the detection process of the imaging target such as a photovoltaic power generation panel in the fourth embodiment, the determination target detection system 1000 detects the imaging target and then sequentially identifies the predetermined position of the imaging target according to the arrangement direction of the detected imaging target. Furthermore, the determination target detection system 1000 groups a plurality of imaging targets identified in the infrared image into one cluster according to the regularity of the arrangement of the imaging targets. In this embodiment, the imaging targets included in one group of imaging targets (cluster group) that are shifted from the bottom side position of the cluster are corrected based on the bottom side position of the cluster at the bottom end position. Note that the bottom side position indicates the bottom side from the viewpoint of the figure as shown in FIG. 36, and does not indicate the vertically downward position in the real space. In a photovoltaic power generation facility, this bottom side tends to be the south side.

[0301] That is, as mentioned above, solar panels tend to be placed facing south in solar power generation facilities. In this case, the solar panels are mounted on a rack that is higher on the north side and lower on the south side so that the light receiving surface faces south, that is, with a downward slope to the south (see Figure 37).

[0302] In such a configuration, as shown in Fig. 37, a shaded area and a sunny area are created on the south side of each photovoltaic panel belonging to the bottommost cluster. For example, in a panel located at the bottom end of a block, sunlight is blocked by the northern end (upper end in Fig. 36) of the panel adjacent to the south side, creating a shaded area. However, as shown in Fig. 37, sunlight is not blocked by the northern end of the adjacent panel on the south side, so that area (area irradiated with direct sunlight) becomes a sunny area (see Fig. 36; symbol SP).

[0303] Since the temperature in a sunny area is relatively higher than that in a shaded area, this can cause detection noise when determining whether a panel is obstructed. This embodiment makes it possible to reduce this detection noise by using the following configuration.

[0304] (Object detection system 1000) 35, the determination object detection system 1000 of the tenth embodiment includes a position information receiving unit 1010, an image receiving unit 1020, an identifying unit 1030, a temperature information acquiring unit 1040, a control unit C, a storage unit S, a fifth determining unit 1010a, and a correcting unit 1010b. The identifying unit 1030 detects a solar power generation panel from an infrared image in the same manner as in the fourth embodiment.

[0305] (Fifth Determination Unit 1010a) As in the fourth embodiment, an imaging target, for example, a solar power generation panel, is identified from an infrared image. As in the sixth embodiment, the fifth determining unit 1010a can group imaging targets aligned in the X-axis direction in the infrared image into one cluster (see FIG. 36: Cluster_num1 to Cluster_num4).

[0306] Furthermore, the fifth determination unit 1010a obtains the average value of Ymax for each imaging object in the cluster, and sets this as the Ymax of the cluster. Note that the Ymax of a cluster may be the median value of the Ymax of each imaging object belonging to the cluster.

[0307] Furthermore, the fifth determination unit 1010a identifies the bottommost cluster (see Cluster_num4 in FIG. 36) in one block of a two-dimensional array of multiple imaging targets. The method of identification is, for example, as follows. The control unit C causes the infrared image after cluster identification to be displayed on a display means (including a display means outside the system) not shown (see FIG. 36). The control unit C also executes control to display an identification display for each cluster for each array of imaging targets in the infrared image. In the example of FIG. 36, this identification display is a serial number displayed on the cluster. It may also be another identification display that allows the user to specify each cluster.

[0308] In this state, the user can visually confirm the identification markings for each cluster via the display means, and can further specify them using an input means (not shown). In this configuration, when the user specifies any one of the identification marks (tier number, cluster number, etc.) among the group of identification marks on the screen using the input means, the control unit C accepts the specification operation and identifies the cluster in the infrared image.

[0309] It should be noted that other methods may be used to identify the bottommost cluster in a two-dimensional array of multiple imaging targets. For example, it is possible to determine that a cluster whose distance to the nearest other region on the upper side (Ymin) of the cluster is within a predetermined distance and whose distance to the nearest other region on the lower side (Ymax) of the cluster exceeds a predetermined distance is the bottommost cluster. In this configuration, a cluster with no other region on the lower side is also determined to be the bottommost cluster.

[0310] After identifying the bottom cluster, the fifth determination unit 1010a compares the Ymax coordinates of each detection area belonging to the cluster with the Ymax coordinates of the cluster. For example, the fifth determination unit 1010a obtains the difference (distance) between the Ymax coordinates of each detection area and the Ymax coordinates of the cluster. Furthermore, the fifth determination unit 1010a determines whether the difference is within a predetermined range. As an example, the fifth determination unit 1010a compares this difference with a determination value. If the difference exceeds the predetermined range, the fifth determination unit 1010a identifies the detection area.

[0311] (Correction unit 1010b) The correction unit 1010b makes the bottom side of the detection area specified by the fifth determination unit 1010a correspond to the bottom side of the cluster. For example, the correction unit 1010b corrects the Y coordinate of Ymax of the detection area so that it corresponds to the Y coordinate of Ymax of the cluster based on the difference obtained above.

[0312] [Operation] Fig. 38 is a diagram showing a process flow in this embodiment. Below, with reference to Fig. 38, the process flow of the determination object detection system 1000 will be explained along with step numbers (S101 to S108-2).

[0313] (S101) A position information receiving unit 1010 in the determination object detection system 1000 receives position information from the flying object D. An image receiving unit 1020 receives a visible light image and an infrared image.

[0314] (S102) The identification unit 1030 identifies the photovoltaic power generation panel in the infrared image.

[0315] (S103) The fifth determination unit 1010a, based on the arrangement rule of the imaging objects, groups together multiple imaging objects included in a part of the arrangement that follows the rule into one cluster. When multiple imaging objects are arranged two-dimensionally, imaging objects that are aligned in the X-axis direction in the infrared image as shown in FIG. 36 are grouped together into one cluster. For example, a predetermined position (e.g., Ymin and Ymax) is identified for each imaging object in the infrared image. The fifth determination unit 1010a also groups together multiple areas within a predetermined distance based on the identified predetermined position, for example, the Y coordinate, into one cluster.

[0316] (S104) The fifth determining unit 1010a identifies the bottommost cluster from among the clusters (see FIG. 36: Cluster_num1 to Cluster_num4) included in one block of the two-dimensional array of the imaging target.

[0317] (S105) The fifth determination unit 1010a compares the Ymax coordinate of each detection area belonging to the cluster with the Ymax coordinate of the cluster, and further determines whether the difference is within a predetermined range.

[0318] (S106) If the comparison in S105 shows that the bottom side of any detection area in the bottommost cluster is separated from the bottom side of the cluster (S105; Yes), correction unit 1010b corrects the bottom side of that detection area to correspond to the bottom side of the cluster.

[0319] (S107) The temperature information acquisition unit 1040 acquires temperature information including the detection area in the bottommost cluster after the lower side of the detection area has been corrected, and detects faulty panels for each panel (each group of a predetermined number of panels) based on the temperature difference and pattern.

[0320] (S108) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0321] (S106-2) In S0105, if the difference between the bottom side of each of the detection areas in the cluster and the bottom side of the cluster is within a predetermined range (S104; No), temperature information is acquired for each panel (for each group of a predetermined number of panels) without performing correction processing by correction unit 1010b.

[0322] (S107-2) The temperature information acquisition unit 1040 detects a faulty panel based on the temperature difference and pattern for each panel (each group of a predetermined number of panels).

[0323] (S108-2) The control unit C indicates to the user any obstructed panels detected from the infrared image.

[0324] [effect] According to this embodiment, the error of abnormality judgment (faulty panel judgment) of the imaging target of the infrared image is reduced. One of the reasons is that the imaging target in the infrared image is distinguished from other objects, and temperature information of the imaging target is excluded from the information used for abnormality judgment. For example, in a solar power generation facility, between one solar power generation panel group and another solar power generation panel group, there are power generation equipment (power controller and cable), asphalt pavement, water, plants, etc., which present temperature information completely different from that of the solar power generation panel. If temperature abnormality is detected in the entire infrared image, there is a possibility that such noise will be included, which may hinder the abnormality judgment. According to this embodiment, the solar power generation panel is first identified, and the abnormality judgment is performed for each solar power generation panel or each solar power generation panel group. As a result, it is possible to reduce the possibility of noise being included. Furthermore, depending on the arrangement position of the solar power generation panel, for example, the average value of the temperature information may differ between the upper right corner and the lower left corner of the solar power generation panel array due to a shadow, and in this respect, it is also possible to reduce the error of abnormality judgment.

[0325] Furthermore, according to this embodiment, the deviation of the bottom edge position of each detection area from the bottom edge position of the bottommost cluster of each block including the imaging target group is judged, and if there is a detection area that deviates beyond the allowable range, the bottom edge position of that area is corrected to match the cluster. In this embodiment, the obstruction panel is judged after the correction. Therefore, it is possible to reduce the number of errors in identifying obstruction panels.

[0326] [Variation 1] Next, other examples of processing that the judgment object detection system can take after the judgment of a detection error in each of the above embodiments will be described. In the above embodiments (5th to 9th), a detection error of the imaging object is judged, and if a detection error occurs, the detection area or an image including the detection area is excluded from the judgment of an obstructed panel. However, other configurations are also possible. For example, the judgment object detection system can be configured to notify the detection error in any manner, such as by outputting an error message to a display means (including a display means outside the system) not shown, or by outputting an alarm sound to an audio output unit not shown.

[0327] Furthermore, the control unit C of the judgment target detection system presents the user with an image including the area related to the detection error after or without the above-mentioned notification to the user. One example of this is to display the image on any display means (including display means outside the system) not shown. Furthermore, a message is displayed to prompt the user to select whether to manually complement the part in the image that does not match the sequence information.

[0328] As shown in Fig. 3, 39 and 40, etc., the imaging target identified by the identification unit can be made recognizable to the user by the display means. In other words, the control unit C can make the area identified by the identification unit as the imaging target in the infrared image visible to the user in the infrared image. For example, as shown in Fig. 3, 39 and 40, etc., it is possible to draw the outer edge of the imaging target area so that it can be distinguished from others. In addition, it is possible to draw the entire area of ​​each imaging target in any manner so that it can be distinguished from others (shaded, inverted, flashing).

[0329] Furthermore, the control unit C displays the area in the infrared image where a detection error occurred so that an operation can be performed to slide the area on the displayed image. For example, the control unit C highlights the outer edge of the area where an error was determined. Furthermore, when the user uses any input means such as a pointing device to superimpose a pointer or the like on the image on the displayed area, the control unit C transitions to a state in which it can accept an operation to slide the area on the image. Furthermore, when the user moves the area to any position on the image in this state via the input means, the display showing the area moves to the specified position on the image.

[0330] Furthermore, when the user executes an operation to complete the movement of the area via the input means, the control unit C rewrites the coordinate values ​​of the area before the movement to the coordinate values ​​after the movement. This embodiment has a configuration that allows the operation of correcting the area involved in the detection error by the above-mentioned manual operation. This embodiment also makes it possible to reduce mistakes in identifying the obstructed panel.

[0331] [Variation 2] In the above embodiment, the configuration is such that visible light images and infrared images are acquired in the aerial photography of the flying object D. However, depending on the imaging subject, the configuration may be such that only infrared images are acquired.

[0332] [Variation 3] In the above embodiment, the configuration is such that visible light images and infrared images are acquired in the aerial photography of the flying object D. However, depending on the imaging target, instead of an aerial image, a configuration may be adopted in which an image is captured by an imaging means fixed above the imaging target (for example, fixed to the ceiling or to the top of a structure (pole, etc.) protruding upward) and the condition of the image is determined. For example, a configuration may be adopted in which the condition is determined from temperature information of imaging targets (stored items, manufactured items, etc.) arranged in large numbers in a warehouse or factory.

[0333] The determination object detection system understood from the above modification 3 has the following features. [Feature 1] an image receiving unit that receives an infrared image acquired by photographing the imaging target from above the imaging target; an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the infrared image; a temperature information acquisition unit that acquires temperature information based on infrared information for each of the identified imaging targets or each of the imaging target groups; a determination unit that performs an abnormality determination for each of the identified imaging targets or each of the imaging target groups based on the temperature information obtained by the temperature information acquisition unit; A judgment object detection system comprising:

[0334] [Variation 4] In the above embodiment, the state of the imaging target is determined from temperature information based on an image taken from above the imaging target. However, depending on the imaging target, the state may be determined using an image taken from a horizontal direction at a predetermined distance away from the imaging target, rather than from above the imaging target. For example, the imaging means is fixed to a position (e.g., a central position) of a structure (pole, etc.) protruding upward at a position at a predetermined distance away from the imaging targets (window glass of a building) that are two-dimensionally arranged in the vertical direction. In this state, the imaging means may be configured to image the imaging target at the predetermined distance away in the horizontal direction. Alternatively, the state may be determined based on the acquired image.

[0335] [Alignment position detection system] The technical ideas in the fifth to seventh embodiments are not limited to a determination object detection system that uses temperature information of an image pickup object, but can also be used as an alignment position detection system, which will be described below.

[0336] [Diversion of the fifth embodiment] The fifth embodiment is for correcting the tilt of an image. This is not limited to temperature detection, but can be used for detecting the arrangement positions of a large number of imaging targets that are assumed to be aligned or arranged regularly.

[0337] As in the above embodiment, it is possible to detect individual imaging targets from an image of a group of imaging targets arranged in a plurality of arrays. In this case, imaging targets that should be arranged regularly, such as in a two-dimensional array, may deviate from the regularity or may be missing. For example, in the case of packaging the same or identical products in a two-dimensional array, the products may be missing or misaligned. Or, when a car should be parked at a predetermined interval according to a predetermined arrangement rule, the parking position may be misaligned or missing.

[0338] By detecting each of the imaging targets using an image captured of a group of multiple imaging targets arranged in a matrix, it is possible to inspect the position of such an array. In such a configuration, if the image is tilted, it may interfere with the detection of the imaging target, which may result in an error in the inspection for misalignment or omission. In response to this, in this alignment position detection system, the image tilt is first detected as in the fifth embodiment. This image may be an infrared image or a visible light image. After the tilt is detected, tilt correction may be performed as in the fifth embodiment. Alternatively, a process may be performed to exclude the image from the inspection target for misalignment or omission as an imaging error as in the fifth embodiment.

[0339] In the alignment position detection system, like the identification unit of the fourth embodiment, the imaging targets are identified, and the arrangement of the imaging targets is further identified (e.g., 4 rows vertically and 7 columns horizontally, etc.). This may be done using an inference model created by deep learning using so-called artificial intelligence (AI), or other methods such as pattern matching may be used. Next, a predetermined control unit etc. compares the arrangement information set by the user with the arrangement of the imaging targets identified from the image, and performs an inspection process for misalignment and omission of arrangement.

[0340] The alignment position detection system grasped from the diversion of the fifth embodiment can have the following features. [Feature 2] an image receiving unit that receives an image of an imaging target; an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the image; a correction unit that calculates a tilt of an imaging direction with respect to the imaging target and performs tilt correction based on the calculated tilt; a seventh determination unit that determines whether or not there is an imaging object that deviates from a default arrangement rule or an imaging object that has been removed from a default arrangement rule in the corrected image; An alignment position detection system comprising:

[0341] [Diversion of the sixth embodiment] The sixth embodiment aims to reduce detection errors of the imaging target by utilizing the fact that the size of the imaging target (the size of the imaging target as viewed from at least one direction) is uniform. This is not limited to temperature detection, but can be used to detect the arrangement positions of many imaging targets that are simply assumed to be aligned or arranged regularly.

[0342] Like the alignment position detection system repurposed from the fifth embodiment, by detecting individual imaging targets from an image captured of a group of multiple aligned imaging targets, it is possible to check for misalignment or missing alignment of the aligned imaging targets.

[0343] In such a configuration, if an image target that should be uniform in size is detected as large or small due to a detection error of the image target, an error may occur in the inspection. In response to this, in the alignment position detection system that is a conversion of the sixth embodiment, the size of the area of ​​each detected image target is first obtained as in the sixth embodiment. The image may be an infrared image or a visible light image.

[0344] Next, similarly to the sixth embodiment, the size of the imaging target or a numerical value calculated based on the size is compared with the judgment value to judge whether or not there has been a detection error.

[0345] After the detection error is determined, the user may be notified of the detection error, the captured object related to the detection error may be highlighted in the image, or the image may be excluded from the inspection target for misalignment or omission of position as the detection error.

[0346] In the alignment position detection system, the arrangement of the imaging targets is identified by a specification unit, similarly to the alignment position detection system to which the fifth embodiment is adapted.

[0347] The alignment position detection system grasped from the diversion of the sixth embodiment can have the following features. [Feature 3] an image receiving unit that receives an image of an imaging target; an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the image; an eighth determination unit that calculates the size of the imaging target or the group of imaging targets identified by the determination unit, compares the variation in the calculated value with a determination value indicating an acceptable range of the variation, and determines that there is an identification error if the variation is in a predetermined relationship with the determination value. [Feature 4] an image receiving unit that receives an image of an imaging target; an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the image; a ninth determination unit that determines whether or not there is an imaging object that deviates from a default arrangement rule or an imaging object that has been removed from a default arrangement rule in the image; An alignment position detection system comprising:

[0348] [Diversion of the seventh embodiment] The seventh embodiment utilizes the arrangement rule of the imaging targets to reduce detection errors of the imaging targets. That is, after the imaging targets are detected, it is determined whether there is a deviation in the alignment position of the imaging targets. For example, it is determined whether there is a false detection in the detection of the boundary part of the outer edge of the imaging targets or the area of ​​each imaging target. This is not limited to temperature detection, but can be used to detect the arrangement positions of many imaging targets that are simply assumed to be aligned or arranged regularly.

[0349] Like the alignment position detection system repurposed from the fifth embodiment, by detecting individual imaging targets from an image captured of a group of multiple aligned imaging targets, it is possible to check for misalignment or missing alignment of the aligned imaging targets.

[0350] In such a configuration, in the alignment position detection system that is a conversion of the fourth embodiment, similarly to the seventh embodiment, the regularity of the predetermined arrangement specific to the imaging target is utilized to determine whether there is any deviation from the regularity. For example, the predetermined positions in each of the multiple imaging targets identified from the image are further identified, and the deviation amount of the identified positions from the arrangement regularity is calculated. If the deviation amount is large, a detection error is determined. This aims to reduce detection errors of the imaging targets.

[0351] That is, as in the seventh embodiment, a predetermined position in the area of ​​each detected imaging target is further specified. For example, the predetermined position is "Ymin" or "Ymin n The image may be an infrared image or a visible light image.

[0352] Next, as in the seventh embodiment, the variation in the predetermined position of each of the identified imaging targets is obtained. The variation is judged to be within an acceptable range or an unacceptable range by a judgment value. As in the fourth embodiment, if the variation is within an unacceptable range, it is judged to be a detection error.

[0353] After the detection error is determined, the user may be notified of the detection error, the captured object related to the detection error may be highlighted in the image, or the image may be excluded from the inspection target for misalignment or omission of position as the detection error.

[0354] In this alignment position detection system, the arrangement of the imaging targets is identified by a specification unit, similarly to the alignment position detection system to which the fifth embodiment is adapted.

[0355] The alignment position detection system grasped from the diversion of the seventh embodiment can have the following features. [Feature 5] an image receiving unit that receives an image of an imaging target; an identification unit that identifies an imaging target or a group of a predetermined number of imaging targets from the image; a tenth determination unit that determines an amount of deviation from an arrangement of any of the imaging targets or the imaging target group based on the positional relationship of the imaging targets or the imaging target group identified by the determination unit, compares the amount of deviation with a determination value that indicates an allowable range of the amount of deviation, and determines an identification error when the amount of deviation is in a predetermined relationship with the determination value; An alignment position detection system comprising:

[0356] The above three alignment position detection systems can be combined as appropriate.

[0357] Although the embodiments of the present invention have been described, the above embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the invention and its equivalents described in the claims. [Explanation of symbols]

[0358] 100, 200, 300, Status Judgment System 400, 500, 600, 700, 800, 900, 1000 Object detection system 110, 210, 310, 410, 510, 610, 710 Location information receiver 120, 220, 320, 420, 520, 620, 720, 820, 920, 1020 Image receiving unit 130, 230, 330, 430, 530, 630, 730, 830, 930, 1030 Specific part 140, 240, 340, 440, 540, 640, 740, 840, 940, 1040 Temperature information acquisition section 150, 150A, 250, 350 High temperature information acquisition section 160, 260, 360 Medium temperature range acquisition part 170, 270, 370 Reference value acquisition section 180, 280, 380 Judgment target value acquisition section 190, 290, 390 Judgment section 391 Suspect Identification Department 410a Setting section 510a, 1010b correction section 610a, 710a, 810a, 910a, 1010a 1st to 5th judgment units D. Flying object D1 Photography Department D2 Position acquisition part D3 Gyroscope D4 Gimbal D5 Geomagnetic Sensor

Claims

1. an image receiving unit for receiving an infrared image from the flying object; An identification unit that identifies a preset region including an imaging target from an infrared image; a temperature information acquisition unit that obtains a distribution of temperature information based on infrared information for each specified set region; a high temperature information acquiring unit that calculates a first average value of temperature information included in a first set range including the maximum temperature in the distribution; a middle temperature range acquisition unit that calculates a second average value of temperature information included in a second set range that is a temperature below the first set range and does not overlap with the first set range, from among the distribution; a reference value acquiring unit that acquires a reference value that is a difference between the first average value and the second average value; a determination object value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas and obtains a determination object value based on at least one of a value indicating a variation in the reference value of each set area within the group and an average value; A determination unit that identifies an imaging target that may be impaired based on the determination target value. Status determination system.

2. By varying the setting range for the first average value, the type of fault can be determined.

2. The state determination system according to claim 1 .

3. an image receiving unit for receiving an infrared image from the flying object; An identification unit that identifies a preset region including an imaging target from an infrared image; a temperature information acquisition unit that obtains a distribution of temperature information based on infrared information for each specified set region; a middle temperature range acquisition unit that calculates a second average value of temperature information included in a second set range that is less than the maximum temperature in the distribution; a reference value acquisition unit that obtains a reference value that is a difference between the maximum temperature and the second average value; a determination object value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas and obtains a determination object value based on at least one of a value indicating a variation in the reference value of each set area within the group and an average value; A determination unit that determines an abnormality in the imaging target based on the determination target value. Status determination system.

4. the determination target value acquisition unit obtains a standard deviation of the reference value as a variation in the reference value, and further obtains a deviation value for each set region; The determination unit performs abnormality determination by comparing the deviation value with a first threshold value.

4. The state determination system according to claim 1, wherein the state determination system is a state determination system for determining a state of a vehicle.

5. When the determination unit determines that the determination target value is greater than the first threshold value, the determination unit identifies a suspected fault area that caused the determination. The reference value is calculated again so as not to include the suspected fault area, and the imaging target that may be at fault is identified based on the calculated reference value.

5. The state determination system according to claim 4.

6. The determination unit repeats the process of recalculating the reference value and identifying the suspected fault area until the suspected fault area is no longer identified or until the area or ratio of the suspected fault area falls below a predetermined value.

6. The state determination system according to claim 5.

7. an image receiving unit for receiving an infrared image from the flying object; An identification unit that identifies a preset region including an imaging target from an infrared image; a temperature information acquisition unit that obtains a distribution of temperature information based on infrared information for each specified set region; a high temperature information acquiring unit that calculates a first average value of temperature information included in a first set range including the maximum temperature in the distribution; a middle temperature range acquisition unit that calculates a second average value of temperature information included in a second set range that is a temperature below the first set range and does not overlap with the first set range, from among the distribution; a reference value acquiring unit that acquires a reference value that is a difference between the first average value and the second average value; a determination object value acquisition unit that groups a plurality of set areas including the set area and other adjacent set areas and obtains a determination object value based on at least one of a value indicating a variation in the reference value of each set area within the group and an average value; a suspect target identification unit that identifies a suspect target that is an imaging target that may be impaired based on a judgment target value and a first threshold value; a determination unit that classifies the disorder of the imaging target based on a pattern of high temperature parts in the identified suspect target, and in the case of a specific classification, identifies the suspect target again based on a determination target value of the suspect target and a second threshold value; Equipped Status determination system.

8. The classification of the failure of the judgment unit includes at least two classifications of cluster failure, crack, dirt, and cell failure.

8. The state determination system according to claim 7.

9. A second threshold is used when the determination unit determines that the failure classification of the suspected target is other than a cluster failure, The second threshold is set so that the number of suspected targets included in the set area is reduced compared to the first threshold.

9. The state determination system according to claim 7 or 8.

10. The determination unit infers a classification of a suspected fault based on deep learning technology.

10. The state determination system according to claim 7, wherein the state determination system is a state determination system for determining a state of a vehicle.

11. receiving an infrared image from the air vehicle; Identifying a preset region including an imaging target from the infrared image; determining a distribution of temperature information based on infrared information for each specified set region; determining a first average value of temperature information included in a first set range including the maximum temperature in the distribution; determining a second average value of temperature information included in a second set range that is less than the first set range and does not overlap with the first set range; determining a reference value which is a difference between the first average value and the second average value; a step of grouping a plurality of set areas including the set area and other adjacent set areas, and determining a judgment target value based on at least one of a value indicating a variation in the reference value of each set area within the group and an average value; and identifying an imaging target that may be a problem based on the judgment target value. Status determination method.

12. receiving an infrared image from the air vehicle; A process section for identifying a preset region including an imaging target from an infrared image; determining a distribution of temperature information based on infrared information for each specified set region; determining a second average value of the temperature information included in a second set range of the distribution that is less than the maximum temperature; determining a reference value which is a difference between the maximum temperature and the second average value; a step of grouping a plurality of set areas including the set area and other adjacent set areas, and determining a judgment target value based on at least one of a value indicating the variation in the reference value of each set area within the group and an average value; and determining whether the imaging target has an abnormality based on the determination target value. Status determination method.

13. receiving an infrared image from the air vehicle; Identifying a preset region including an imaging target from the infrared image; determining a distribution of temperature information based on infrared information for each specified set region; determining a first average value of temperature information included in a first set range including the maximum temperature in the distribution; determining a second average value of temperature information included in a second set range that is less than the first set range and does not overlap with the first set range; determining a reference value which is a difference between the first average value and the second average value; a step of grouping a plurality of set areas including the set area and other adjacent set areas, and determining a judgment target value based on at least one of a value indicating the variation in the reference value of each set area within the group and an average value; A step of identifying a suspected object that is an imaging object that may be impaired based on a judgment object value and a first threshold value; A step of classifying the disorder of the imaging target based on the pattern of the high temperature part in the identified suspect target, and in the case of a specific classification, re-identifying the suspect target based on the judgment target value of the suspect target and a second threshold value. Status determination method.

Images