Recognition System

Abstract

To provide a recognition system capable of enhancing recognizability of users toward subjects to be recognized.SOLUTION: A recognition device 30 is a recognition system for recognizing a subject to be recognized from a photographed image in which a prescribed region of a structure is captured, comprising: a recognition section 35a for recognizing the subject to be recognized based on a color, a position, a shape, and a size of a region included in the photographed image, or a marking included in the region; a calculation section 35b for, based on a size of a referential subject included in the photographed image, calculating an area of the subject to be recognized or a ratio of the area to the photographed image; and a provision section 35c for providing result information indicating a recognition result of the recognition section 35a and a calculation result of the calculation section 35b.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a recognition system. [Background technology]

[0002] One of the techniques for recording construction work at a construction site has been disclosed in the past, in which images taken at the construction site by a camera are supplemented with on-site information about the construction site and compiled onto a single printed sheet (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-307761 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, there has been an increasing need to record a specific part (for example, a tile base material) included in a captured image so that it can be easily recognized. However, in the above-mentioned conventional technology, on-site information is simply added to the captured image and compiled on a single printed sheet, making it difficult for the user to recognize the specific part. Therefore, there is room for improvement in terms of improving the recognizability of recognition targets such as the specific part.

[0005] The present invention has been made in view of the above, and has an object to provide a recognition system that enables a user to improve the recognizability of a recognition target. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the recognition system according to claim 1 is a recognition system for recognizing a recognition target from a captured image of a predetermined area of ​​a structure, the recognition system comprising: recognition means for recognizing the recognition target based on a color, position, shape, or size of an area included in the captured image, or a marking included in the area; calculation means for calculating an area of ​​the recognition target or a ratio of the area to the captured image based on a size of a reference object included in the captured image; and provision means for providing result information including information indicating the recognition result of the recognition means and information indicating the calculation result of the calculation means, the provision means causing a display means to display information indicating the recognition result of the recognition means, which is included in the result information, in an identifiable manner, and the display means causing the information indicating the recognition result of the recognition means, which is included in the result information, to be displayed graphically, and the information indicating the calculation result of the calculation means, which is included in the result information, to be displayed in text near corresponding information indicating the recognition result of the recognition means, When the recognition means recognizes a plurality of recognition targets of different types, information indicating the recognition results of the recognition means for the plurality of recognition targets is displayed in different formats.

[0007] The recognition system according to claim 2 is the recognition system according to claim 1, wherein when the captured image contains a plurality of recognition targets of different types, the recognition means identifies and recognizes each of the plurality of recognition targets based on the color, position, or shape of an area included in the captured image, or a marking included in the area.

[0008] The recognition system according to claim 3 is the recognition system according to claim 1 or 2, The recognition target includes a body of the structure coated with paint or a fluorescent substance, or a member located in the vicinity of the body. [Effects of the Invention]

[0010] According to the recognition system of claim 1, the system includes a recognition means for recognizing a recognition target based on the color, position, shape, or size of an area included in a captured image or a marking included in the area, a calculation means for calculating the area of ​​the recognition target or the ratio of the area to the captured image based on the size of a reference object included in the captured image, and a provision means for providing result information indicating the recognition result of the recognition means and the calculation result of the calculation means, so that the result information can be provided. Thus, a user who is provided with the result information can easily recognize the recognition target and can accurately recognize the area of ​​the recognition target, thereby improving the user's ability to recognize the recognition target. In addition, since the providing means displays the recognition result of the recognition means included in the result information in an identifiable manner on the display means, the user to whom the result information is provided can visually recognize the recognition result, which makes it easier to improve the user's recognizability of the recognition result.

[0011] According to the recognition system of claim 2, when a captured image contains multiple recognition targets of different types, the recognition means identifies and recognizes each of the multiple recognition targets based on the color, position, or shape of an area included in the captured image, or a marking included in the area, so that result information regarding the multiple recognition targets can be provided, allowing a user who is provided with the result information to easily recognize the multiple recognition targets and accurately recognize the areas of the multiple recognition targets.

[0012] According to the recognition system of claim 3, Since the recognition target includes the structure's main body or a component located near the main body to which paint or fluorescent substance is applied, the recognition means can more easily recognize the recognition target based on the paint or fluorescent substance contained in the recognition target, thereby improving the recognition accuracy of the recognition target by the recognition means. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram conceptually illustrating a management system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a block diagram showing the electrical configuration of the management system. [Figure 3] 1 is a flowchart of a recognition process according to an embodiment; [Figure 4] FIG. 10 is a diagram illustrating an example of display of result information. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, with reference to the accompanying drawings, a detailed description will be given of an embodiment of a recognition system according to the present invention. First, [I] the basic concept of the embodiment will be described, then [II] specific contents of the embodiment will be described, and finally, [III] modifications of the embodiment will be described. However, the present invention is not limited to the embodiment.

[0016] [I] Basic Concept of the Embodiment First, the basic concept of the embodiment will be described. The embodiment generally relates to a recognition system for recognizing a recognition target from a captured image of a predetermined area of ​​a structure.

[0017] Here, the specific structure and type of "structure" are arbitrary, but the concept includes, for example, architectural structures such as office buildings, commercial facilities, public facilities (for example, schools, hospitals, etc.), and apartment complexes such as condominiums, as well as civil engineering structures such as bridges, levees, and dams. In the embodiment, however, it will be described as a public facility under construction with multiple floors.

[0018] Furthermore, "recognition target" refers to an object recognized by a recognition system, and is a concept that includes, for example, the skeleton of a structure (e.g., walls, floors, pillars, beams, etc.), permanent or temporary components installed around the skeleton (e.g., tiles, tile base material or mortar, scaffolding, etc.), and phenomena that occur in the skeleton (e.g., cracks, junk, misalignments, etc.).

[0019] [II] Specific details of the embodiment Next, specific details of the embodiment will be described.

[0020] (composition) First, the configuration of a management system to which a recognition system according to an embodiment of the present invention is applied will be described. FIG. 1 is a conceptual diagram of a management system according to an embodiment of the present invention. FIG. 2 is a block diagram showing the electrical configuration of the management system. In the following description, the X direction in FIG. 1 will be referred to as the left-right direction of a body 3 (described later) (the -X direction is the left direction of the body 3 (described later), and the +X direction is the right direction of the body 3 (described later)), the Y direction in FIG. 1 will be referred to as the up-down direction of the body 3 (described later) (the +Y direction is the up direction of the body 3 (described later), and the -Y direction is the down direction of the body 3 (described later)), and the direction perpendicular to the X direction and the Y direction in FIG. 1 will be referred to as the front-to-back direction (the direction leading to the front side of the paper surface of FIG. 1 is the front direction of the body 3 (described later), and the direction leading to the back side of the paper surface of FIG. 1 is the back direction of the body 3 (described later).

[0021] The management system 1 is a system for managing the construction status of a recognition target RO (specifically, a tile base material) in a structure under construction (specifically, an office building under construction), and as shown in FIG. 1, it includes at least one or more imaging devices 10, a collection device 20, and a recognition device 30, and each of the at least one or more imaging devices 10 and the recognition device 30 is connected to the collection device 20 so that they can communicate with each other via a network 2. Note that the at least one or more imaging devices 10 can be configured similarly to each other, so the following description will focus on the configuration of one imaging device 10, and descriptions of the configurations of the other imaging devices 10 will be omitted.

[0022] Here, in the embodiment, the "recognition target RO" refers to a base material provided between the structure's skeleton 3 (exterior wall) and tiles (not shown), and is applied to at least a part of the exterior surface of the exterior wall as shown in Fig. 1. Also, in the embodiment, the "construction status of the recognition target RO" is explained as the application status over time regarding the installation position, size, shape, or installation order of the base material.

[0023] (Configuration - Imaging device) Next, the configuration of the imaging device 10 will be described.

[0024] The imaging device 10 is a device that captures an image of a predetermined area of ​​a structure (specifically, an area including the recognition target RO), and as shown in Fig. 2, it is equipped with an imaging-side operation unit 11, an imaging-side communication unit 12, an imaging unit 13, an imaging-side output unit 14, an imaging-side power supply unit 15, an imaging-side control unit 16, and an imaging-side storage unit 17. In the embodiment, the imaging device 10 can be configured using a known mobile terminal or the like carried by a worker at a construction site, and therefore a detailed description thereof will be omitted.

[0025] (Configuration - Imaging device - Imaging side operation unit) The imaging side operation unit 11 is an imaging side operation means that accepts operation inputs to the imaging device 10, and is configured using, for example, known operation means (e.g., a switch, a touchpad, etc.) (the same applies to the recognition side operation unit 31 of the recognition device 30 described later).

[0026] (Configuration - Imaging device - Imaging communication unit) The imaging side communication unit 12 is an imaging side communication means for communicating between the imaging device 10 and the collection device 20, and is configured using a known communication means for communicating using a wireless communication network or a wired communication network (the same applies to the collection side communication unit 21 of the collection device 20 described later and the recognition side communication unit 32 of the recognition device 30 described later).

[0027] (Configuration - Imaging device - Imaging section) The imaging unit 13 is an imaging means for capturing an image of a predetermined area of ​​a structure, and is configured using, for example, a known imaging means (specifically, a camera for capturing two-dimensional images).

[0028] (Configuration - Imaging device - Imaging output section) The imaging side output unit 14 is an imaging side output means that outputs various information based on the control of the imaging side control unit 16, and is configured using, for example, a known display means (for example, a display) or audio output means (for example, a speaker) (the same applies to the recognition side output unit 33 of the recognition device 30 described later).

[0029] (Configuration - Imaging device - Imaging power supply unit) The imaging side power supply unit 15 is an imaging side power supply means that supplies the power stored in the imaging side power supply unit 15 to each part of the imaging device 10, and is configured using, for example, a known power supply means (for example, a battery for a mobile terminal, etc.) (the same applies to the recognition side power supply unit 34 of the recognition device 30 described later).

[0030] (Configuration - Imaging device - Imaging side control unit) The imaging side control unit 16 is an imaging side control means that controls the imaging device 10, and specifically, is a computer that is configured with a CPU, various programs that are interpreted and executed on the CPU (including basic control programs such as an OS and application programs that are started on the OS and realize specific functions), and an internal memory such as a RAM for storing programs and various data (the same applies to the collection side control unit 22 of the collection device 20 described later and the recognition side control unit 35 of the recognition device 30 described later).

[0031] (Configuration - Imaging device - Imaging side storage unit) The imaging-side storage unit 17 is an imaging-side storage means for recording programs and various data necessary for the operation of the imaging device 10, and is configured using, for example, a hard disk (not shown) as an external recording device. However, instead of or together with the hard disk, any other recording medium can be used, including a magnetic recording medium such as a magnetic disk, an optical recording medium such as a DVD or a Blu-ray disk, or an electrical recording medium such as a Flash ROM, a USB memory, or an SD card (the same applies to the collection-side storage unit 23 of the collection device 20 and the recognition-side storage unit 36 ​​of the recognition device 30, which will be described later).

[0032] (Configuration-Collection Device) Next, the configuration of the collection device 20 will be described.

[0033] The collection device 20 is a device that collects captured images captured by the imaging device 10 and transmits the collected captured images to the recognition device 30, and as shown in Fig. 2, includes a collection-side communication unit 21, a collection-side control unit 22, and a collection-side storage unit 23. In the embodiment, the collection device 20 can be configured by a known server device such as a cloud server on the Internet (or a stationary server provided in a management facility), and therefore a detailed description thereof will be omitted.

[0034] (Configuration - Collection device - Collection side communication unit) The collection-side communication unit 21 is a collection-side communication means for communicating between the collection device 20 and each of the image capture device 10 and the recognition device 30.

[0035] (Configuration - Collection device - Collection side control unit) The collection-side control unit 22 is a collection-side control means that controls the collection device 20 .

[0036] (Configuration - Collection device - Collection side memory unit) The collection side storage unit 23 is a collection side storage means for recording programs and various data required for the operation of the collection device 20 .

[0037] (Configuration-Recognizer) Next, the configuration of the recognition device 30 will be described.

[0038] The recognition device 30 is a recognition system for recognizing a recognition target RO from a captured image, and as shown in Fig. 2, includes a recognition-side operation unit 31, a recognition-side communication unit 32, a recognition-side output unit 33, a recognition-side power supply unit 34, a recognition-side control unit 35, and a recognition-side memory unit 36. In the embodiment, the recognition device 30 can be configured by a known terminal device (for example, a stationary personal computer or a mobile terminal (smartphone or tablet terminal)) installed in an office located away from the construction site, and therefore a detailed description thereof will be omitted.

[0039] (Configuration - Recognition device - Recognition side operation section) The recognition-side operation unit 31 is a recognition-side operation means that receives operation input to the recognition device 30.

[0040] (Configuration - Recognition device - Recognition side communication unit) The recognition-side communication unit 32 is a recognition-side communication means for communicating between the recognition device 30 and the collection device 20 .

[0041] (Configuration - Recognition device - Recognition side output part) The recognition-side output unit 33 is a recognition-side output means that outputs various information under the control of the recognition-side control unit 35 .

[0042] (Configuration - Recognition device - Recognition side power supply section) The recognition-side power supply unit 34 is a recognition-side power supply means that supplies the power stored in the recognition-side power supply unit 34 to each unit of the recognition device 30.

[0043] (Configuration - Recognition device - Recognition side control unit) The recognition-side control unit 35 is a recognition-side control means that controls the recognition device 30, and as shown in FIG. 2, conceptually includes a recognition unit 35a, a calculation unit 35b, and a provision unit 35c.

[0044] The recognition unit 35a is a recognition means that recognizes the recognition target RO based on the color, position, shape, size of an area included in the captured image, or a marking included in the area. Here, the specific type of "marking" is arbitrary, but it may be, for example, a marking directly written on the actual recognition target RO, a marking written on a part of the captured image corresponding to the recognition target RO, or the like.

[0045] The calculation unit 35b is a calculation means for calculating the area of ​​the recognition target RO or the ratio of the area to the captured image based on the size of a reference target included in the captured image. Here, the "reference target" means an object that serves as a reference when calculating the area, etc. of the recognition target RO, and corresponds to, for example, a part of the skeleton 3, an opening or hole (e.g., P concrete, etc.) provided in the skeleton 3, a peripheral member provided around the skeleton 3 (e.g., scaffolding, protective net, etc.), a dimensional tape provided on the skeleton 3, etc.

[0046] The providing unit 35c is a providing means for providing information indicating the recognition result of the recognizing unit 35a and the calculation result of the calculating unit 35b (hereinafter referred to as "result information"). Details of the processing executed by the recognition-side control unit 35 will be described later.

[0047] (Configuration - Recognition device - Recognition side storage unit) The recognition-side storage unit 36 ​​is a recognition-side storage means for recording programs and various data required for the operation of the recognition device 30.

[0048] The result information can be provided by the management system 1 as described above. Therefore, the user to whom the result information is provided can easily recognize the recognition target RO, and can also accurately recognize the area of ​​the recognition target RO, thereby improving the recognizability of the recognition target RO.

[0049] (Recognition processing) Next, a description will be given of the recognition process executed by the recognition-side control unit 35 of the recognition device 30 in the management system 1 configured as above. Fig. 3 is a flowchart of the recognition process according to the embodiment (in the following description of each process, steps are abbreviated as "S"). Fig. 4 is a diagram showing an example of displaying result information.

[0050] The recognition process is a process for recognizing a recognition target RO from a captured image of a predetermined area of ​​a structure. The timing for executing this recognition process is arbitrary, but in the embodiment, the recognition process is started after the imaging device 10, the collection device 20, and the recognition device 30 are powered on.

[0051] When the recognition process is started, as shown in FIG. 3, the recognition-side control unit 35 acquires information including at least one captured image captured by the imaging device 10 (hereinafter referred to as "captured image information") at SA1.

[0052] The method for acquiring this captured image information is arbitrary, but in the embodiment, the image capturing side control unit 16 of the image capturing device 10 uploads the captured image information to the collection device 20, and then the recognition side control unit 35 acquires the captured image information by automatically (or manually) downloading it from the collection device 20 at a predetermined timing (for example, a predetermined time after the upload, or at regular intervals, etc.).

[0053] In the embodiment, the captured image information will be described as including at least one captured image, image identification information that uniquely identifies the captured image, and imaging position information that indicates the imaging position of the captured image.

[0054] In SA2, the recognition-side control unit 35 determines whether a correction instruction has been received via the recognition-side operation unit 31 (for example, whether a correction instruction has been received within a predetermined time after the start of the processing of SA2). Here, the "correction instruction" means an instruction to adjust the shape, orientation, or size of the captured image, to delete a part of the captured image, or to combine multiple captured images.

[0055] Then, if the recognition side control unit 35 determines that the correction instruction has been accepted (SA2, Yes), it proceeds to SA3, and if it determines that the correction instruction has not been accepted (SA2, No), it proceeds to SA4.

[0056] In SA3, the recognition-side control unit 35 corrects at least one captured image included in the captured image information acquired in SA1.

[0057] The captured image may be corrected in any manner, but may be corrected, for example, as follows.

[0058] That is, for adjusting the shape of the captured images, a known image correction method may be used to adjust the shape of each of the multiple captured images according to the lens type of the lens type information included in the captured image information acquired by SA1.

[0059] As an example, when the lens type information is a fisheye lens (or a wide-angle lens), the captured image may be fisheye corrected (or wide-angle corrected) based on the correction amount (or a predetermined correction amount) input via the recognition side operation unit 31, and then the corrected image may be adjusted by performing trapezoid correction.

[0060] Furthermore, the orientation of each of the plurality of captured images may be adjusted using a known image correction method based on the position, shape, or size of a correction reference object included in the captured image. Here, the "correction reference object" refers to an object that serves as a reference when correcting a captured image, and includes, for example, a part of the skeleton 3, an opening or hole (e.g., P-con, etc.) provided in the skeleton 3, peripheral members provided around the skeleton 3 (e.g., scaffolding, protective net, etc.), dimensional tape provided on the skeleton 3, etc.

[0061] As an example, after displaying the captured image in a correction area (not shown) on the screen of the recognition side output unit 33 (display means) for correcting the captured image, multiple positions of correction reference targets included in the captured image can be specified via the recognition side operation unit 31, and the area connecting the positions of the specified correction reference targets with lines can be deformed via the recognition side operation unit 31, thereby making adjustments.

[0062] In addition, to delete a portion of the captured image, a known image correction method may be used to delete the deletion target included in each of the multiple captured images based on the position, shape, or size of the correction reference target included in the captured image.

[0063] As an example, after the captured image is displayed in the correction area on the screen of the recognition-side output unit 33, the position, shape, or size of the peripheral member 4, which is the correction reference object included in the captured image, may be specified via the recognition-side operation unit 31, a portion included in the captured image that matches the specified position, shape, or size may be extracted, and the extracted portion may be deleted. Alternatively, after the captured image is displayed on the screen of the recognition-side output unit 33, the position, shape, and size of a partial area of ​​the body 3, which is the correction reference object included in the captured image, may be specified via the recognition-side operation unit 31, an area other than the specified area may be deleted (so-called trimming), and then the specified area may be enlarged and deformed via the recognition-side operation unit 31.

[0064] Furthermore, the synthesis of the captured images may be performed using a known image synthesis method, after (or without) adjusting the shape of the captured images, by arranging the multiple captured images in a predetermined matrix based on the image identification information and imaging position information contained in the captured image information acquired by SA1, and then connecting adjacent captured images.

[0065] In SA4, the recognition unit 35a recognizes the recognition target RO included in the captured image corrected or not corrected in SA3. Here, in the embodiment, "recognizing the recognition target RO" means identifying the shape, size (width and height), or position of the recognition target RO in the captured image.

[0066] The method for recognizing this recognition target RO is arbitrary, but in the embodiment, a known image analysis method is used to recognize it based on the color, position, shape, size of the area contained in the captured image, or the markings contained in the area.

[0067] For example, after the captured image is displayed in a recognition area (not shown) for recognizing the recognition target RO among the areas on the screen of the recognition side output unit 33, an area corresponding to a predetermined color, position, shape, size, or the marking specified (or predetermined) via the recognition side operation unit 31 may be extracted from the captured image, and the extracted area may be automatically recognized as the recognition target RO to be recognized.

[0068] In particular, when the captured image includes multiple recognition targets RO of different types (in FIG. 1, a first recognition target RO1 and a second recognition target RO2), the color, position, shape, size, or marking corresponding to each recognition target RO may be specified for each type (or may be determined in advance for each type), and areas corresponding to the specified color, etc. may be extracted from the captured image, and the extracted areas may be recognized as the multiple recognition targets RO to be recognized (i.e., the multiple recognition targets RO may be identified and recognized based on the color, position, shape, or size of the area included in the captured image). This makes it possible to provide result information regarding the multiple recognition targets RO, allowing a user provided with the result information to easily recognize the multiple recognition targets RO and accurately recognize the areas of the multiple recognition targets RO.

[0069] Alternatively, in addition to the above-described automatic recognition method, a predetermined area may be selected from the captured image displayed in the recognition area via the recognition-side operation unit 31, and the selected area may be recognized as the recognition target RO to be recognized.

[0070] Returning to FIG. 3, in SA5, the calculation unit 35b calculates the area of ​​the recognition target RO recognized in SA4 or the ratio of the area to the captured image (specifically, the captured image corrected or not corrected in SA3).

[0071] The method for calculating the area or the proportion of the area of ​​this recognition target RO is arbitrary, but in the embodiment, a known image analysis method is used to calculate it based on the size of the reference target included in the captured image corrected or uncorrected by SA3.

[0072] For example, when the reference object is a hole provided in the body 3, the area of ​​the recognition object RO may be calculated by acquiring the area of ​​the hole in a predetermined manner (for example, by inputting it via the recognition side operation unit 31, or by acquiring it from the recognition side memory unit 36 ​​in which information indicating the area of ​​the hole is stored), and multiplying the ratio of the area corresponding to the hole included in the captured image to the area of ​​the recognition object RO by the acquired area of ​​the hole (note that if multiple recognition objects RO are recognized in SA4, the area is calculated for each recognition object RO).

[0073] Furthermore, the calculation of the area ratio may be performed by acquiring the area of ​​the captured image in a predetermined manner (for example, by inputting it via the recognition-side operation unit 31, or by acquiring it from the recognition-side memory unit 36 ​​in which information indicating the area of ​​the captured image is stored), and then dividing the calculated area of ​​the recognition target RO by the acquired area of ​​the captured image (note that if multiple recognition targets RO are recognized in SA4, the calculation is performed for each recognition target RO).

[0074] In SA6, the providing unit 35c creates result information.

[0075] The method for creating this result information is arbitrary, but in the embodiment, it may be created by associating with each other recognition result information indicating the recognition result of SA4 (e.g., information indicating the shape, size, and position of the recognition target RO in the captured image), calculation result information indicating the calculation result of SA5 (e.g., information indicating the area of ​​the recognition target RO or the ratio of that area to the captured image), image information acquired by a predetermined method (e.g., information acquired from an external device or the recognition-side storage unit 36, indicating a drawing image corresponding to the skeleton 3 included in the captured image (or the captured image used in SA4)), and creation date and time information indicating the creation date and time. However, without being limited to this, for example, the image information or creation date and time information may be omitted, or other information other than the above information (e.g., skeleton information specifying the skeleton 3 related to the recognition target RO (e.g., information indicating the name, position, or size (width, height) of the skeleton)) may also be associated with each other.

[0076] In SA7, the providing unit 35c provides the result information created in SA6 to various terminal devices (specifically, the imaging device 10 or a terminal device other than the imaging device 10).

[0077] The method for providing this result information is arbitrary, but in this embodiment, the result information is provided by uploading it to a website on the Internet that can be accessed from various terminal devices and from which the information posted on the website can be viewed. However, this is not limited to this, and the result information may be provided by, for example, sending it to various terminal devices by email. Alternatively, the result information may be provided to various terminal devices by uploading it to the collection device 20.

[0078] In addition, in the embodiments, after downloading the result information from the website, etc., the various terminal devices display the result information on the screen of the display means (e.g., a display) of the various terminal devices. Note that the "display means of the various terminal devices" corresponds to the "display means" in the claims.

[0079] In this case, when displaying the result information, it is preferable to display the recognition result information contained in the result information on the display means in an identifiable manner (for example, in an identifiable color, or line thickness or type, etc.).

[0080] For example, when the recognition result information included in the result information is displayed graphically, the recognition result information R1 may be displayed by superimposing the recognition result information R1 on the image information RP included in the result information R, as shown in FIG. 4. In particular, when multiple recognition targets RO of different types (specifically, a first recognition target RO1 and a second recognition target RO2) are recognized in SA4, the recognition result information R11 and R12 relating to the multiple recognition targets RO may be displayed in different modes (for example, different colors, line thicknesses, types, etc.) as shown in FIG. 4 so that the multiple recognition targets RO can be easily recognized. Furthermore, the calculation result information R2 included in the result information R may be displayed in association with the corresponding recognition result information R1. As an example, as shown in FIG. 4, text may be displayed near the corresponding recognition result information R1 (in FIG. 4, information indicating the area of ​​the recognition target RO is displayed as "(1000 cm)"). 2 )").

[0081] Furthermore, when the recognition result information is displayed in text, the recognition result information and the calculation result information for each recognition target RO among the result information may be displayed in a table format in association with each other, and the recognition result information may be displayed in a manner different from the calculation result information (for example, a different color, type of character, or line thickness, etc.) (however, this is not limited to this, and for example, the information may be displayed in the same manner).

[0082] By displaying the result information on various terminal devices in this way, users of the various terminal devices can visually recognize the recognition results, which makes it easier for the users to recognize the recognition results.

[0083] Returning to FIG. 3, at SA8, the recognition-side control unit 35 determines whether or not the timing to end the recognition process (hereinafter referred to as "end timing") has arrived.

[0084] The method for determining whether or not this end timing has arrived is arbitrary, but for example, the determination may be made based on whether or not a predetermined operation has been accepted via the recognition side operation unit 31, and if the predetermined operation has been performed, it is determined that the end timing has arrived, and if the predetermined operation has not been performed, it is determined that the end timing has not arrived.

[0085] If the recognition-side control unit 35 determines that the end timing has arrived (SA8, Yes), it ends the recognition process. On the other hand, if the recognition-side control unit 35 determines that the end timing has not arrived (SA8, No), it returns to SA1 and repeats the processes from SA1 to SA8 until it determines that the end timing has arrived at SA8.

[0086] By performing the above-described recognition process, result information can be provided to various terminal devices. Therefore, users of various terminal devices can easily recognize the recognition target RO and can accurately grasp the area of ​​the recognition target RO, thereby improving the recognizability of the recognition target RO.

[0087] (Effects of the embodiment) As described above, according to the embodiment, the recognition unit 35a recognizes the recognition target RO based on the color, position, shape, or size of an area included in the captured image or a marking included in the area, the calculation unit 35b calculates the area of ​​the recognition target RO or the ratio of the area to the captured image based on the size of a reference object included in the captured image, and the providing unit 35c provides result information indicating the recognition result of the recognition unit 35a and the calculation result of the calculation unit 35b, so that the result information can be provided. Therefore, a user who is provided with the result information can easily recognize the recognition target RO and can accurately recognize the area of ​​the recognition target RO, thereby improving the user's recognition of the recognition target RO.

[0088] Furthermore, when the captured image contains multiple recognition targets RO of different types, the recognition unit 35a identifies and recognizes each of the multiple recognition targets RO based on the color, position, or shape of the area included in the captured image, or the markings included in the area, so that result information regarding the multiple recognition targets RO can be provided, allowing the user who is provided with the result information to easily recognize the multiple recognition targets RO and accurately recognize the area of ​​the multiple recognition targets RO.

[0089] Furthermore, since the providing unit 35c causes the display means to display the recognition result of the recognizing unit 35a included in the result information in an identifiable manner, the user to whom the result information is provided can visually recognize the recognition result, which makes it easier to improve the user's recognition of the recognition result.

[0090] [III] Modifications to the embodiment Although the embodiments of the present invention have been described above, the specific configurations and means of the present invention can be modified and improved as desired within the scope of the technical ideas of the inventions set forth in the claims. Such modifications will be described below.

[0091] (About the problem to be solved and the effects of the invention) First, the problems that the invention aims to solve and the effects of the invention are not limited to those described above, and the present invention may solve problems that are not described above or achieve effects that are not described above, or may solve only some of the problems that are described or achieve only some of the effects that are described.

[0092] (Regarding decentralization and integration) Furthermore, the electrical components described above are merely functional concepts and do not necessarily need to be physically configured as illustrated. In other words, the specific distribution and integration of each component is not limited to those illustrated. All or part of the components can be functionally or physically distributed or integrated in any unit depending on various loads and usage conditions. Furthermore, the term "system" in this application is not limited to a system consisting of multiple devices, but also includes a system consisting of a single device. Furthermore, the term "device" in this application is not limited to a system consisting of a single device, but also includes a system consisting of multiple devices. Furthermore, the data structure of each piece of information described in the above embodiment may be arbitrarily modified. For example, the recognition device 30 may be distributed across multiple devices configured to communicate with each other, with the recognition-side control unit 35 provided in some of these devices and the recognition-side memory unit 36 ​​provided in others of these devices.

[0093] (shape, numbers, structure, time series) The components illustrated in the embodiments and drawings may be modified and improved as desired within the scope of the technical concept of the present invention in terms of shape, numerical value, or the structure or chronological relationship of multiple components.

[0094] (About the object of recognition) In the above embodiment, the recognition target RO has been described as a base material provided between the structure's skeleton 3 and the tile, but is not limited thereto. For example, it may be the skeleton 3 to which paint or fluorescent material is applied, or a member (e.g., a base material) located near the skeleton 3. This makes it easier for the recognition unit 35a to recognize the recognition target RO based on the paint or fluorescent material contained in the recognition target RO, thereby improving the recognition accuracy of the recognition unit 35a for the recognition target RO.

[0095] (About the management system) In the above embodiment, the management system 1 is described as including the collection device 20, but this is not limited to this. For example, if the imaging device 10 and the recognition device 30 can communicate directly with each other via the network 2, the collection device 20 may be omitted. In this case, in SA1 of the recognition process, the recognition-side control unit 35 may directly receive and acquire the captured image information transmitted from the imaging device 10.

[0096] (About the imaging device) In the above embodiment, the imaging device 10 has been described as a mobile terminal, but the imaging device 10 is not limited to this and may be, for example, a camera device that can communicate with the collection device 20.

[0097] In the above embodiment, the imaging device 10 is described as including the imaging unit 13, which is a camera for capturing two-dimensional images, but this is not limited thereto. For example, the imaging device 10 may be a 3D scanner, and may include the imaging unit 13, which is a camera for capturing three-dimensional images. Alternatively, the imaging device 10 may include the imaging unit 13, which is an all-around camera.

[0098] (About recognition processing) In the above embodiment, the processing of SA3 is described as being performed by the recognition device 30, but this is not limiting. For example, the processing may be performed by another device (for example, the imaging device 10, the collection device 20, or a terminal device other than these devices).

[0099] In the above embodiment, SA4 is described as using a known image analysis method to recognize the recognition target RO based on the color, position, shape, and size of an area included in the captured image, or on markings included in the area. However, this is not limited to this. For example, in addition to the above method, the recognition target RO may be recognized using a machine learning method using artificial intelligence. As an example, a teacher image may be created based on various captured images related to the recognition target RO, machine learning may be performed based on the created teacher image, and the learning results may be used to recognize the recognition target RO from the captured image included in the captured image information. This can improve the recognition accuracy of the recognition unit 35a.

[0100] In the above embodiment, SA6 generates result information including recognition result information indicating the recognition result of SA4 and calculation result information indicating the calculation result of SA5. However, this is not limited to this. For example, if result information (hereinafter referred to as "past result information") related to the same structure 3 has been generated in the past, the recognition result information (or the calculation result information) may be compared with the recognition result information (or the calculation result information) of the past result information, and result information including comparison information indicating the comparison result may be generated. Here, the specific content of the comparison information is arbitrary, and may include, for example, information indicating the fluctuation value of the position, size, or area of ​​the recognition target RO. This makes it possible to provide comparison information, making it possible to easily grasp the difference between the most recent recognition result information (or calculation result information) and the past recognition result information (or calculation result information).

[0101] In the above embodiment, the result information is provided to various terminal devices in SA7. However, the present invention is not limited to this. For example, the result information may be provided to a user of the recognition device 30. Specifically, the providing unit 35c may provide the result information by causing the recognition-side output unit 33 (display means) to display the result information. In this case, the "recognition-side output unit 33" corresponds to the "display means" in the claims.

[0102] (About the results information) In the above embodiment, the display method of the calculation result information in various terminal devices has been described as displaying the calculation result information in association with the corresponding recognition result information. However, the present invention is not limited to this. For example, the total value of the area or area ratio of each recognition target RO in the calculation result information may be displayed.

[0103] (About recognition processing) In the above embodiment, it has been described that the processes SA2 and SA3 are performed in the recognition process, but this is not limiting, and for example, the processes SA2 and SA3 may be omitted.

[0104] (Addendum) The recognition system of Appendix 1 is a recognition system for recognizing a recognition target from an image captured of a predetermined area of ​​a structure, and includes: recognition means for recognizing the recognition target based on the color, position, shape, or size of an area included in the captured image, or a marking included in the area; calculation means for calculating the area of ​​the recognition target or the ratio of that area to the captured image based on the size of a reference object included in the captured image; and provision means for providing result information indicating the recognition result of the recognition means and the calculation result of the calculation means.

[0105] The recognition system of Supplementary Note 2 is the recognition system described in Supplementary Note 1, wherein when the captured image includes a plurality of recognition targets of different types, the recognition means identifies and recognizes each of the plurality of recognition targets based on the color, position, or shape of an area included in the captured image, or a marking included in the area.

[0106] The recognition system of Supplementary Note 3 is the recognition system according to Supplementary Note 1 or 2, wherein the providing means causes the display means to display the recognition result of the recognition means included in the result information in an identifiable manner.

[0107] The recognition system of Supplementary Note 4 is a recognition system according to any one of Supplementary Notes 1 to 3, wherein the recognition target includes a body of the structure coated with paint or fluorescent material or a component located in the vicinity of the body.

[0108] (Effect of supplementary notes) The recognition system described in Supplementary Note 1 includes a recognition means for recognizing a recognition target based on the color, position, shape, or size of an area included in a captured image or a marking included in the area, a calculation means for calculating the area of ​​the recognition target or the ratio of the area to the captured image based on the size of a reference object included in the captured image, and a provision means for providing result information indicating the recognition result of the recognition means and the calculation result of the calculation means, so that the result information can be provided. Thus, a user who is provided with the result information can easily recognize the recognition target and can accurately recognize the area of ​​the recognition target, thereby improving the user's ability to recognize the recognition target.

[0109] According to the recognition system described in Appendix 2, when a captured image contains multiple recognition targets of different types, the recognition means identifies and recognizes each of the multiple recognition targets based on the color, position, or shape of an area included in the captured image, or a marking included in the area, so that result information regarding the multiple recognition targets can be provided, allowing a user who is provided with the result information to easily recognize the multiple recognition targets and accurately recognize the areas of the multiple recognition targets.

[0110] According to the recognition system described in Appendix 3, the providing means causes the display means to display the recognition result of the recognition means included in the result information in an identifiable manner, so that the user to whom the result information is provided can visually recognize the recognition result, which makes it easier to improve the user's recognizability of the recognition result.

[0111] According to the recognition system described in Appendix 4, the recognition target includes the structure's skeleton or a component located near the skeleton to which paint or fluorescent substance is applied, making it easier for the recognition means to recognize the recognition target based on the paint or fluorescent substance contained in the recognition target, thereby improving the recognition accuracy of the recognition means. [Explanation of symbols]

[0112] 1 Management System 2 Network 3 skeleton 10. Imaging device 11. Imaging control panel 12. Imaging communication unit 13 Imaging unit 14. Image capture output section 15 Imaging power supply unit 16. Imaging control section 17 Imaging side memory unit 20 Collection Device 21 Collector's Communications Department 22 Collection side control unit 23 Collection side memory unit 30 recognition device 31 Recognition side operation section 32 Recognition side communication unit 33 Recognition side output unit 34 Recognition side power supply section 35 Recognition side control unit 35a Recognition part 35b Calculation part 35c supply department 36 Recognition side memory unit R result information R1 recognition result information R11 Recognition result information R12 Recognition result information R2 calculation result information RO Recognition Target RO1 First recognition target RO2 Second recognition target RP Image Information

Claims

1. A recognition system for recognizing a recognition target from a captured image of a predetermined area of ​​a structure, a recognition means for recognizing the recognition target based on the color, position, shape, or size of an area included in the captured image, or a marking included in the area; a calculation means for calculating an area of ​​the recognition target or a ratio of the area to the captured image based on the size of a reference target included in the captured image; providing means for providing result information including information indicating a recognition result of the recognition means and information indicating a calculation result of the calculation means; The providing means comprises: displaying information indicating the recognition result of the recognition means, which is included in the result information, in a distinguishable manner by a display means; the display means displays information indicating the recognition result of the recognition means, which is included in the result information, in graphics, and displays information indicating the calculation result of the calculation means, which is included in the result information, in text, in a position adjacent to information indicating the corresponding recognition result of the recognition means; When the recognition means recognizes a plurality of recognition targets of different types, information indicating the recognition results of the recognition means for the plurality of recognition targets is displayed in different modes. Recognition system.

2. When the captured image includes a plurality of recognition targets of different types, the recognition means identifies and recognizes each of the plurality of recognition targets based on the color, position, or shape of an area included in the captured image, or a marking included in the area. The recognition system of claim 1 .

3. The recognition target includes a skeleton of the structure coated with paint or a fluorescent substance or a member located in the vicinity of the skeleton.

3. A recognition system according to claim 1 or 2.

Images