Equipment and materials counting device, method, and program

Abstract

To provide a standalone and user-friendly equipment counting device, method, and program. [Solution] The equipment counting device 1 includes a trained model 100 that is machine-learned to individually identify the areas of equipment in image data and output the identified equipment areas, an image data acquisition unit 110 that acquires image data to be counted using a camera 10, a counting range input unit 120 that outputs the image data to be counted to a display device 20 and allows the user to input the counting range using an input device 30, a counting processing unit 140 that counts the number of equipment areas to be counted that are identified by the trained model 100 and included in the counting range, and an output control unit 150 that outputs the counted number to the display device 20 and / or controls the output of the equipment areas to be counted superimposed on the image data to be counted to the display device 20.

Description

Technical Field

[0001] The present invention relates to an equipment counting device, method, and program for counting the number of equipment and materials.

Background Art

[0002] In civil engineering and construction sites, various equipment and materials are used. When the use of equipment and materials at the site is completed, they are stored in a predetermined storage location and then transported to other sites for use. On the other hand, equipment and materials may be damaged or lost during use at the site. Therefore, it is important to count the actual number of equipment and materials stored in the storage location and manage them. However, since a large number of the same type of equipment and materials are stacked and stored in the storage location, the visual counting operation is very complicated and time-consuming.

[0003] In order to solve such problems, a system has been proposed that images the equipment and materials stored in the storage location with a camera and counts the number of equipment and materials using image analysis AI technology from the captured image data (see Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the devices described in Patent Documents 1 and 2 consist of a first device for acquiring image data and a second device for counting the number of equipment items using AI technology, which are configured as separate devices. However, considering actual operation, the second device for counting the number of equipment items needs to be installed in an office or other location remote from the work site, as it needs to be able to handle multiple sites. As a result, there is a problem in that workers at the storage location cannot know the number of equipment items that have been imaged in real time.

[0006] To solve these problems, one might consider connecting a first device that acquires image data and a second device that counts the number of materials and equipment via some kind of network, and sending and receiving image data and counting information between the two. However, the storage location of materials and equipment is not always within the coverage area of ​​a mobile communication network. Therefore, it would be necessary to form a local area network at the storage location, but this is not practical due to cost and other issues. For these reasons, there has been a demand for a standalone counting device that can count the number of materials and equipment.

[0007] Furthermore, since such counting devices need to be transported to each site and storage location, a portable, compact design is preferable. On the other hand, compact devices have limitations in processing power. Image analysis AI technology, in particular, places a heavy processing load on the system, so there is a need for a device with low processing load and ease of use.

[0008] This invention has been made in view of the above circumstances, and its objective is to provide a standalone and user-friendly equipment counting device, method, and program. [Means for solving the problem]

[0009] To achieve the above objective, the present invention provides a material counting device for counting the number of materials, comprising: a trained model that has been machine-learned to take image data containing materials as input, individually identify the regions of materials in the image data, and output the identified material regions; a means for acquiring countable image data containing materials using a camera; a means for outputting the acquired countable image data to a display device and for allowing a user to input a counting range within the countable image data using an input device; a counting means that uses the trained model to identify material regions based on the countable image data and includes those within the input counting range as countable material regions, and counts the number of countable material regions; and an output control means that outputs the counted number to a display device and / or controls the output of the countable material regions superimposed on the countable image data to the display device.

[0010] Furthermore, the present invention is a method for counting the number of equipment and materials using a computer, comprising: an acquisition step of target image data to be counted, in which a camera is used to acquire target image data to be counted that includes equipment and materials; a counting range input step, in which the acquired target image data to be counted is output to a display device and the user inputs a counting range within the target image data using an input device; a counting step, in which a trained model, which has been trained to take image data containing equipment and materials as input, individually identifies the areas of equipment and materials in the image data and outputs the identified equipment and materials areas, is used to define the equipment and materials areas identified based on the target image data that are included in the input counting range as target equipment and materials areas, and the number of target equipment and materials areas is counted; and an output control step, in which the counted number is output to a display device and / or the target equipment and materials areas are superimposed on the target image data and output to the display device. [Effects of the Invention]

[0011] According to the present invention, a series of processes from image capture of equipment and materials to counting of equipment and materials are performed by a single device called an equipment and materials counting device, enabling standalone operation. Furthermore, since the present invention counts equipment and materials within a counting range input by the user, the processing load can be reduced or the response speed can be increased, resulting in improved usability. [Brief explanation of the drawing]

[0012] [Figure 1] Functional block diagram of the equipment counting device according to the first embodiment [Figure 2] A diagram illustrating an example of equipment and materials placed in a storage area. [Figure 3] An example of the display screen when entering the counting range. [Figure 4] A diagram explaining the criteria for determining whether or not something is included in the counting range. [Figure 5] An example of a display screen showing the counting results. [Figure 6] A flowchart explaining the process of counting equipment and materials. [Figure 7] Functional block diagram of the equipment counting device according to the second and third embodiments. [Figure 8] A diagram illustrating an example of equipment and materials placed in a storage area. [Figure 9] A diagram illustrating an example of equipment and materials placed in a storage area. [Figure 10] A diagram illustrating an example of equipment and materials placed in a storage area. [Figure 11] Functional block diagram of the equipment counting device according to the fourth embodiment [Figure 12] An example of a display screen showing the counting results. [Figure 13] System configuration diagram relating to the fifth embodiment [Modes for carrying out the invention]

[0013] (First Embodiment) The capital equipment counting device according to the first embodiment of the present invention will be described with reference to the drawings. The capital equipment counting device according to the present embodiment counts the number of capital equipment included in the image data captured by a camera. Examples of the capital equipment include angle steel, suspension fittings, and saddle materials.

[0014] As shown in FIG. 1, the capital equipment counting device 1 includes a camera 10, a display device 20, an input device 30, a position detection unit 40, a learned model 100, an image data acquisition unit 110, a counting range input unit 120, a parameter setting unit 130, a counting processing unit 140, an output control unit 150, a storage unit 200, and a form creation unit 210.

[0015] The capital equipment counting device 1 is composed of a conventionally well-known computer including a main arithmetic unit, a main storage unit, an auxiliary storage unit, an input device, an output device, etc. The capital equipment counting device 1 is composed of a small computer that can be carried and used by a user. The implementation form of the capital equipment counting device 1 is not limited. The capital equipment counting device 1 can be implemented as dedicated hardware. Also, the capital equipment counting device 1 can be implemented by installing a program in a computer via a storage medium or a network. In the present embodiment, a portable tablet terminal having a display as the display device 20, a touch panel as the input device 30 attached to the display, and a camera 10 is used, and the capital equipment counting device 1 is implemented by installing a program in the tablet terminal. Note that the camera 10, the display device 20, and the input device 30 can be implemented as separate bodies from the capital equipment counting device 1 main body and connected to the main body to implement the capital equipment counting device 1.

[0016] The camera 10 is an imaging device that captures at least a two-dimensional still image in the visible light region and outputs image data. The camera 10 may further be capable of capturing an infrared region or obtaining depth information.

[0017] The display device 20 is a well-known display device that displays and outputs various types of information, including image data, as images. The display device 20 may also be equipped with a speaker or other output device that outputs audio information.

[0018] The input device 30 is a device for inputting various types of information from the user. Examples of input devices 30 include a touch panel, a pointing device such as a mouse or trackball, a keyboard, and a microphone. In this embodiment, a touch panel is used.

[0019] The position detection unit 40 is a device that detects its own current position. The position detection unit 40 detects its position using a satellite positioning system such as GNSS (Global Navigation Satellite System). In addition, to improve detection accuracy, the position detection unit 40 can arbitrarily combine and use any position detection technology, such as satellite signals related to the satellite positioning system, carrier signals from mobile communication networks, and position information from wireless LAN (Local Area Network) beacon signals installed in various locations.

[0020] The trained model 100 is a machine learning model that takes image data containing equipment as input, individually identifies the regions of equipment in the image data, and outputs the identified equipment regions. In this embodiment, the trained model 100 is trained without supervision using an instance segmentation algorithm.

[0021] Furthermore, the trained model 100 according to this embodiment is trained to correspond to the type of equipment. Here, the trained model 100 may recognize the entire equipment as the equipment area, or it may recognize a specific part, location, or surface of the equipment as the equipment area, depending on the shape and storage method of the equipment. For example, as shown in Figure 2, equipment 500 consisting of long members such as angle steel is often stacked vertically and stored on a stand such as a pallet 510. For this reason, in order to count the number of equipment 500, the trained model 100 is trained to recognize the end face of the angle steel as the equipment area based on image data taken of the equipment 500 from the side, where the end face is visible. Also, for example, lifting brackets are placed on a surface in the storage location without being stacked vertically. For this reason, in order to count the equipment, the trained model 100 is trained to recognize the surface other than the bottom surface of the lifting bracket as the equipment area based on image data taken of the equipment from above or diagonally above.

[0022] In this embodiment, there are multiple pre-trained models 100, each corresponding to a different type of equipment. The user can select which pre-trained model 100 to use using the input device 30.

[0023] The image data acquisition unit 110 acquires counting target image data 901, which is image data containing equipment and materials, using the camera 10. The image data acquisition unit 110 can store the acquired image data 901 in the storage unit 200 as needed. At this time, the image data acquisition unit 110 can store additional information such as the date and time of shooting and location information detected by the location detection unit 40 in the storage unit 200.

[0024] The counting range input unit 120 outputs the counting target image data 901 acquired by the image data acquisition unit 110 to the display device 20 and allows the user to input the counting range 902 within the counting target image data 901 using the input device 30. The counting range 902 may be a rectangular area or an arbitrary closed area drawn freehand. Figure 3 is an example of the display screen when the counting range 902 is input by the counting range input unit 120. In Figure 3, an example of counting angle steel 500 is shown, and the trained model 100 is trained to recognize the end face of the angle steel 500 as the material area.

[0025] The parameter setting unit 130 is a functional unit that allows the user to set the operating parameters of the counting processing unit 140, which will be described later, using the input device 30 and the display device 20. In the example shown in Figure 3 above, a parameter setting area 910 is provided below the counting target image data 901, and in this area 910, the "size upper limit" and "size lower limit," which are two of the operating parameters, can be set.

[0026] The counting processing unit 140 uses the trained model 100 to extract equipment and materials regions 903 identified based on the image data 901 to be counted, which are included in the input counting range 902, as equipment and materials regions 904 to be counted, and counts the number of equipment and materials regions 904.

[0027] In this embodiment, the counting processing unit 140 first takes the entire image data 901 to be counted acquired by the image data acquisition unit 110 as input and uses the trained model 100 to perform identification processing of the equipment area 903. Then, it receives the counting range 902 as input from the counting range input unit 120, extracts the equipment area 904 to be counted based on the input counting range 902, and counts the number of equipment areas 904 to be counted.

[0028] Furthermore, various criteria can be used to determine whether or not the equipment area 903 is included in the counting range 902. Figure 4 shows a diagram illustrating the criteria for determining the counting range. Figure 4 is an example of counting angle steel, similar to Figure 3, and it is assumed that the trained model 100 has been trained to recognize the end face of the angle steel as the equipment area 903.

[0029] One example of a case where the equipment area 903 is included in the counting range 902 is when the entire equipment area 903 is included in the counting range 902, as shown in Figure 4(a). Another example is when a portion of the equipment area 903 is included in the counting range 902, as shown in Figure 4(b). Yet another example is when the centroid point 903a of the equipment area 903 is included in the counting range 902, as shown in Figure 4(c). In this case, multiple centroid points 903a of the equipment area 903 may be set.

[0030] Furthermore, as shown in Figure 1, the counting processing unit 140 includes a parameter storage unit 141 and a filtering processing unit 142. The parameter storage unit 141 stores the operating parameters of the counting processing unit 140. In this embodiment, the parameter storage unit 141 stores the recognition sensitivity range 141a for dimensions within the image of the equipment area 903, which is the equipment area 904 to be measured. This recognition sensitivity range 141a corresponds to the "size upper limit" and "size lower limit" mentioned above. In addition, this recognition sensitivity range 141a stores predetermined values ​​and, as mentioned above, can be set and changed by the user using the parameter setting unit 130.

[0031] The filtering processing unit 142 filters the equipment regions 903 output from the trained model 100 using the recognition sensitivity range 141a. That is, the processing target in the counting processing unit 140, which extracts the equipment regions 904 to be counted from the equipment regions 903 using the counting range 902, is the equipment regions 903 that have been filtered by the filtering processing unit 142. In this embodiment, only equipment regions 903 whose occupied area in the image data 901 is greater than or equal to the "lower size limit" and less than or equal to the "upper size limit" are processed. Through this filtering process, even if the equipment regions 903 recognized by the trained model 100 include misrecognized areas, the misrecognized areas can be manually removed based on size.

[0032] The output control unit 150 outputs the number of countable equipment areas 904 counted by the counting processing unit 140 to the display device 20, and / or controls the output of the countable equipment areas 904 superimposed on the countable image data 901 to the display device 20. In this embodiment, the output control unit 150 performs both count output and superimposed output of the countable equipment areas 904. Figure 5 is an example of an output image by the output control unit 150. As shown in Figure 5, the count 920 is displayed in a separate area from the countable image data 901, and the countable equipment areas 904 are superimposed on the countable image data 901 with a predetermined color and transparency. Here, the color assigned to the countable equipment areas 904 can be arbitrarily determined. For example, all countable equipment areas 904 can be the same color. Alternatively, adjacent countable equipment areas 904 can be assigned different colors. For example, a predetermined number of countable equipment areas 904 that are close to each other (e.g., 10) can be set as a single group, and each group can be assigned a different color. Alternatively, for example, the countable image data 901 can be divided into multiple areas (e.g., into four equal parts), and the countable equipment areas 904 contained within each area can be assigned a different color.

[0033] Furthermore, the output control unit 150 stores the number of countable equipment areas 904 counted by the counting processing unit 140, as well as the countable equipment areas 904 themselves, in the storage unit 200. The countable equipment areas 904 may be superimposed on the countable image data 901.

[0034] The report creation unit 210 creates report data from various information stored in the storage unit 200, such as countable image data 901, the number of countable equipment items 904, the countable equipment items 904, the date and time of shooting, and location information, either at the user's request or automatically. The report data can be displayed on the display device 20 or output to other devices via communication means (not shown).

[0035] Next, the counting process for equipment and materials using the equipment and materials counting device will be explained with reference to the flowchart in Figure 6.

[0036] First, image data containing equipment and materials is acquired using camera 10 (Step S1). Next, the acquired image data is input to the trained model 100 to detect the equipment and materials area (Step S2). Next, the image data is displayed on the display device 20 and the user is prompted to input the counting range using the input device 30 (Step S3). Next, the equipment and materials areas that fall within the counting range are extracted as the equipment and materials area to be counted, and their number is calculated (Step S4). Next, the equipment and materials area to be counted is superimposed on the image data and displayed, along with the number of items (Step S5). The user can check the display and, if necessary, re-enter the counting range (Step S6) or adjust the recognition sensitivity range (Step S7). A report is also generated at the user's request or automatically (Step S8).

[0037] According to this embodiment, a series of processes from image capture of the equipment to counting the equipment are performed by a single device, the equipment counting device 1, thus enabling standalone operation.

[0038] Furthermore, in the equipment counting device 1 according to this embodiment, since the equipment is counted within the counting range input by the user, the processing load can be reduced or the response speed can be increased, resulting in improved usability. Generally, recognition processing by the trained model 100 has a high processing load and takes time to process. On the other hand, in this embodiment, the recognition processing by the trained model 100 is performed before the input processing of the counting range 902 by the user, so the processing from the input of the counting range 902 by the user to the display of the counting result can be performed in a short time. In other words, the responsiveness is improved, making it user-friendly.

[0039] Furthermore, in this embodiment, the recognition accuracy is improved by filtering the recognition results of the trained model 100 using the recognition sensitivity range 141a, which is one of the operating parameters, in the filtering processing unit 142. In other words, instead of improving accuracy by changing the parameters and performing the recognition processing by the trained model 100 again, the processing is performed on the recognition results of the trained model 100 that have already been recognized. As mentioned above, recognition processing by the trained model 100 is generally computationally intensive and takes time to process. Therefore, in this embodiment, the process of changing parameters to improve recognition accuracy can be performed with low processing load. This results in good responsiveness and makes the system user-friendly.

[0040] Furthermore, in the equipment counting device 1 according to this embodiment, not only are the counting results and the equipment area 904 to be counted displayed on the display device 20, but various information is also output as reports to the display device 20 and other devices, making it user-friendly in operation.

[0041] (Second Embodiment) A second embodiment of the present invention, a material counting device, will now be described. The difference between this embodiment and the first embodiment lies in the method of setting the recognition sensitivity range 141a. Other aspects are the same as in the first embodiment, so only the differences will be explained here.

[0042] As shown in Figure 7, the equipment counting device 1a according to this embodiment is further equipped with a setting unit 160 compared to the equipment counting device 1 according to the first embodiment. The setting unit 160 is equipped with an image analysis unit 161.

[0043] The equipment counting device 1a according to this embodiment automatically sets the recognition sensitivity range 141a using a reference object with known dimensions, in addition to setting it by the user. More specifically, the reference object is placed near the equipment. For example, as shown in Figure 8, a cubic reference object 520 is placed on the pallet 510 on which the equipment 500 is placed. Then, counting target image data 901 is acquired so that the reference object 520 is included together with the equipment 500, and the recognition sensitivity range 141a is automatically set using this counting target image data 901.

[0044] The image analysis unit 161 of the setting unit 160 identifies reference objects 520 contained in the counting target image data 901 using well-known image recognition technology. Based on the known dimensional information of the identified reference object 520 and the size occupied by the reference object 520 in the counting target image data 901, the setting unit 160 calculates and sets the recognition sensitivity range 141a.

[0045] With this type of equipment counting device 1a, the recognition sensitivity range 141a is automatically set, improving convenience. After the recognition sensitivity range 141a is automatically set by the setting unit 160, the user can also arbitrarily set the recognition sensitivity range 141a. Other functions and effects are the same as in the first embodiment.

[0046] (Third embodiment) A third embodiment of the present invention, a materials counting device, will now be described. The difference between this embodiment and the first embodiment lies in the method of setting the materials counting device. Other aspects are the same as in the first embodiment, so only the differences will be explained here. Also, the device configuration of the materials counting device 1 is the same as in the second embodiment, so it will not be shown in the illustration.

[0047] The equipment counting device 1a according to this embodiment performs automatic settings using a setting information carrier that displays setting information, in addition to settings made by the user. Here, the various settings of the equipment counting device 1a can include setting the recognition sensitivity range 141a, as in the second embodiment, as well as selecting a trained model 100.

[0048] A configuration information carrier is a medium on which configuration information is displayed in a predetermined format. Examples of such media include paper or resin panels. The medium may also be the equipment itself, or a structure such as a pallet on which the equipment is placed, or the wall of a storage area. The configuration information on the configuration information carrier may be displayed using characters, numbers, graphics, or other methods, as well as representing the information using barcodes or two-dimensional information codes. A typical example is shown in Figure 9, where the configuration information is printed as a QR code (registered trademark) on a sheet of paper 530, and this sheet of paper 530 is attached to the equipment 500 or pallet 510.

[0049] Examples of configuration information include information indicating the type of equipment or the dimensions of the equipment. Furthermore, the configuration information may consist solely of the information used for configuration, or it may be a combination of information indicating the configuration item and the information used to configure that item. Additionally, various types of supplementary information, such as information regarding storage locations, can be added to the configuration information.

[0050] In this embodiment, the image data acquisition unit 110 acquires setting image data using the camera 10, which includes a setting information carrier displaying the setting information as an image. Here, the setting image data may be the same as the counting target image data 901. That is, the setting image data may include the setting information alone along with the equipment in the counting target image data 901. Alternatively, the setting image data may be acquired separately from the counting target image data 901. In this case, the setting image data may have a different imaging position, direction, and field of view than the counting target image data 901.

[0051] The image analysis unit 161 of the setting unit 160 reads setting information from setting image data using well-known image recognition technology. The setting unit 160 uses the setting information to perform various settings for the equipment counting device 1a. For example, the type of equipment is used as setting information, and a trained model 100 is automatically selected based on this setting information.

[0052] With this type of equipment counting device 1a, the settings for the equipment counting device 1a are automatically set, thus improving convenience. Other functions and effects are the same as in the first embodiment.

[0053] (Fourth embodiment) A fourth embodiment of the present invention, a material counting device, will now be described. This embodiment differs from the first embodiment in the learning state of the trained model 100 and the processing of the counting processing unit 140. Other aspects are the same as in the first embodiment, so only the differences will be described here.

[0054] In each of the embodiments described above, angle steel is given as an example of equipment. Angle steel is stored in a stacked state in the storage area. For this reason, in the embodiments described above, the trained model 100 is configured to recognize the end faces of the angle steel as equipment areas.

[0055] Incidentally, some of the materials and equipment are flat. As shown in Figure 10, these flat materials and equipment 500 are placed in a stacked state in the thickness direction in the storage area. Here, because the materials and equipment 500 are flat, they are in close contact with each other. For this reason, even when observing the materials and equipment 500 from the end face side, it is difficult to individually recognize the end face 502 of each material and equipment 500. Also, because the end face 502 of the flat materials and equipment 500 has a simple rectangular shape, it is difficult to distinguish it from other components included in the background, etc. For these reasons, it is difficult to count stacked flat materials and equipment with the materials and equipment counting devices 1,1a according to the above embodiments.

[0056] Therefore, the pre-trained model 100 of the equipment counting device 1b according to this embodiment is configured as a pre-trained model 100a that recognizes the upper surface 501 of the flat equipment 500 as the equipment area. Furthermore, as shown in Figure 11, the equipment counting device 1b includes an equipment dimension detection unit 170 that detects the dimensions of the equipment within the image of the equipment included in the image data 901 to be counted. The equipment dimension detection unit 170 includes an image analysis unit 171.

[0057] On the other hand, a reference object of known dimensions is placed near the equipment 500. For example, as shown in Figure 10, a regular cube-shaped reference object 520 is placed on the pallet 510 on which the equipment 500 is placed. Here, it is preferable to place the reference object 520 in a position where its front surface is substantially flush with the end face 402 of the equipment 500.

[0058] The image data 901 to be counted is acquired such that it includes the reference object 520 along with the equipment 500. Here, the image data 901 to be counted is acquired by imaging the flat equipment 500 from diagonally above the end face side of the equipment 500 so that it includes both the top surface 501 and the end face 502 of the stacked flat equipment 500, and also includes the reference object 520.

[0059] The image analysis unit 171 of the equipment dimension detection unit 170 identifies reference objects 520 contained in the counting target image data 901 using well-known image recognition technology. Based on the known dimensional information of the identified reference object 520 and the size occupied by the reference object 520 in the counting target image data 901, the equipment dimension detection unit 170 calculates dimensional information that shows the correspondence between the dot pitch in the counting target image data 901 and the dimensions in real space.

[0060] The counting processing unit 140b includes an out-of-area counting unit 143. The counting processing unit 140b recognizes the top surface 501 of the uppermost of the stacked flat-plate-shaped equipment 500 as the equipment area 903 by inputting the image data 901 to be counted into the trained model 100a. The counting processing unit 140b also calculates the equipment area 904 to be counted based on the counting range 902 input using the counting range input unit 120, similar to the embodiments described above. Furthermore, if necessary, the filtering processing unit 142 filters the equipment area 904 to be counted.

[0061] The out-of-area counting unit 143 counts the number of materials and equipment located in areas other than the material and equipment area in the counting target image data 901, based on the dimensions detected by the material and equipment dimension detection unit 170. Specifically, the out-of-area counting unit 143 calculates the number of flat materials and equipment 500 in the counting target image data 901 based on the number of vertical dots in the lower part of area 905 of the material and equipment area 904, the dimension information calculated by the material and equipment dimension detection unit 170, and the known thickness information of the flat material and equipment 500.

[0062] The output control unit 150 calculates the number of equipment items by adding the number of items in the counting target equipment area 904 counted by the counting processing unit 140 and the number of equipment items 500 in areas other than the counting target equipment area counted by the area outside the area counting unit 143, and then subtracting 1. The reason for subtracting 1 is that the topmost equipment item in the area other than the counting target equipment area is equipment item detected by the trained model 100.

[0063] The output control unit 150 controls the system to output the calculated count to the display device 20. The output control unit 150 also controls the system to superimpose the count target equipment area 904 and the area 905 of equipment located outside the count target equipment area detected by the out-of-area counting unit 143 onto the count target image data 901 and output it to the display device 20. Figure 12 is an example of an output image from the output control unit 150.

[0064] As described above, the equipment counting device 1b according to this embodiment can detect the number of flat pieces of equipment with high accuracy. Other functions and effects are the same as in the first embodiment. The functions of the equipment counting device 1b according to this embodiment and the functions of the equipment counting device 1 according to the first embodiment may be implemented as a single device. In this case, the choice of which function to use may be switched by input from the user using the input device 30, or it may be switched automatically as in the third embodiment described above.

[0065] (Fifth embodiment) A fifth embodiment of the present invention will now be described. This embodiment involves linking a materials counting device with a materials management system.

[0066] As shown in Figure 13, the equipment counting device 1 can communicate with the equipment management system 1000 via the network 1090. The communication path between the equipment counting device 1 and the equipment management system 1000 is not specified.

[0067] The Equipment Management System 1000 is a well-known system for managing equipment and materials. The implementation location of the Equipment Management System 1000 is not restricted. For example, the Equipment Management System 1000 can be implemented as a cloud server on the internet. Alternatively, the Equipment Management System 1000 can be implemented as an on-premise system within a company.

[0068] The equipment management system 1000 comprises an equipment management unit 1010, a database 1020, and an inventory processing unit 1030. The equipment management unit 1010 manages the location and quantity of equipment stored or used using the database 1020. The equipment management unit 1010 also plans the use, movement, and purchase of equipment according to the progress and schedule at the site. The inventory processing unit 1030 obtains the actual number of equipment items from the equipment counting device 1 via the network 1090 and uses this number to appropriately correct the data in the database 1020.

[0069] Although one embodiment of the present invention has been described in detail above, the present invention is not limited to the above embodiment, and various improvements and modifications may be made without departing from the spirit of the present invention.

[0070] For example, in each of the above embodiments, the user was asked to input the counting range after the equipment area detection process was performed using a trained model from the image data to be counted, but the order of these two processes may be reversed. Also, in each of the above embodiments, the recognition sensitivity range was given as an example of the operating parameter of the counting processing unit that can be set by the user, but other parameters may also be used. Furthermore, in the above embodiments, a trained model that has been trained for each type of equipment is used, but a trained model that has been trained to recognize multiple types of equipment may also be used.

[0071] Furthermore, although a tablet terminal was used as the hardware constituting the equipment counting device in the above embodiment, VR (Virtual Reality) equipment, AR (Augmented Reality) equipment, MR (Mixed Reality) equipment, etc. may also be used. [Explanation of symbols]

[0072] 1, 1a, 1b... Equipment and materials counting device 10…Camera 20…Display device 30… Tablet 40...Position detection unit 100,100a…Pre-trained models 110...Image data acquisition unit 120...Counting range input section 130...Parameter setting section 140,140b...Counting Processing Unit 141...Parameter storage unit 141a... Recognition sensitivity range 142...Filtering Processing Unit 143... Out-of-bounds counting unit 160...Settings section 161…Image Analysis Department 170... Equipment Dimension Detection Unit 171…Image Analysis Department 200...Storage section 210...Document Creation Department 500… Equipment and supplies 510...Palette 520...Reference object 530... Paper 901...Image data to be counted 902...Counting range 903... Equipment and Materials Area 904... Area of ​​equipment and materials subject to counting 910...Parameter setting area 1000... Equipment Management System 1010... Equipment Management Department 1020…Database 1030...Inventory Processing Unit

Claims

1. A counting device for materials and equipment, A pre-trained machine learning model that takes image data containing equipment as input, individually identifies the regions of equipment in the said image data, and outputs the identified equipment regions. A means for acquiring image data to be counted, which uses a camera to acquire image data of materials and equipment, A counting range input means that outputs the acquired image data to be counted to a display device and allows the user to input the counting range within the image data to be counted using an input device, A counting means that counts the number of equipment and materials regions identified based on image data to be counted using a trained model and included in the input counting range, The system includes output control means for outputting the counted number to a display device and / or for controlling the output of the counting target equipment area superimposed on the counting target image data to the display device. A material and equipment counting device characterized by the following features.

2. The counting means stores a recognition sensitivity range for dimensions within an image of the equipment area to be measured as an operating parameter, and the recognition sensitivity range is configured to be changeable by the user using an input device. The means also includes a filtering means for filtering the equipment area to be counted using the recognition sensitivity range. The equipment counting device according to feature 1.

3. The system includes a recognition sensitivity range setting means that identifies a reference object from counting target image data that includes a known reference object along with the equipment, and sets the recognition sensitivity range based on known dimensional information pre-associated with the identified reference object. The equipment counting device according to claim 2.

4. A means for acquiring setting image data, which uses a camera to acquire setting information carriers that display setting information, and which include setting image data as an image. The system includes a setting means that identifies setting information from acquired setting image data and sets the operating parameters of a counting means based on the identified setting information. The equipment counting device according to feature 1.

5. It has multiple pre-trained models corresponding to the type of equipment and materials. A means for acquiring setting image data, which uses a camera to acquire setting information carriers that display setting information, and which include setting image data as an image. The system includes a setting means that identifies setting information from acquired setting image data and selects a learned setting to be used by the counting means based on the identified setting information. The equipment counting device according to feature 1.

6. It includes a means for detecting the dimensions of equipment and materials within the image of the data to be counted, The counting means includes an out-of-area counting means that counts the number of equipment items located in areas other than the equipment item area in the image data to be counted, based on the dimensions detected by the equipment item dimension detection means. The equipment counting device according to feature 1.

7. The image data acquisition means for counting acquires image data to be counted, and after performing identification processing of the equipment area using the trained model with the image data to be counted as input, the counting range input means inputs the counting range, and after the counting range is input, the counting means performs the counting process. The equipment counting device according to feature 1.

8. The aforementioned trained model is a trained model that is trained without supervision using an instance segmentation algorithm. A material counting device according to any one of claims 1 to 7, characterized by the features described herein.

9. A method for counting the number of materials and equipment using a computer, A step of acquiring image data to be counted, which includes equipment and materials, using a camera, A counting range input step involves outputting the acquired image data to be counted to a display device and allowing the user to input the counting range within the image data using an input device. A counting step is performed using a pre-trained model that takes image data containing equipment as input, individually identifies the areas of equipment in the image data, and outputs the identified equipment areas. The equipment areas identified based on the image data to be counted that fall within the input counting range are designated as the equipment areas to be counted, and the number of these equipment areas is counted. The system includes an output control step that controls the output of the counted number to a display device and / or the output of the counting target equipment area superimposed on the counting target image data to the display device. A method for counting materials and equipment, characterized by the following features.

10. A materials counting program that causes a computer to function as a materials counting device according to claim 1.

Images