Object individual recognition method and program, and object individual recognition device

Abstract

To grasp a state of a construction site from a video image of a wide area, and to estimate a position of an object while individually recognizing the object even when the objects are of a same type.SOLUTION: An object individual recognition device 4 includes estimated position input means 41, predicted position input means 42, conversion means 43, associating means 44, and granting means 45. The estimated position input means 41 receives the estimated position of an object 9 detected in an estimation image captured by a fixed camera 2. The predicted position input means 42 receives a predicted position of the object obtained by a position information acquisition device 6 attached to the object. The conversion means 43 converts the estimated position and the predicted position into positions in a common plane coordinate system. The associating means 44 associates the object with the position information acquisition device 6 having the predicted position closest to the estimated position of the object. The grating means 45 gives, to the object, identification information held by the position information acquisition device 6 associated with the object.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an object individual recognition method and program, and an object individual recognition device. [Background technology]

[0002] A technology has been proposed for determining the positions of people and construction machinery at construction sites for dam construction and large-scale development works through image recognition using a trained object detection model (see Patent Document 1). This technology uses images captured by fixed cameras covering as wide an area as possible of the construction site to determine the positions of people, construction machinery, etc. This makes it possible to grasp the state of the construction site using images of a wide area captured by fixed cameras, and to estimate the positions of objects while distinguishing between different types of objects. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-026281 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the technology of Patent Document 1, it is possible to distinguish between types of objects, such as a person, an excavator, and a bulldozer. However, it is difficult to distinguish between objects that look the same, such as when using the same model of excavator as Unit 1 and Unit 2. The present invention aims to solve the above-mentioned problems, grasp the state of a construction site using video footage captured over a wide area, and estimate the position of each object while recognizing each object individually, even if the objects are of the same type. [Means for solving the problem]

[0005] In order to solve the above problem, the present invention provides a method for detecting a construction site by using a fixed camera, the method comprising: N pieces (where N is a natural number greater than or equal to 2) The object individual recognition method includes an estimated position input step, a predicted position input step, a conversion step, a correlation step, and an assignment step. In the estimated position input step, estimated position coordinates of each object detected in the estimation video obtained by estimating the position of the object are input. In the predicted position input step, predicted position coordinates of each object obtained by predicting the position of the object by a position information acquisition device attached to each object are input. The conversion step converts the estimated position coordinates and the predicted position coordinates into positions in a common plane coordinate system. The correlation step converts the estimated position coordinates and the predicted position coordinates into positions in a common plane coordinate system. In the correlation step, estimated position coordinates of each object detected in the estimation video obtained by estimating the position of the object are input. N before chronicle body One of For each object, the estimated position coordinates and each of the N predicted position coordinates, and the difference is most Become smaller Associating the location acquisition device with the predicted location coordinates. The operation is performed on N of the objects. The assigning step assigns, to each of the objects detected in the estimation video, identification information held by the position information acquisition device and associated with the object. In the present invention, identification information held by a position information acquisition device attached to an object is assigned to the object. Therefore, even if objects are of the same type, they can be individually identified by the identification information assigned to each object. In addition, the movement of construction machinery and the working status, for example, can be visually confirmed by the image of a wide area captured by a fixed camera. Therefore, according to the present invention, the state of a construction site can be grasped by the image captured over a wide area, and the position of each object can be estimated while individually recognizing the object even if the object type is the same. In the method for individual object recognition, it is preferable to estimate the position of each object detected in the estimation video using a trained object detection model. In this configuration, it is possible to more accurately estimate the position of each object while distinguishing the type of object.

[0006] The object individual recognition method includes the steps of: each The identification information assigned to the object is attached to the object. each It is preferable that the method further includes a display step of displaying the estimation image including the object on a display device. In this configuration, the manager can grasp the state of the construction site and recognize the object individually by simply looking at the estimation image displayed on the display device. In the object individual recognition method, each The object is preferably a construction machine. In this configuration, even if the same model of excavator is used as the first and second machines, the first and second machines can be identified by the identification information assigned to each construction machine. In other words, even if the construction machines are the same model and look exactly the same, the construction machines can be individually recognized and their positions can be estimated.

[0007] The present invention can also be realized as a program that causes a computer to execute the object individual recognition method, using hardware resources such as a CPU, memory, and hard disk provided in the computer.

[0008] In addition, the present invention relates to a method for detecting a construction site using a fixed camera, the method comprising: N pieces (where N is a natural number greater than or equal to 2) The object individual recognition device is provided with an estimated position input means, a predicted position input means, a conversion means, a correlation means, and an assignment means. The estimated position input means receives an estimated position coordinate of each object detected in the estimation video, the estimated position coordinate being obtained by estimating the position of the object. The predicted position input means receives a predicted position coordinate of each object, the predicted position coordinate being obtained by predicting the position of the object by a position information acquisition device attached to each object. The conversion means converts the estimated position coordinate and the predicted position coordinate into positions in a common plane coordinate system. The correlation means receives an estimated position coordinate of each object detected in the estimation video, the predicted position coordinate being obtained by predicting the position of the object by a position information acquisition device attached to the ... N before chronicle body One of For each object, the estimated position coordinates and each of the N predicted position coordinates, and the difference is most Become smaller Associating the location acquisition device with the predicted location coordinates. The operation is performed on N of the objects. The assigning means assigns, to each of the objects detected in the estimation video, identification information held by the position information acquisition device and associated with the respective objects. Effect of the Invention

[0009] According to the present invention, the state of a construction site can be grasped by using images captured over a wide area, and even if the objects are of the same type, the objects can be individually recognized and their positions can be estimated. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of an object management system according to an embodiment. [Diagram 2] 1 is a block diagram showing a configuration of an object management system according to an embodiment. [Diagram 3] 1 is an explanatory diagram illustrating a method for extracting frame images from video captured by a fixed camera in an embodiment. [Figure 4] FIG. 2 is an explanatory diagram illustrating a method for generating teacher data in an embodiment. [Diagram 5] 4A to 4C are explanatory diagrams for explaining a method for detecting an object type and an object region in an embodiment. [Figure 6] FIG. 4 is an explanatory diagram for explaining the coordinates of four corners of an object region in the embodiment. [Figure 7] In the embodiment, (a) is an explanatory diagram for explaining an image coordinate system, and (b) is an explanatory diagram for explaining a world coordinate system. [Figure 8] FIG. 4 is an explanatory diagram for explaining a method for setting reference coordinates in the embodiment. [Figure 9] 5 is a flowchart showing an operation of a learning mode of the object position estimation device in the embodiment. [Figure 10] 5 is a flowchart showing an operation of a position estimation mode of the object position estimation device in the embodiment. [Figure 11]5 is a flowchart showing an operation of an individual recognition mode of the object individual recognition device in the embodiment. [Figure 12] FIG. 1 is a diagram illustrating an example of an estimation image captured by a fixed camera. [Figure 13] 13 is a diagram showing an estimated position of a construction machine included in the estimation image shown in FIG. 12 in a plane coordinate system. FIG. [Figure 14] This figure adds predicted positions of the construction machines obtained by the position information acquisition device to FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings shown below, the same or similar components are given the same reference numerals, and duplicated descriptions are omitted. In addition, the size and shape of the components may be deformed or exaggerated to be schematic for the convenience of explanation.

[0012] [Overall configuration of the object management system] The overall configuration of an object management system 1 according to an embodiment will be described with reference to FIG. As shown in FIG. 1, an object management system 1 estimates the position of an object 9 using an image captured by a fixed camera 2 of a construction site L such as a dam construction site, and individually recognizes the object 9. The object management system 1 includes the fixed camera 2, an object management device 5, and a position information acquisition device 6. Here, objects 9 to be managed by the object management system 1 include construction machinery (e.g., a vibrating roller 91, a bulldozer 92, and an excavator 93), a worker 94, and materials 95. The position information acquisition device 6 is attached to the object 9. Specifically, the position information acquisition device 6 is placed at a predetermined location near the driver's seat of the construction machinery, carried by the worker 94, or attached to the materials 95.

[0013] The object management system 1 has three operation modes: a "learning mode" for learning an object detection model, a "position estimation mode" for estimating the position of the object 9 using the object detection model, and an "individual recognition mode" for individually recognizing the object 9. Thus, the object management system 1 learns the object detection model in advance in the learning mode, and then actually estimates the position of the object 9 in the position estimation mode and individually recognizes the object 9 in the individual recognition mode. An administrator of the object management system 1 commands the object management device 5 to select either the learning mode, or the position estimation mode and the individual recognition mode. Hereinafter, the image captured by the fixed camera 2 in the learning mode, that is, the image used for learning the object detection model, will be referred to as the "learning image." Also, the image captured by the fixed camera 2 in the position estimation mode, that is, the image used for estimating the position of the object 9, will be referred to as the "estimation image."

[0014] The fixed camera 2 is a general camera or web camera that photographs the construction site L and outputs the photographed image of the construction site L to the object management device 5. The fixed camera 2 has a fixed position, attitude, and photographing angle of view. In other words, the fixed camera 2 photographs the learning image and the estimation image at the same position, attitude, and photographing angle of view. For example, one fixed camera 2 is installed in a position that overlooks the entire construction site L and can clearly photograph the objects 9 within the construction site L.

[0015] For example, a smartphone may be used as the location information acquisition device 6. The location information acquisition device 6 has a function of receiving radio waves emitted from an artificial satellite (GPS satellite), measuring (predicting) a current location, and transmitting the predicted location to the object management device 5 via a communication line. The location information acquisition device 6 also has a function of holding predetermined identification information set by an administrator or the like in a storage unit, and transmitting this identification information to the object management device 5 via a communication line. The location information acquisition device 6 is not limited to one that uses GPS positioning, and may be one that uses other positioning techniques, such as Wi-Fi positioning or LPWA (Low Power Wide Area) base station positioning. The location information acquisition device 6 is not limited to a smartphone, and may be another mobile terminal or a dedicated device for positioning.

[0016] [Configuration of the object management device] 2, the object management device 5 includes an object position estimation device 3 and an object individual recognition device 4. Similarly to a typical laptop computer, the object management device 5 also includes an input device 51 such as a keyboard so that an administrator of the object management system 1 can make various settings, and a display device 52 such as a liquid crystal display.

[0017] [Configuration of object position estimation device] The configuration of the object position estimation device 3 will be described with reference to FIG. The object position estimation device 3 estimates the type and position of an object 9 included in an estimation video captured by the fixed camera 2 of the construction site L, using the learned object detection model. The object position estimation device 3 includes a learning image input means 30, a teacher data generation means 31, a deep learning means 32, a memory means 33, an estimation image input means 34, an object detection means 35, an affine projection means (transformation matrix calculation means) 36, and an object position estimation means 37.

[0018] <Learning mode> In the learning mode, the learning image input means 30 receives learning images from the fixed camera 2. The learning image input means 30 has an extraction rate, which will be described later, preset according to the moving speed of the object 9. Then, as shown in FIG. 3, the learning image input means 30 outputs the learning image M based on the extraction rate. A Frame image F A The extracted frame image F A is output to the teacher data generating means 31. The extraction rate is the learning video M A Frame image F A The extraction rate represents the rate (interval) at which the object 9 is extracted, and is set manually by the administrator of the object management system 1. Here, the extraction rate should be set higher as the moving speed of the object 9 increases, and the extraction rate should be set lower as the moving speed of the object 9 decreases. For example, when a learning video M is extracted at 30 fps (frames per second) consisting of 30 frame images per second, A In this case, one frame image F is selected from 30 frame images. A (i.e., Extraction rate = 1 / 30).

[0019] In the learning mode, the teacher data generating means 31 generates the learning image M A And learning video M A In other words, the teacher data generating means 31 annotates (tags to tell the correct answer) the types of objects 9 included in the learning video M input from the learning video input means 30 as shown in FIG. A Frame image F A And learning video M A Frame image F A The teacher data is generated by associating the tag T with the type of the object 9 included in the frame image F. Here, the correct tag T is manually input by the administrator of the object management system 1. In the example of FIG. A The tag T of "vibration roller" is associated with the vibration roller 91 of the frame image F. AThe vehicle identification number "T1234" that can uniquely identify the vibrating roller 91 may be associated with the vibrating roller 91 as a tag T. Thereafter, the teacher data generation means 31 outputs the generated teacher data to the deep learning means 32.

[0020] In the learning mode, the deep learning means 32 receives training data from the training data generating means 31 and generates a trained object detection model from the training data by deep learning. Then, the deep learning means 32 writes the generated object detection model into the storage means 33. Here, the deep learning means 32 is not particularly limited in the deep learning method. For example, it is preferable that the deep learning means 32 uses YOLO (You Only Look Once) as deep learning, which is easy to learn and has fast detection. In addition, the deep learning means 32 may use deep learning such as SSD (Single Shot MultiBox Detector), Faster R-CNN (Regional Convolution Neural Network), Fast R-CNN, R-CNN, etc. The storage means 33 is a storage device such as a memory, a hard disk drive (HDD), or a solid state drive (SSD) that stores the object detection model generated by the deep learning means 32. The object detection model stored in the storage means 33 is referenced by the object detection means 35 described later.

[0021] <Location Estimation Mode> In the position estimation mode, estimation video is input to estimation video input means 34 from fixed camera 2. Note that estimation video input means 34 is similar to learning video input means 30 except for extracting frame images from the estimation video, so further explanation will be omitted. Then, estimation video input means 34 outputs the frame images extracted from the estimation video to object detection means 35.

[0022] The object detection means 35 detects the type and object area of ​​the object 9 included in the estimation video input from the estimation video input means 34, using an object detection model stored in the storage means 33. That is, the object detection means 35 detects the type and object area of ​​the object 9 included in the estimation video input from the estimation video input means 34, using an object detection model stored in the storage means 33. B is input to the object detection model, and the frame image F B The type and object area of ​​the object 9 in the frame image F in FIG. B In the example, "T1234" indicating a vibrating roller 91 and "PERSON" indicating a worker 94 are detected as the types of the object 9. Here, the object detection model detects not only the type of object 9 but also an object region A, as shown in FIG. B The area that each object 9 occupies is the coordinates P Gi (X Gi ,Y Gi ) (where i=1,...,4). The object area A of the vibration roller 91 is located at the top left coordinate P G1 (X G1 ,Y G1 ), lower left coordinate P G2 (X G2 ,Y G2 ), lower right coordinate P G3 (X G3 ,Y G3 ), and the upper right coordinate P G4 (X G4 ,Y G4 ) The image coordinate system is a two-dimensional coordinate system of an image (frame image) captured by a fixed camera 2, as shown in FIG. Thereafter, the object detection means 35 detects the type of the detected object 9 and the coordinates P Gi (X Gi ,Y Gi ) to the affine projection means 36.

[0023] The affine projection means 36 calculates the coordinates P of the four corners of the object area A in the image coordinate system. Gi (X Gi ,Y Gi) to the world coordinate system of the construction site L. The world coordinate system of the construction site L is a three-dimensional coordinate system that represents the construction site L, as shown in Fig. 7(b), and is also called the local coordinate system. In order to affinely project the image coordinate system to the world coordinate system, the coordinates of the four corners of the object region A in the image coordinate system P Gi (X Gi ,Y Gi ), and the reference coordinates P of the four points in the world coordinate system Li (X Li ,Y Li ) are required. As mentioned above, the coordinates of the four corners P Gi (X Gi ,Y Gi ) are input from the object detection means 35 and are therefore known. Li (X Li ,Y Li ) must be set.

[0024] First, the reference coordinate P Li (X Li ,Y Li As shown in Fig. 8, the construction site L has a reference coordinate P Li (X Li ,Y Li Four reference positions are provided as the basis for measuring the distance (P) between the object and the ground. The reference positions may be of any type and position as long as they can be captured in the measurement video. For example, the reference positions may be posts or pillars in the construction site L, or markers placed to indicate the reference positions. Then, the manager of the object management system 1 determines the four reference positions P as shown in FIG. L1 (X L1 ,Y L1 )~P L4 (X L4 ,Y L4 ) at the construction site L, and the measurement result is set in the affine projection means 36. Since the construction site L is at the same height, the height direction (Z L axis) is omitted. Next, the affine projection means 36 calculates the coordinates P of the four corners of the object area A as shown in the following equation (1).Gi (X Gi ,Y Gi ), and the reference coordinates P of the four points Li (X Li ,Y Li ) is used to calculate the transformation matrix k for performing the affine projection. Note that in equation (1), the subscript T indicates the transposition of the matrix.

[0025]

number

[0026] Thereafter, the affine projection means 36 calculates the transformation matrix k calculated, the type of the object 9 detected by the object detection means 35, and the coordinates P of the four corners of the object region A. Gi (X Gi ,Y Gi ) to the object position estimation means 37. The transformation matrix k is determined based on the position, orientation, or angle of view of the fixed camera 2, and the reference coordinate P Li (X Li ,Y Li ) remains constant unless its position changes. Therefore, the affine projection means 36 does not need to calculate the transformation matrix k for all frame images extracted from the estimation video by the estimation video input means 34. For example, the affine projection means 36 may calculate the transformation matrix k for the frame image extracted first by the estimation video input means 34, or may calculate the transformation matrix k at regular time intervals. This allows the object position estimation device 3 to reduce the amount of calculation and increase the processing speed.

[0027] The object position estimation means 37 estimates the position of the object 9 by transforming any coordinate within the object region A into the position of the object 9 in the world coordinate system using the transformation matrix k input from the affine projection means 36. In the following, any coordinate within the object region A in the image coordinate system is P G (X G ,Y G ) and the position of object 9 in the world coordinate system is denoted as P L (X L ,Y LFor example, the object position estimation means 37 may be expressed as follows: G (X G ,Y G ) and calculate the lower middle coordinate of the object region A input from the affine projection means 36 using the following equation (2). As shown in FIG. 6, the lower middle coordinate P G (X G ,Y G ) is the bottom left coordinate P of the object area A G2 (X G2 ,Y G2 ) to the bottom right coordinate P G3 (X G3 ,Y G3 ), so it is expressed by equation (2). In this way, any coordinate P G (X G ,Y G ) and determining the position where the object 9 touches the ground, the position of the object 9 can be accurately estimated.

[0028]

number

[0029] Furthermore, the object position estimation means 37 calculates the lower middle coordinate P of the object region A in the image coordinate system using the following equation (3): G (X G ,Y G ) is the position P of object 9 in the world coordinate system. L (X L ,Y L ) The object position estimation means 37 then converts the obtained position P L (X L ,Y L ) as the estimated position. The object position estimation means 37 also outputs the type of the object 9 input from the affine projection means .

[0030]

number

[0031] The object position estimation means 37 detects an arbitrary coordinate P G (X G ,Y G ) in the object region A, but the present invention is not limited to this. For example, the object position estimation means 37 may calculate an arbitrary coordinate P G (X G ,Y G ) to calculate the intermediate coordinates of the upper side, the left side, or the right side of the object region A. G (X G ,Y G ) to calculate the center coordinates and center of gravity coordinates of the object region A.

[0032] [Configuration of the object individual recognition device] The configuration of the object individual recognition device 4 will be described with reference to FIG. The object individual recognition device 4 uses the object position estimation device 3 and the position information acquisition device 6 to individually recognize the object 9 . The object individual recognition device 4 includes an estimated position input means 41 , a predicted position input means 42 , a conversion means 43 , an association means 44 , and an assignment means 45 .

[0033] <Individual recognition mode> In the individual recognition mode, an estimated position of the object 9 obtained by estimating the position of the object 9 detected in the estimation video is input to the estimated position input means 41. The type of the object 9 is also input to the estimated position input means 41 from the object position estimation means 37 together with the estimated position of the object 9. The predicted position input means 42 receives a predicted position (measured position) of the object 9 obtained by predicting (measuring) the position of the object 9 using a position information acquisition device 6 attached to the object 9. The estimated position input means 41 receives, from the position information acquisition device 6, the predicted position of the object 9 as well as identification information held by the position information acquisition device 6. The conversion means 43 converts the estimated position sent from the estimated position input means 41 and the predicted position sent from the predicted position input means 42 into positions in a common plane coordinate system. Here, the common plane coordinate system is a plane coordinate system in the world coordinate system of the construction site L, which is a plane coordinate system in the height direction (Z L It is a plane coordinate system in which the axial (coordinate) axes are omitted.

[0034] The associating means 44 associates the position information acquisition device 6 having the predicted position closest to the estimated position of the object 9 with the object 9 detected in the estimation video. For example, if there are N objects 9 at the construction site L, there are also N position information acquisition devices 6 attached to the objects 9. In this case, N estimated positions and N predicted positions of the object 9 are acquired. Then, one of the N objects 9 is selected, and the difference (distance in a plane coordinate system) between the estimated position of the selected object 9 and each of the acquired N predicted positions is calculated. The position information acquisition device 6 having the predicted position with the smallest difference from the estimated position of the selected object 9 among the N predicted positions is associated with the selected object 9. This procedure is sequentially repeated for the N objects 9, whereby the position information acquisition device 6 having the predicted position closest to the estimated position of the object 9 is associated with the N objects 9. The assigning means 45 assigns, to an object 9 detected in the estimation video, identification information held by the position information acquisition device 6 and associated with the object 9. Information indicating the association between the object 9 and the position information acquisition device 6 is sent from the associating means 44. In addition, the identification information held by the position information acquisition device 6 is sent from the predicted position input means 42. The display device 52 can display various images, such as an estimation image including the object 9 with the identification information given to the object 9.

[0035] [Operation of object position estimation device: learning mode] The operation of the object position estimation device 3 in the learning mode will be described with reference to FIG. 9, in step S1, a learning video is input to the learning video input means 30 from the fixed camera 2. Then, the learning video input means 30 extracts frame images from the learning video based on the extraction rate. In step S2, the teacher data generating means 31 generates teacher data that associates the frame images of the learning video extracted in step S1 with the types of objects 9 included in the frame images of the learning video. In step S3, the deep learning means 32 generates an object detection model from the teacher data generated in step S2 by deep learning such as YOLO. Then, the deep learning means 32 writes the generated object detection model to the storage means 33.

[0036] [Operation of the object position estimation device: Position estimation mode] The operation of the object position estimation device 3 in the position estimation mode will be described with reference to FIG. 10, in step S10, the estimation video input means 34 receives an estimation video from the fixed camera 2. The estimation video input means 34 then extracts frame images from the estimation video based on the extraction rate. In step S11, the object detection means 35 detects the type of object 9 and the object region A included in the estimation video using the object detection model stored in the storage means 33. That is, the object detection means 35 inputs the frame image of the estimation video extracted in step S10 to the object detection model, and obtains the type of object 9 and the object region A in the frame image.

[0037] In step S12, the affine projection means 36 calculates the coordinates P Gi (X Gi ,Y Gi ) to the world coordinate system. Specifically, the affine projection means 36 calculates a transformation matrix for affine projecting the four corner coordinates P Gi (X Gi ,Y Gi ), and the reference coordinates P of the four points Li (X Li ,YLi ) to calculate the transformation matrix k for performing affine projection.

[0038] In step S13, the object position estimation means 37 calculates an arbitrary coordinate P G (X G ,Y G ) is the position P of object 9 in the world coordinate system L (X L ,Y L ) to estimate the position of the object 9. Specifically, the object position estimation means 37 estimates the position of the object 9 by converting an arbitrary coordinate P G (X G ,Y G ) and calculates the lower middle coordinate of the object region A using the above-mentioned equation (2). Then, the object position estimation means 37 calculates the lower middle coordinate P G (X G ,Y G ) is the position P of object 9 in the world coordinate system. L (X L ,Y L )

[0039] [Operation of the object individual recognition device: individual recognition mode] The operation of the object individual recognition device 4 in the individual recognition mode will be described with reference to FIG. 11, in step S20, an estimated position of the object 9 obtained by estimating the position of the object 9 detected in the estimation video is input to the estimated position input means 41. In addition, the type of the object 9 is also input to the estimated position input means 41. Fig. 12 is a diagram showing an example of an estimation image captured by a fixed camera. In the example of Fig. 12, one bulldozer 92 and three excavators 93 are detected in the estimation image.

[0040] In step S21, the predicted position input means 42 receives the predicted position of the object 9 obtained by predicting the position of the object 9 using the position information acquisition device 6 attached to the object 9. In addition, the predicted position input means 42 also receives the identification information held by the position information acquisition device 6.

[0041] In step S22, the conversion means 43 converts the estimated position sent from the estimated position input means 41 and the predicted position sent from the predicted position input means 42 into positions in a common plane coordinate system. Fig. 13 is a diagram showing, in a plane coordinate system, the estimated position of the construction machine included in the estimation image shown in Fig. 12. Fig. 14 is a diagram in which the predicted position of the construction machine obtained by the position information acquisition device 6 is added to Fig. 13.

[0042] In step S23, the associating means 44 associates, with an object 9 detected in the estimation image, the position information acquisition device 6 having a predicted position closest to the estimated position of the object 9. In the example of Fig. 14, four pairs of objects 9 and position information acquisition devices 6 that are associated with each other are indicated by large-diameter circular dashed lines. In step S24, the assigning means 45 assigns, to the object 9 detected in the estimation video, identification information held by the position information acquisition device 6 and associated with the object 9. The identification information is, for example, ID=001 to 004 as shown in Fig. 14, but can be changed and set as appropriate.

[0043] [effect] As described above, in this embodiment, the identification information held by the position information acquisition device 6 attached to the object 9 is assigned to the object 9. Therefore, even if the objects 9 are of the same type, they can be individually identified by the identification information assigned to each object 9. In addition, the image of a wide area captured by the fixed camera 2 allows visual confirmation of, for example, the movement of construction machinery and the work status. Therefore, according to this embodiment, the state of the construction site can be grasped by the image captured over a wide area, and even if the objects 9 are of the same type, the objects 9 can be individually recognized and their positions can be estimated. Furthermore, in this embodiment, the position of the object 9 detected in the estimation video is estimated using a trained object detection model, so that the type of object 9 can be distinguished and its position can be estimated more accurately.

[0044] In the present embodiment, an estimation video including the object 9 with the identification information given to the object 9 may be displayed on the display device 52. Therefore, the manager can grasp the state of the construction site and individually recognize the object 9 simply by looking at the estimation video displayed on the display device 52. Note that various images including the object 9 with the identification information, such as an image shown in a planar coordinate system as shown in FIG. 13, may be displayed on the display device 52. Furthermore, in addition to the identification information, information indicating the type of the object 9 may also be displayed. Furthermore, the object 9 may be a construction machine. In this configuration, even if the same model of excavator 93 is used as the first and second machines, the first and second machines can be identified by the identification information given to each construction machine. In other words, even if the construction machines are the same model and look exactly the same, the construction machines can be individually recognized and their positions can be estimated.

[0045] Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and can be appropriately modified without departing from the spirit of the present invention. For example, in the above embodiment, the objects to be managed are described as construction machines, workers, and materials, but the objects are not limited to this. In the above embodiment, the construction site is photographed by one fixed camera, but two or more fixed cameras may be used. For example, when the construction site is rectangular, two fixed cameras are installed on the diagonal extension of the construction site, and the position of an object is estimated using the estimation video photographed by each fixed camera. For example, even if an object is occluded in the estimation video photographed by one fixed camera, the position of the object can be estimated from the estimation video photographed by the other fixed camera. Furthermore, when the position of the same object is estimated from the estimation video photographed by both fixed cameras, the two position estimation results may be integrated (for example, averaged).

[0046] In the above embodiment, the object position estimation device and the object individual recognition device are described as independent hardware, but the present invention is not limited to this. For example, the present invention can be realized by a program that operates hardware resources such as a CPU, memory, and hard disk of a computer as the object position estimation device and the object individual recognition device. This program may be distributed via a communication line, or may be written on a recording medium such as a CD-ROM or a flash memory and distributed. [Explanation of symbols]

[0047] 1. Object Management System 2 Fixed Camera 3 Object position estimation device 4 Individual object recognition device 41 Estimated position input means 42 Predicted position input means 43 Conversion methods 44 Linking Means 45 Means of Granting 5 Object management device 52 Display device 6 Location information acquisition device 9 objects L Construction site

Claims

1. A method for individually recognizing N objects (where N is a natural number of 2 or more) included in an estimation image captured by a fixed camera of a construction site together with a reference position whose coordinates are known, comprising: an estimated position input step of inputting estimated position coordinates of each of the objects detected in the estimation video, the estimated position coordinates being obtained by estimating the position of each of the objects; a predicted position input step of inputting predicted position coordinates of each object obtained by predicting the position of each object using a position information acquisition device attached to each object; a transformation step of transforming the estimated position coordinates and the predicted position coordinates into positions in a common planar coordinate system; a correlation step of calculating, for one of the N objects detected in the estimation video, a difference between the estimated position coordinates of the object and each of the N predicted position coordinates, and correlating the position information acquisition device having the predicted position coordinates with which the difference is smallest, for the N objects; an assignment step of assigning, to each of the objects detected in the estimation video, identification information held by the position information acquisition device and associated with each of the objects; The object individual recognition method includes:

2. 2. The method according to claim 1, further comprising estimating the position of each of the objects detected in the estimation image using a trained object detection model.

3. The method for individual object recognition according to claim 1 or 2, further comprising a display step of displaying, on a display device, the estimation image including each of the objects to which the identification information assigned to each of the objects has been added.

4. 4. The method for identifying individual objects according to claim 1, wherein each of the objects is a construction machine.

5. A program causing a computer to execute the object individual recognition method according to any one of claims 1 to 4.

6. An object individual recognition device that individually recognizes N objects (where N is a natural number equal to or greater than 2) included in an estimation image captured by a fixed camera of a construction site together with a reference position whose coordinates are known, an estimated position input means for inputting estimated position coordinates of each of the objects detected in the estimation image, the estimated position coordinates being obtained by estimating the position of each of the objects detected in the estimation image; a predicted position input means for inputting predicted position coordinates of each object obtained by predicting the position of each object using a position information acquisition device attached to each of the objects; a conversion means for converting the estimated position coordinates and the predicted position coordinates into positions in a common plane coordinate system; an association means for calculating, for one of the N objects detected in the estimation video, a difference between the estimated position coordinates of the object and each of the N predicted position coordinates, and associating the position information acquisition device having the predicted position coordinates that have the smallest difference with respect to the N objects; an assignment means for assigning, to each of the objects detected in the estimation video, identification information held by the position information acquisition device and associated with each of the objects; An object individual recognition device comprising:

Images