Detection device, detection method, and detection program

Abstract

To provide a detection device capable of stably detecting an object even in a dark environment such as at night.SOLUTION: A detection device (10) includes an estimation unit (12) that calculates afterimage trajectory rectangular information and object rectangle information from image information with afterimage using a first learned model, and estimates a position of an object in the image information with afterimage on the basis of the calculated afterimage trajectory rectangular information and object rectangle information, and a detection unit (13) that detects an object on the basis of a result of estimation by the estimation unit (12).SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for detecting an object.

Background Art

[0002] When detecting an object at night using a camera, since the object has to be imaged in a dark environment, the luminance of the image signal output from the camera is adjusted to clearly display the object.

[0003] Specifically, methods for adjusting the luminance include increasing the gain value or increasing the signal accumulation period, which is the period for accumulating image information such as a plurality of frames. When the gain value is increased, the luminance of the image signal increases, but there is a drawback that the noise increases and the visibility deteriorates. On the other hand, when the signal accumulation period is long, the luminance of the image signal can be increased while setting the noise to a value less than the threshold value, and compared with the case of increasing the gain value, the object can be clearly displayed in a dark environment, so detection by image processing becomes possible.

[0004] As a technique for accumulating images of a plurality of frames, Patent Document 1 discloses a technique for switching a tracking mode for tracking a specified tracking target to a non-tracking mode when the accumulation magnification indicated by the accumulation magnification information in the accumulation mode in an imaging device (camera) is equal to or greater than a predetermined threshold magnification. That is, Patent Document 1 discloses a technique for predicting that object detection will fail when the signal accumulation period reaches a predetermined length or more and stopping object detection in advance.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, in the prior art that assumes a long signal accumulation period to increase brightness, such as the technology described in Patent Document 1, if the signal accumulation period is too long, when the object to be detected moves, there is a problem that a ghost image along the movement trajectory of the object is generated and the object cannot be detected.

[0007] Hereinafter, with reference to FIG. 9, the above-described problem will be described in more detail. FIG. 9 shows an image diagram of a ghosting phenomenon output when imaging a moving object, a subject, when the signal accumulation period is lengthened. The left side of FIG. 9 is an image diagram showing video information in a normal mode where the signal accumulation period is one frame, and the right side is an image diagram showing video information in an accumulation mode where an image signal with a signal accumulation period of four frames is accumulated.

[0008] As shown in FIG. 9, when the signal accumulation period is one frame, the image information is updated every frame. On the other hand, when the signal accumulation period is four frames, the image information is updated every four frames. In this case, since the fourth frame is output in a state where the images of the first to third frames overlap, a ghost image is generated. When such a ghost image occurs, it becomes difficult to extract feature amounts by image processing, and there is a problem that object detection fails.

[0009] One aspect of the present invention aims to provide a detection device and related technologies capable of stably detecting an object even in a dark environment such as at night.

Means for Solving the Problem

[0010] The detection device according to one aspect of the present invention uses teacher data in which learning image information with afterimages including an object and an afterimage of the object is associated with learning afterimage trajectory rectangle information including the trajectory of the afterimage of the object and learning object rectangle information including the object, and is machine-learned. When the image information with afterimages is input, a first learned model that outputs the afterimage trajectory rectangle information and the object rectangle information is used to calculate the afterimage trajectory rectangle information and the object rectangle information from the image information with afterimages. Based on the calculated afterimage trajectory rectangle information and the object rectangle information, estimation means for estimating the position of the object in the image information with afterimages, and detection means for detecting the object based on the estimation result by the estimation means.

[0011] The detection method according to one aspect of the present invention includes at least one processor using teacher data in which learning image information with afterimages including an object and an afterimage of the object is associated with learning afterimage trajectory rectangle information including the trajectory of the afterimage of the object and learning object rectangle information including the object, and is machine-learned. When the image information with afterimages is input, a first learned model that outputs the afterimage trajectory rectangle information and the object rectangle information is used to calculate the afterimage trajectory rectangle information and the object rectangle information from the image information with afterimages. Based on the calculated afterimage trajectory rectangle information and the object rectangle information, estimating the position of the object in the image information with afterimages, and detecting the object based on the estimation result of the estimation.

[0012] A detection program according to one aspect of the present invention causes a computer to perform machine learning using teacher data in which image information with a learning afterimage including an object and an afterimage of the object is associated with learning afterimage trajectory rectangle information including a trajectory of the afterimage of the object and learning object rectangle information including the object, and when the image information with an afterimage is input, uses a first learned model that outputs the afterimage trajectory rectangle information and the object rectangle information to calculate the afterimage trajectory rectangle information and the object rectangle information from the image information with an afterimage, and based on the calculated afterimage trajectory rectangle information and the object rectangle information, includes an estimation means for estimating the position of the object in the image information with an afterimage, and a detection means for detecting the object based on the estimation result by the estimation means.

Advantages of the Invention

[0013] According to one aspect of the present invention, it is possible to provide a detection device and related technologies capable of stably detecting an object even in a dark environment such as at night.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Mode for Carrying Out the Invention

[0015] 〔Exemplary Embodiment 1〕 Exemplary Embodiment 1 of the present invention will be described in detail with reference to the drawings. This embodiment is a basic form of the embodiments described later.

[0016] (Configuration of Detection Device 10) The configuration of the detection device 10 according to this embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the configuration of the detection device 10. The detection device 10 detects an object. The object is not particularly limited as long as it is an object that is imaged as a moving subject and can be included in the image information, and examples thereof include a person and a vehicle. Examples of the image information include video information including one or more frames. As shown in FIG. 1, the detection device 10 includes an estimation unit (estimation means) 12 and a detection unit (detection means) 13.

[0017] The estimation unit 12 uses a first learned model to calculate residual trajectory rectangle information including the trajectory of the residual image of the object and object rectangle information including the object from the image information with residual image including the object and the residual image of the object. Further, the estimation unit 12 estimates the position of the object in the image information with residual image based on the calculated residual trajectory rectangle information and object rectangle information. Here, the first learned model is machine-learned using teacher data in which learning residual image-containing image information including an object and the residual image of the object is associated with learning residual trajectory rectangle information including the trajectory of the residual image of the object and learning object rectangle information including the object. Further, the first learned model outputs residual trajectory rectangle information and object rectangle information when image information with residual image is input. The detection unit 13 detects an object based on the estimation result by the estimation unit 12.

[0018] (Flow of Detection Method S1) The flow of the detection method S1 according to the present exemplary embodiment will be described with reference to FIG. 2. FIG. 2 is a flowchart showing the flow of the detection method S1. The detection method is a method for detecting an object. As shown in FIG. 2, the detection method S1 includes steps S11 and S12.

[0019] Hereinafter, the case where the detection method S1 is executed by each part of the detection device 10 will be described as an example. However, the detection method S1 may be executed by at least one processor. The at least one processor may be provided in the detection device 10, may be provided in another device, or may be provided in a plurality of devices.

[0020] (Step S11) In step S11, the estimation unit 12 uses the first learned model to calculate afterimage trajectory rectangle information including the trajectory of the afterimage of the object and object rectangle information including the object from the afterimage-containing image information including the object and the afterimage of the object. Further, the estimation unit 12 estimates the position of the object in the afterimage-containing image information based on the calculated afterimage trajectory rectangle information and object rectangle information. For example, the estimation unit 12 inputs one or more pieces of afterimage-containing image information into the first learned model, and estimates the position of the object in the afterimage-containing image information based on the positions of the afterimage trajectory rectangle information and object rectangle information calculated using the first learned model. In this case, the estimation unit 12 may estimate that the position of the region where the afterimage trajectory rectangle information and the object rectangle information overlap is the position of the object.

[0021] The first pre-trained model is not particularly limited as long as it is a model capable of performing machine learning on the above-described image information and rectangular information. Examples of the first pre-trained model include a model for performing deep learning, a model using a classification method such as logistic regression, support vector machine, random forest, and neighborhood method, and a model using a Bayesian network. The afterimage trajectory rectangular information and the afterimage of the object in the afterimage trajectory rectangular information are usually located at the start point or the end point of the afterimage of the object in the object rectangular information, and occur when the signal accumulation period, which is the time for accumulating the image signals of a plurality of frames, is equal to or greater than a threshold value. Therefore, it can be said that the afterimage trajectory rectangular information and the object rectangular information usually include the afterimage of the object that occurs when the signal accumulation period, which is the time for accumulating the image signals of a plurality of frames, is equal to or greater than a threshold value.

[0022] The object included in the learning afterimage trajectory rectangular information and the learning object rectangular information, which are teacher data for performing machine learning on the first pre-trained model, is not particularly limited. However, since the rectangular information changes depending on the height and width of the object, it is better to have a larger number of object types. For example, even if the object is a car, the rectangular information varies greatly depending on the car model because the outer shape is different. In this case, in the learning afterimage trajectory information and the learning object rectangular information, the car may be imaged as an object for each type of car above a predetermined number.

[0023] (An example of step S11) Hereinafter, an example of step S11 will be described with reference to FIG. 3. FIG. 3 shows an image diagram of the image information and the rectangular information.

[0024] First, as shown in the horizontal view in FIG. 3, a case where an object M moves horizontally and an afterimage is generated will be described. The "horizontal direction" means the horizontal direction when viewed from the direction facing FIG. 3 and is also the direction horizontal to the optical axis of the lens of the camera that acquires the afterimage-containing image information I1. In the example shown in FIG. 3, the object rectangle information i1b to i3b represents the end points of the trajectory of the afterimage of the object M, and the object M is an automobile. Here, the afterimage-containing image information I1 is the afterimage-containing image information input to the first learned model, and a case will be described where the afterimage trajectory rectangle information i1a and the object rectangle information i1b including the object M are the afterimage trajectory rectangle information and the object rectangle information output from the first learned model. In this case, the estimation unit 12 inputs the afterimage-containing image information I1 to the first learned model. The estimation unit 12 calculates the afterimage trajectory rectangle information i1a and the object rectangle information i1b from the afterimage-containing image information I1 using the first learned model. Further, the estimation unit 12 estimates that the position corresponding to the region where these pieces of information overlap, based on at least one of the coordinates and sizes of the calculated afterimage trajectory rectangle information i1a and object rectangle information i1b, is the position of the object M in the afterimage-containing image information I1. The coordinates of the afterimage trajectory rectangle information i1a and the object rectangle information i1b indicate, for example, the coordinates of the positions of the dotted lines corresponding to the outer frames of these pieces of information shown in FIG. 3.

[0025] Next, as shown in the vertical view in FIG. 3, a case where the object M moves in the vertical direction and an afterimage is generated will be described. The "vertical direction" means the vertical direction when viewed from the direction facing FIG. 3, and is also the direction perpendicular to the optical axis of the lens of the camera that acquires the image information I2 with an afterimage. Here, the image information I2 with an afterimage is the image information with an afterimage input to the first learned model, and a case where the afterimage trajectory rectangle information i2a and the object rectangle information i2b including the object M are the afterimage trajectory rectangle information and the object rectangle information output from the first learned model will be described. In this case, the estimation unit 12 inputs the image information I2 with an afterimage to the first learned model. The estimation unit 12 calculates the afterimage trajectory rectangle information i2a and the object rectangle information i2b from the image information I2 with an afterimage using the first learned model. Further, the estimation unit 12 estimates that the position corresponding to the region where these pieces of information overlap is the position of the object M in the image information I2 with an afterimage, based on at least one of the coordinates and sizes of the calculated afterimage trajectory rectangle information i2a and object rectangle information i2b.

[0026] Next, as shown in the diagonal-direction figure in FIG. 3, a case where the object M moves in the diagonal direction and an afterimage is generated will be described. The "diagonal direction" means a diagonal direction when viewed from the direction facing FIG. 3 and a direction perpendicular to the optical axis of the lens of the camera that acquires the afterimage-containing image information I3. Here, it is assumed that the afterimage-containing image information I3 is the afterimage-containing image information input to the first learned model, and the afterimage trajectory rectangular information i3a is the afterimage-containing image information input to the first learned model. Also, it is assumed that the afterimage trajectory rectangular information i3a and the object rectangular information i3b including the object M are the afterimage trajectory information and the object rectangular information output from the first learned model. In this case, the estimation unit 12 inputs the afterimage-containing image information I3 to the first learned model. Then, the estimation unit 12 uses the first learned model to calculate the afterimage trajectory rectangular information i3a and the object rectangular information i3b from the afterimage-containing image information I3. Further, the estimation unit 12 estimates that the position corresponding to the region where these pieces of information overlap, based on at least one of the coordinates and sizes of the calculated afterimage trajectory rectangular information i3a and the object rectangular information i3b, is the position of the object M in the afterimage-containing image information I3.

[0027] (Step S12) In step S12, the detection unit 13 detects an object based on the estimation result by the estimation unit 12.

[0028] (An example of step S12) Hereinafter, an example of step S12 will be described with reference to FIG. 3.

[0029] First, as shown in the horizontal-direction figure in FIG. 3, a case where the object M moves in the horizontal direction and an afterimage is generated will be described. The detection unit 13 detects the object M from the estimated position of the object M corresponding to the region where the afterimage trajectory rectangular information i1a and the object rectangular information i1b overlap.

[0030] Next, as shown in the vertical view in FIG. 3, a case where the object M moves in the vertical direction and an afterimage is generated will be described. The detection unit 13 detects the object M from the estimated position of the object M corresponding to the region where the afterimage trajectory rectangle information i2a and the object rectangle information i2b overlap.

[0031] Next, as shown in the diagonal view in FIG. 3, a case where the object M moves in the diagonal direction and an afterimage is generated will be described. The detection unit 13 detects the object M from the estimated position of the object M corresponding to the region where the afterimage trajectory rectangle information i3a and the object rectangle information i3b overlap.

[0032] (Detection Program) When the detection device 10 is configured by a computer, the functions of the detection device 10 can also be realized by the following detection program stored in the memory referred to by the computer. The detection program causes the computer to function as the estimation unit 12 and the detection unit 13.

[0033] (Effects of Exemplary Embodiment 1) In this exemplary embodiment, the estimation unit 12 estimates the position of the object based on the afterimage trajectory rectangle information and the object rectangle information calculated using the first learned model, and the detection unit 13 detects the object based on the estimation result by the estimation unit 12.

[0034] According to this configuration, the estimation unit 12 can estimate the position of the object in the afterimage-containing image information based on the afterimage trajectory rectangle information and the object rectangle information calculated using the first learned model even when the signal accumulation time becomes long in a dark environment such as at night and an afterimage is generated in the image information. Further, the detection unit 13 can detect the object based on the position of the object estimated by the estimation unit 12. Therefore, according to this exemplary embodiment, an effect is obtained that a detection device 10 and related technologies capable of stably performing object detection even in a dark environment such as at night can be provided.

[0035] In addition, this configuration does not use an infrared cut filter according to the surrounding darkness. Therefore, in this exemplary embodiment, in order to detect an object in a dark environment, the insertion and extraction of the infrared cut filter are repeated, resulting in a problem that the switching between black-and-white image information and color image information occurs and the image is difficult to view. Also, in this exemplary embodiment, it is difficult to adjust the brightness so that object detection is possible only by switching the infrared cut filter, and there is no problem that object detection may become difficult when it gets dark. From the above, according to this exemplary embodiment, the effect that color image information can be used and an object can be detected even in a dark environment can be obtained.

[0036] 〔Exemplary Embodiment 2〕 The exemplary embodiment 2 of the present invention will be described in detail with reference to the drawings. Components having the same functions as those described in the exemplary embodiment 1 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0037] (Configuration of Detection Device 10X) The configuration of the detection device 10X according to this exemplary embodiment will be described with reference to FIG. 4. FIG. 4 is a block diagram showing the configuration of the detection device 10X. The detection device 10X detects an object. As shown in FIG. 4, the detection device 10X includes a camera (imaging means) 14, a control unit (control means) 15, a storage unit (storage means) 17, and a display unit (display means) 18.

[0038] (1) Configuration of Camera 14 The camera 14 images an object to be detected, which is a subject. Examples of the camera 14 include a swivel camera including a lens unit and an imaging unit (imaging means).

[0039] (2) Configuration of Control Unit 15 The control unit 15 controls each part of the detection device 10X such as the camera 14, the storage unit 17, and the display unit 18. As shown in FIG. 4, the control unit 15 includes a learning unit (learning means) 11X, an estimation unit (estimation means) 12X, a detection unit (detection means) 13, and a calculation unit (calculation means) 16.

[0040] The learning unit 11X machine-learns a first learned model using teacher data in which learning afterimage-present image information including an object is associated with learning afterimage trajectory rectangle information and learning object rectangle information. Further, the learning unit 11X further machine-learns a second learned model using teacher data in which learning afterimage-absent image information including an object is associated with learning object rectangle information. The second learned model outputs object rectangle information when afterimage-absent image information is input from the camera 14.

[0041] Thus, when image information is input to the second learned model, the output rectangle information is different from when image information is input to the first learned model, and is only one rectangle information, i.e., object rectangle information, and the afterimage trajectory rectangle information is not output. The learning unit Ⅰ1X also functions as an image information input unit that receives input of image information and a rectangle information input unit that receives input of rectangle information, and may generate teacher data by associating the received learning image information and rectangle information. Further, in the above example, a single learning unit 11X machine-learns two learned models, i.e., a first learned model and a second learned model, but the learning unit that machine-learns the first learned model and the learning unit that machine-learns the second learned model may be separate.

[0042] The estimation unit 12X further uses the second learned model to calculate object rectangle information from the afterimage-absent image information, and estimates the position of the object in the afterimage-absent image information based on the calculated object rectangle information. The control unit 15 determines whether to change the orientation of the camera 14 that captures the object based on whether the estimation unit 12X can estimate the position of the object using the second learned model, and controls the orientation of the camera 14. The calculation unit 16 calculates the moving speed of the object based on the afterimage trajectory rectangle information and the object rectangle information. Examples of the calculation unit 16 include various ones such as a CPU, a GPU, or digital parallel processing. However, as long as the above-described calculation process can be executed, the calculation unit 16 may use any one and is not particularly limited.

[0043] Each process performed by the control unit 15 can be executed by arithmetic operations using a CPU, GPF, digital sequential circuit, etc., but the means for each process is not particularly limited as long as the above-described processes can be executed. In the above example, the control unit 15 controls the orientation of the camera 14, but may also control the lens of the camera 14 so as to enlarge, reduce, or keep constant the size of the object by zooming. In the above example, the camera 14 exists outside the control unit 15, but may be incorporated into and included inside the control unit 15. Also, in the above example, the learning unit 11X is included inside the control unit 15, but may exist outside the control unit 15.

[0044] (3) Configuration of the storage unit 17 The storage unit 17 stores information regarding settings of the camera 14 such as the orientation of the camera 14, noise, and signal accumulation period, and various data such as the first learned model and the second learned model. The learned models stored in the storage unit 17 may be two or more of these, and in addition to these learned models, the storage unit 17 may store further learned models. That is, the detection device 10X may detect an object using a further learned model.

[0045] (4) Configuration of the display unit 18 The display unit 18 displays the above-described various image information as image information for monitoring.

[0046] (Flow of the detection method S1X) The flow of the detection method S1X according to this exemplary embodiment will be described with reference to FIG. 5. FIG. 5 is a flowchart showing the flow of the detection method S1X. As shown in FIG. 5, the detection method S1X includes steps S101 to S115. Also, in the example shown in FIG. 5, the case where the detection method S1X is executed for each frame will be described. Steps S107 and S110 are the same as step S13 in Exemplary Embodiment 1.

[0047] Hereinafter, the case where the detection method S1X is executed by each part of the detection device 10X will be mainly described. However, the detection method S1X may be executed by at least one processor. The at least one processor may be provided in the detection device 10X, may be provided in another device, or may be provided in a plurality of devices.

[0048] (Step S101) In step S101, the learning unit 11X performs machine learning on the first learned model using the teacher data in which the image with learning afterimage and the learning afterimage trajectory rectangle information and the learning object rectangle information are associated. After the learning phase is completed, the learning unit 11X may store the first learned model in the storage unit 17.

[0049] (Step S102) In step S102, the learning unit 11X performs machine learning on the second learned model using the teacher data in which the image without learning afterimage and the learning object rectangle information are associated. After the learning phase is completed, the learning unit 11X may store the second learned model in the storage unit 17.

[0050] (Step S103) In step S103, the camera 14 captures an image of an object and inputs the image information including the object to the control unit 15. In this case, the control unit 15 also functions as an image information input unit.

[0051] (Step S104) In step S104, the estimation unit 12X further estimates the position of the object using the second learned model in addition to the first learned model. For example, first, the estimation unit 12X inputs one or more afterimage-free image information into the second learned model, and estimates the position of the object in the afterimage-free image information based on the object rectangle information calculated by being output from the second learned model. In this case, the estimation unit 12X may estimate that the position of the region represented by the object rectangle information is the position of the object in the afterimage-free image information based on at least one of the coordinates and the size of the object rectangle information. Next, the estimation unit 12X inputs one or more afterimage-present image information into the first learned model, and estimates the position of the object based on the afterimage trajectory rectangle information and the object rectangle information calculated by being output from the first learned model. In this case, the estimation unit 12X may estimate that the position corresponding to the region where the afterimage trajectory rectangle information and the object rectangle information overlap is the position of the object in the afterimage-free image information based on at least one of the coordinates and the size of the afterimage trajectory rectangle information and the object rectangle information.

[0052] (Step S105) In step S105, the estimation unit 12X determines whether it is possible to estimate the position of the object. If the estimation unit 12X determines that it is possible to estimate the position of the object (Y in step S105), the process proceeds to step S106. If the estimation unit 12X determines that it is impossible to estimate the position of the object (N in step S105), the process proceeds to step S114.

[0053] For example, first, the estimation unit 12X uses the second learned model to perform matching of the object shown in one or more afterimage-free image information input to the second learned model, such as the input frame, and determines whether it is possible to estimate the position of the object based on the object rectangle information. The estimation unit 12X stores the determination result by storing the determination result including the object rectangle information in the storage unit 17. Next, the estimation unit 12X uses the first learned model to perform matching of the object shown in one or more afterimage-present image information input to the first learned model, such as the input frame, and determines whether it is possible to estimate the position of the object based on the afterimage trajectory rectangle information and the object rectangle information. The estimation unit 12X stores the determination result by storing the determination result including the afterimage trajectory rectangle information and the object rectangle information in the storage unit 17.

[0054] When the estimation unit 12X can estimate the position of the object using either the second learned model or the first learned model, that is, when the detection unit 13 can detect the object (Y in step S105), the process proceeds to step S106. When the estimation unit 12X cannot estimate the position of the object using either the second learned model or the first learned model, that is, when the detection unit 13 cannot detect the object (N in step S105), the process proceeds to step S114.

[0055] (Step S106) In step S106, the estimation unit 12X calculates object rectangle information from the non-afterimage image information using the second learned model, and determines whether it is possible to estimate the position of the object in the non-afterimage image information based on the calculated object rectangle information. When the estimation unit 12X can estimate the position of the object using the second learned model (Y in step S106), that is, when the detection unit 13 can detect the object in this case, the process proceeds to step S107. When the estimation unit 12X cannot estimate the position of the object using the second learned model (N in step S106), that is, when the detection unit 13 cannot detect the object in this case and the object can be detected using the first learned model, the process proceeds to step S110.

[0056] (Step S108) In step S108, since the estimation unit 12X estimates the position of the object using the second learned model and the detection unit 13 can detect the object, the control unit 15 continues to control the camera 14 while maintaining the current orientation of the camera 14. That is, the control unit 15 does not perform a pan that swings the camera 14 left and right or a tilt that swings the camera 14 up and down.

[0057] (Step S109) In step S109, the camera 14 acquires the next image information by taking a picture of the next frame and executes each process for detecting the object again.

[0058] (Step S111) In step S111, when the estimation unit 12X cannot estimate the position of the object using the second learned model, the estimation unit 12X estimates the moving direction of the object based on the afterimage trajectory rectangle information and the object rectangle information. For example, the estimation unit 12X estimates the moving direction of the object based on the coordinates of the afterimage trajectory rectangle information and the object rectangle information output from the first learned model when the afterimage-containing image information is input to the first learned model.

[0059] (An example of step S111) Hereinafter, an example of step S111 will be described with reference to FIG. 3. Hereinafter, a case will be described in which the image information I1 to I3 is input to the first learned model, and the afterimage trajectory rectangle information i1a, i2a, and i3a and the object rectangle information i1b, i2b, and i3b are output from the first learned model. In addition, a case where the object M is an automobile will be described.

[0060] First, as shown in the horizontal direction diagram in FIG. 3, a case where the object M moves in the horizontal direction and an afterimage is generated will be described. Since the coordinates of the object rectangle information i1b are overall larger in the X-axis direction indicating the horizontal direction than the coordinates of the afterimage trajectory rectangle information i1a, the estimation unit 12X estimates that the moving direction of the object M is the horizontal direction.

[0061] Next, as shown in the vertical direction diagram in FIG. 3, a case where the object M moves in the vertical direction and an afterimage is generated will be described. Since the coordinates of the object rectangle information i2b are overall smaller in the Y-axis direction indicating the vertical direction than the coordinates of the afterimage trajectory rectangle information i2a, the estimation unit 12X estimates that the moving direction of the object M is the vertical direction.

[0062] Next, as shown in the diagonal direction diagram in FIG. 3, a case where the object M moves in the diagonal direction and an afterimage is generated will be described. Since the coordinates of the object rectangle information i3b are overall larger in the Z-axis direction indicating the diagonal direction than the coordinates of the afterimage trajectory rectangle information i3a, the estimation unit 12X estimates that the moving direction of the object M is the diagonal direction.

[0063] (Step S112) In step S112, the estimation unit 12X estimates the moving speed of the object based on the afterimage trajectory rectangle information and the object rectangle information. For example, the estimation unit 12X estimates that the moving speed of the object calculated by the calculation unit 16 based on at least one of the coordinates and the size of the afterimage trajectory rectangle information and the object rectangle information is the moving speed of the object.

[0064] (An example of step S112) Hereinafter, an example of step S112 will be described.

[0065] (1) An example of calculating and estimating the moving speed of the object M in the horizontal direction First, as shown in the horizontal direction diagram in FIG. 3, taking the case where the object M is moving in the horizontal direction as an example, an example of calculating and estimating the moving speed V H of the object M in the horizontal direction will be described with reference to FIG. 6. FIG. 6 is an explanatory diagram of the calculation and estimation of the moving speed V H of the object M in the horizontal direction. The contents of the various parameters shown in FIG. 6 are as follows. Also, hereinafter, the case where the camera 14 includes a lens and an image sensor will be described. L: The distance between the object M and the camera 14 f: The focal length of the lens a H : The size of the imaging surface of the image sensor in the horizontal direction A H : The imaging range of the subject by the camera 14 in the horizontal direction b H : The moving amount of the object M on the imaging surface of the image sensor in the horizontal direction B H : The moving amount of the object M imaged by the camera 14 in the horizontal direction Here, A H is also the imaging range of the subject on the plane horizontal to the optical axis of the lens of the camera 14. B H is also the moving amount of the object M projected on the plane horizontal to the optical axis of the lens of the camera 14. As shown in FIG. 3, bH can be represented by the following formula (1). m in the following formula (1) H and O H are obtained by the calculation of the calculation unit 16 based on the coordinates of the afterimage trajectory rectangle information i1a and the object rectangle information i1b output from the first learned model, which are referred to by the estimation unit 12X when estimating the position of the object M.

[0066]

Number

[0067] Here, assuming that the ratio of the movement amount of the object M to the size of the imaging surface of the image sensor in the horizontal direction is x, x can be expressed as in the following formula (2).

[0068]

Number

[0069] Also, the size a of the imaging surface of the image sensor in the horizontal direction H and the imaging range A of the subject by the camera 14 in the horizontal direction H are related as in the following formula (3). And from the following formula (3), the following formula (4) is obtained.

[0070]

Number

[0071] The calculation unit 16 substitutes the above formula (2) into the above formula (4) to derive the following formula (5).

[0072]

Number

[0073] Also, the horizontal movement speed V of the object M in the horizontal direction HUsing the signal accumulation period t of the camera 14, it can be expressed as the following formula (6).

[0074] [Number]

[0075] Therefore, the calculation unit 16 derives the following formula (7) by substituting the above formula (6) into the above formula (5).

[0076] [Number]

[0077] a H Since all of a, L, f, x, and t are known, the calculation unit 16 can calculate the moving speed V of the object M in the horizontal direction by using the above formula (7). Here, regarding the distance L between the object M and the camera 14, the one measured in advance using a laser rangefinder or the like may be used. Also, when the position information indicating the positions of the object M and the camera 14 such as map information, and the set values of the rotation angle and elevation angle of the camera 14 are stored in advance in the storage unit 17, the calculation unit 16 may calculate the above distance L based on these information. H

[0078] The estimation unit 12X estimates that the moving speed V of the object M in the horizontal direction calculated by the calculation unit 16 H is the moving speed V of the object M in the horizontal direction H .

[0079] (2) An example of calculating and estimating the moving speed of the object M in the vertical direction Next, as shown in the vertical view in FIG. 3, taking the case where the object M is moving in the vertical direction as an example, an example of calculating and estimating the moving speed V of the object M in the vertical direction will be described using FIGS. 7 and 8. FIG. 7 shows the moving speed V of the object M in the vertical direction V V ​​It is an explanatory diagram about the calculation and estimation of. FIG. 8 is an enlarged view of a partial region R of FIG. 7. The contents of the various parameters shown in FIG. 7 are as follows. Also, hereinafter, the case where the camera 14 includes a lens and an image sensor will be described. L: The distance between the object M and the camera 14 f: The focal length of the lens a V : The size of the imaging surface of the image sensor in the vertical direction A V : The imaging range of the subject by the camera 14 in the vertical direction b V : The movement amount of the object M on the imaging surface of the image sensor in the vertical direction B V : The movement amount of the object M imaged by the camera 14 in the vertical direction ω: The angular width of the afterimage locus of the object M in the vertical direction θ: The inclination of the camera 14 in the vertical direction Here, A V is also the imaging range of the subject in a plane perpendicular to the optical axis of the lens of the camera 14. B V is also the movement amount of the object M projected onto a plane perpendicular to the optical axis of the lens of the camera 14. As shown in FIG. 3, b V can be expressed by the following formula (8). m in the following formula (8) V and O V are obtained by the calculation unit 16 based on the coordinates of the afterimage locus rectangle information i2a and the object rectangle information i2b output from the first learned model, which are referred to by the estimation unit 12X when estimating the position of the object M.

[0080]

Equation

[0081] The movement amount of the object M actually corresponds to the length of e on the ground G in FIG. 8, but when the object M is imaged by the camera 14, it is projected as the length of Bv in FIG. 7 onto a plane perpendicular to the optical axis of the lens of the camera 14 and the object M.

[0082] Here, assuming that the ratio of the amount of movement of the object M to the size of the imaging surface of the image sensor of the camera 14 is x, x can be expressed as in the following formula (9), similar to the above formula (2).

[0083]

Equation

[0084] Therefore, the calculation unit 16 calculates the amount of movement B of the object M in the vertical direction captured by the camera 14 V can be derived as in the following formula (10), similar to the horizontal direction case.

[0085]

Equation

[0086] Next, a method for the calculation unit 16 to obtain the actual amount of movement e of the object M based on the amount of movement B of the object M captured by the camera 14 will be described. As shown in FIG. 8, the actual amount of movement e of the object M can be expressed by the following formula (11). V Here, using the sine theorem, e1 and e2 can be expressed as in the following formulas (12) and (13), respectively.

[0087]

Equation

[0088] Here, using the sine theorem, e1 and e2 can be expressed as in the following formulas (12) and (13), respectively.

[0089]

Equation

[0090] The angles shown in FIG. 8 can be expressed as in the following formulas (14) to (20), respectively.

[0091]

Equation

[0092] The calculation unit 16 derives the following formulas (21) and (22) by substituting the above formulas (14) to (20) into the above formula (13).

[0093]

Number

[0094] The calculation unit 16 derives the formula of e represented by the following formula (23) by substituting the above formulas (21) and (22) into the above formula (11).

[0095]

Number

[0096] Also, from FIG. 7, ω can be expressed by the following formula.

[0097]

Number

[0098] Also, the moving speed V of the object M in the vertical direction V can be expressed as follows using the signal accumulation time t of the camera 14.

[0099]

Number

[0100] B V 、b V 、f, t, and θ are all known. Therefore, the calculation unit 16 can calculate the moving speed V of the object M in the vertical direction V using the above formulas (10) and (23) to (25).

[0101] The estimation unit 12X estimates that the moving speed V of the object M in the vertical direction calculated by the calculation unit 16 V is the moving speed V of the object M in the vertical direction V and makes such an estimation.

[0102] (3) An Example of Calculating and Estimating the Moving Speed of the Object M in the Diagonal Direction Next, as shown in the diagram of the diagonal direction in FIG. 3, taking the case where the object M is moving in the diagonal direction as an example, an example of calculating and estimating the moving speed V of the object M in the diagonal direction will be described with reference to FIG. 3. D will be described.

[0103] As shown in the diagram of the diagonal direction in FIG. 3, since the diagonal velocity component is composed of the horizontal velocity component and the vertical velocity component, when expressing this relationship by an equation, it becomes as shown in the following equation (26).

[0104]

Equation

[0105] From the above equation (26), V D becomes the following equation (27).

[0106]

Equation

[0107] The calculation unit 16 calculates V H and V V based on the size of the rectangular information and the signal accumulation time in the same manner as the above method, and calculates V D by using those values.

[0108] The estimation unit 12X estimates that the moving speed V of the object M in the diagonal direction calculated by the calculation unit 16 D is the moving speed V of the object M in the diagonal direction D and makes such an estimation.

[0109] (Step S113) In step S113, the control unit 15 changes the orientation of the camera 14 according to the movement direction and movement speed of the object estimated by the estimation unit 12X. In this case, the control unit 15 may perform at least one of panning and tilting to change the orientation of the camera 14.

[0110] Also, like in steps S106, S108, and S113, the control unit 15 determines whether to change the orientation of the camera 14 based on whether the estimation unit 12X can estimate the position of the object using the second learned model, and may control the orientation of the camera 14. Specifically, when the estimation unit 12X can estimate the position of the object using the second learned model, like Y in step S106, the control unit 15 may maintain the orientation of the camera 14, like in step S108. Also, when the estimation unit 12X cannot estimate the position of the object using the second learned model, like N in step S106, the control unit 15 may change the orientation of the camera 14 according to the movement direction and movement speed of the object, similar to the above example in step S113.

[0111] As described above, when the control unit 15 changes the orientation of the camera 14 according to the movement direction and movement speed of the object, the estimation unit 12X may estimate the position of the object using the image information of the object captured by the camera 14 whose orientation has been changed and the second learned model.

[0112] (Step S114) In step S114, when the estimation unit 12X cannot estimate the position of the object using either the first learned model or the second learned model and the detection unit 13 cannot detect the object, the control unit 15 determines whether the value of the gain of the camera 14 is equal to or greater than a threshold value. The threshold value is not particularly limited as long as it is a luminance value at which it is theoretically possible to detect an object at night. Also, the threshold value may be set in advance.

[0113] The control unit 15 determines whether the gain value set in the camera 14 is equal to or greater than a threshold value by reading the gain value set in the camera 14 from the camera 14. When the control unit 15 determines that the gain value of the camera 14 is equal to or greater than the threshold value (Y in step S114), the process proceeds to step S115. When the control unit 15 determines that the gain value of the camera 14 is less than the threshold value (N in step S114), that is, when the control unit 15 determines that the gain value of the camera 14 is less than the threshold value, the process proceeds to step S109.

[0114] As described above, when the estimation unit 12X cannot estimate the position of the object and determines that the gain value is less than the threshold value, the estimation unit 12X may estimate that the object is not imaged in the afterimage-containing image information and the afterimage-free image information. Further, the control unit 15 may cause the camera 14 to capture the next frame to acquire the next image information.

[0115] (Step S115) In step S115, when the estimation unit 12X cannot estimate the position of the object and the gain value of the camera 14 is equal to or greater than the threshold value, the control unit 15 may set the gain value to be equal to or less than the threshold value and change the signal accumulation period according to the gain value set to be equal to or less than the threshold value. As described above, the control unit 15 determines whether to change the gain value and the signal accumulation period, which is the period for accumulating a plurality of pieces of afterimage-containing image information or afterimage-free image information, based on the gain value of the camera 14, thereby controlling the gain value and the signal accumulation period.

[0116] (Effect of Exemplary Embodiment 2) In this exemplary embodiment, the estimation unit 12X further uses the second learned model to calculate object rectangle information from the afterimage-free image information, and estimates the position of the object in the afterimage-free image information based on the calculated object rectangle information.

[0117] According to this configuration, the estimation unit 12X estimates the position of an object using two types of learned models, namely, the first learned model and the second learned model. Therefore, according to this exemplary embodiment, an effect is obtained in that the detection unit 13 can stably detect an object even in a dark environment such as at night, as compared with Exemplary Embodiment 1.

[0118] In this exemplary embodiment, a configuration is adopted in which a control unit 15 is further provided to determine whether to change the orientation of the camera 14 that images an object based on whether the estimation unit 12X can estimate the position of the object using the second learned model.

[0119] According to this configuration, when the estimation unit 12X can estimate the position of the object using the second learned model for reasons such as the image information not including an afterimage, the control unit 15 maintains the orientation of the camera 14. Also, when the image information includes an afterimage and the estimation unit 12X cannot estimate the position of the object using the second learned model, the control unit 15 changes the orientation of the camera 14. Therefore, according to this exemplary embodiment, in addition to the effect exhibited by Exemplary Embodiment 1, an effect is obtained in that even when the estimation unit 12X cannot estimate the position of the object using the second learned model, the orientation of the camera 14 can be changed to track the object.

[0120] In this exemplary embodiment, a configuration is adopted in which the estimation unit 12X estimates the moving direction and moving speed of the object based on the afterimage trajectory rectangle information and the object rectangle information, and the control unit 15 changes the orientation of the camera 14 according to the moving direction and moving speed of the object.

[0121] According to this configuration, even when the image information includes an afterimage and the estimation unit 12X cannot estimate the position of the object using the second learned model, the control unit 15 can change the orientation of the camera 14 at the same moving direction and moving speed as the object in accordance with the moving direction and moving speed of the object. Therefore, according to this exemplary embodiment, in addition to the effect exhibited by Exemplary Embodiment 1, an effect is obtained in that even in the above-described case, the camera 14 can be made to follow the object.

[0122] In the present exemplary embodiment, the estimation unit 12X is configured to estimate the position of an object using the image information of the object captured by the camera 14 whose orientation has been changed according to the moving direction and speed of the object and a second learned model.

[0123] According to this configuration, since the object is imaged by the camera 14 that follows the object, it is easy to obtain image information in which afterimages are less likely to occur, and the estimation unit 12X estimates the position of the object using the rectangular information output by inputting the image information in which afterimages are less likely to occur into the second learned model. Therefore, according to the present exemplary embodiment, in addition to the effects achieved by the exemplary embodiment 1, an effect that it becomes easier to estimate the position of the object using the second learned model can be obtained.

[0124] In the present exemplary embodiment, the control unit 15 is configured to determine whether to change the gain value and the signal accumulation period, which is a period for accumulating a plurality of afterimage-containing image information or afterimage-free image information, based on the gain value of the camera 14.

[0125] According to this configuration, when the gain value is large and the noise in the image information is large, the gain value can be decreased to reduce the noise, and the signal accumulation period can be changed to maintain the luminance. Therefore, according to the present exemplary embodiment, in addition to the effects achieved by the exemplary embodiment 1, even when the gain value is large, the estimation unit 12X can estimate the position of the object using image information suitable for detecting the object in which the noise is reduced while the luminance is maintained.

[0126] In the present exemplary embodiment, when the estimation unit 12X cannot estimate the position of the object and the gain value of the camera 14 is equal to or greater than the threshold value, the control unit 15 is configured to set the gain value to be equal to or less than the threshold value and change the signal accumulation period so as to correspond to the gain value set to be equal to or less than the threshold value.

[0127] According to this configuration, when the gain value is equal to or greater than the threshold value and the noise in the image information is large, the gain value is decreased to reduce the noise, and the signal accumulation period is changed according to the decreased gain value to maintain the luminance. Therefore, according to the present exemplary embodiment, in addition to the effect achieved by the first exemplary embodiment, even when the gain value is equal to or greater than the threshold value, the estimation unit 12X can estimate the position of the object using the image information in which the noise is reduced while the luminance is maintained, and the object is suitable for detection.

[0128] In the present exemplary embodiment, when the estimation unit 12X cannot estimate the position of the object and the gain value is less than the threshold value, the estimation unit 12X estimates that no object is imaged in the afterimage-containing image information and the afterimage-free image information, and the control unit 15 is configured to cause the camera 14 to acquire the next image information.

[0129] According to this configuration, when the estimation unit 12X cannot estimate the position of the object and the noise in the image information is small, i.e., the gain value is less than the threshold value, the estimation unit 12X estimates that the factor preventing the detection unit 13 from detecting the object is not due to noise and that no object is imaged in the image information. Therefore, according to the present exemplary embodiment, in addition to the effect achieved by the first exemplary embodiment, the estimation unit 12X can estimate whether or not an object is imaged in the image information according to whether or not the gain value is less than the threshold value, such as at night when the gain value tends to increase. Further, the control unit 15 can determine whether or not to proceed to the next imaging by the camera 14 according to whether or not the gain value is less than the threshold value.

[0130] 〔Example of Realization by Software〕 The control blocks of the detection devices 10 and 10X may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

[0131] In the latter case, the detection devices 10 and 10X include a computer that executes instructions of a program, which is software for realizing each function. This computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium that stores the above program. Then, in the above computer, when the above processor reads and executes the above program from the above recording medium, the object of the present invention is achieved. As the above processor, for example, a CPU (Central Processing Unit) can be used. As the above recording medium, "non-transitory tangible media" such as ROM (Read Only Memory), tapes, disks, cards, semiconductor memories, programmable logic circuits, etc. can be used. Further, it may further include a RAM (Random Access Memory) etc. for expanding the above program. Also, the above program may be supplied to the above computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. One aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above program is embodied by electronic transmission.

[0132] 〔Supplementary Note 1〕 The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0133] 〔Supplementary Note 2〕 Some or all of the above-described embodiments can also be described as follows. However, the present invention is not limited to the aspects described below.

[0134] (Supplementary Note 1) Using teacher data in which learning image information with an afterimage including an object and an afterimage of the object is associated with learning afterimage trajectory rectangle information including the trajectory of the afterimage of the object and learning object rectangle information including the object, a first learned model that outputs the afterimage trajectory rectangle information and the object rectangle information when the image information with an afterimage is input is used to calculate the afterimage trajectory rectangle information and the object rectangle information from the image information with an afterimage, and based on the calculated afterimage trajectory rectangle information and the object rectangle information, estimation means for estimating the position of the object in the image information with an afterimage, detection means for detecting the object based on the estimation result by the estimation means, A detection device comprising:

[0135] (Appendix 2) The estimation means further uses a second learned model that outputs the object rectangle information when image information without an afterimage, which has been machine-learned using teacher data in which the learning image information without an afterimage including the object is associated with the learning object rectangle information, is input, calculates the object rectangle information from the image information without an afterimage, and based on the calculated object rectangle information, estimates the position of the object in the image information without an afterimage. The detection device according to Appendix 1.

[0136] (Appendix 3) The detection device according to Appendix 2, further comprising control means for determining whether or not to change the orientation of the imaging means for imaging the object based on whether or not the estimation means can estimate the position of the object using the second learned model.

[0137] (Appendix 4) The estimation means estimates the moving direction and moving speed of the object based on the afterimage trajectory rectangle information and the object rectangle information, The control means changes the orientation of the imaging means according to the moving direction and moving speed of the object. The detection device according to Appendix 3.

[0138] (Appendix 5) The detection device according to supplementary note 4, wherein the estimation means estimates the position of the object using the image information of the object captured by the imaging means whose orientation has been changed according to the moving direction and speed of the object, and the second learned model.

[0139] (Supplementary note 6) The detection device according to any one of supplementary notes 1 to 5, wherein the control means determines whether to change the value of the gain and the signal accumulation period, which is the period for accumulating a plurality of the afterimage-containing image information or the non-afterimage image information, based on the value of the gain of the imaging means.

[0140] (Supplementary note 7) When the estimation means cannot estimate the position of the object and the value of the gain of the imaging means is equal to or greater than the threshold value, the control means sets the value of the gain to be equal to or less than the threshold value, and changes the signal accumulation period so as to correspond to the value of the gain set to be equal to or less than the threshold value. The detection device according to supplementary note 6.

[0141] (Supplementary note 8) When the estimation means cannot estimate the position of the object and the gain is less than the threshold value, the estimation means estimates that the object is not imaged in the afterimage-containing image information and the non-afterimage image information, and the control means causes the imaging means to acquire the next image information. The detection device according to supplementary note 7.

[0142] (Supplementary note 9) At least one processor Calculating the afterimage trajectory rectangle information and the object rectangle information from the afterimage-containing image information using a first learned model that outputs the afterimage trajectory rectangle information and the object rectangle information when the afterimage-containing image information, which has been machine-learned using teacher data in which the learning afterimage-containing image information including the object and the afterimage of the object is associated with the learning afterimage trajectory rectangle information including the trajectory of the afterimage of the object and the learning object rectangle information including the object, is input, and estimating the position of the object in the afterimage-containing image information based on the calculated afterimage trajectory rectangle information and the object rectangle information; Detecting the object based on the estimation result by the estimation; A detection method including

[0143] (Appendix 10) Causing a computer to calculate the afterimage trajectory rectangular information and the object rectangular information from the image information with afterimage including the object and the afterimage of the object, using a first learned model that outputs the afterimage trajectory rectangular information and the object rectangular information when the image information with afterimage is input, which is machine-learned using teacher data in which the afterimage trajectory rectangular information including the trajectory of the afterimage of the object and the object rectangular information including the object are associated; estimating the position of the object in the image information with afterimage based on the calculated afterimage trajectory rectangular information and the object rectangular information; and a detection means for detecting the object based on the estimation result by the estimation means; A detection program for causing it to function.

[0144] 〔Supplementary Note 3〕 A detection device including at least one processor, the processor calculating the afterimage trajectory rectangular information and the object rectangular information from the image information with afterimage including the object and the afterimage of the object, using a first learned model that outputs the afterimage trajectory rectangular information and the object rectangular information when the image information with afterimage is input, which is machine-learned using teacher data in which the afterimage trajectory rectangular information including the trajectory of the afterimage of the object and the object rectangular information including the object are associated; estimating the position of the object in the image information with afterimage based on the calculated afterimage trajectory rectangular information and the object rectangular information; and detecting the object based on the estimation result by the estimation.

[0145] This detection device may further include a memory. In this memory, there is teacher data in which image information with afterimages including an object and an afterimage of the object is associated with afterimage trajectory rectangle information including the trajectory of the afterimage of the object and object rectangle information including the object. Using a first trained model that outputs the afterimage trajectory rectangle information and the object rectangle information when image information with afterimages is input, which is trained using the teacher data, the afterimage trajectory rectangle information and the object rectangle information are calculated from the image information with afterimages. Based on the calculated afterimage trajectory rectangle information and the object rectangle information, the position of the object in the image information with afterimages is estimated, and based on the estimation result of the estimation, the object is detected. A detection program for causing the processor to execute the above may be stored. This detection program may be recorded on a non-transitory tangible recording medium readable by a computer.

[0146] [Supplementary Note 4] The present invention can provide a detection device and related technologies that can stably detect an object even in a dark environment such as at night. Therefore, the present invention can contribute to the achievement of Sustainable Development Goal (SDG) 9, "Build the infrastructure for industry and innovation."

Explanation of Reference Numerals

[0147] 10, 10X Detection device 12, 12X Estimation unit (estimation means) 13 Detection unit (detection means) 14 Camera (imaging means) 15 Control unit (control means) I1, I2, I3 Image information i1a, i2a, i3a Afterimage trajectory rectangle information i1b, i2b, i3b Object rectangle information S1, S1X Detection method

Claims

1. Estimation means for calculating the afterimage trajectory rectangular information and the object rectangular information from the image information with afterimage, and estimating the position of the object in the image information with afterimage based on the calculated afterimage trajectory rectangular information and object rectangular information, using a first learned model that is machine-learned using teacher data in which image information with afterimage including an object and an afterimage of the object is associated with afterimage trajectory rectangular information including the trajectory of the afterimage of the object and object rectangular information including the object; Detection means for detecting the object based on the estimation result by the estimation means; A detection device comprising:

2. The detection device according to claim 1, wherein the estimation means further uses a second learned model that is machine-learned using teacher data in which image information without afterimage including the object is associated with the object rectangular information, and outputs the object rectangular information when image information without afterimage is input, calculates the object rectangular information from the image information without afterimage, and estimates the position of the object in the image information without afterimage based on the calculated object rectangular information.

3. The detection device according to claim 2, further comprising control means for determining whether to change the orientation of the imaging means for imaging the object based on whether the estimation means can estimate the position of the object using the second learned model.

4. The estimation means estimates the moving direction and moving speed of the object based on the afterimage trajectory rectangular information and the object rectangular information; The detection device according to claim 3, wherein the control means changes the orientation of the imaging means according to the moving direction and moving speed of the object.

5. The detection device according to claim 4, wherein the estimation means estimates the position of the object using the image information obtained by imaging the object with the imaging means whose orientation has been changed according to the moving direction and moving speed of the object, and the second learned model.

6. When the control means cannot estimate the position of the object using either the first learned model or the second learned model, and the detection means cannot detect the object, the control means determines whether to change the value of the gain of the imaging means and the signal accumulation period, which is the period for accumulating a plurality of the afterimage-containing image information or the non-afterimage image information, based on the value of the gain. The detection device according to claim 5.

7. When the estimation means cannot estimate the position of the object and the value of the gain of the imaging means is equal to or greater than a threshold value, the control means sets the value of the gain to be equal to or less than the threshold value, and changes the signal accumulation period so as to correspond to the value of the gain set to be equal to or less than the threshold value. The detection device according to claim 6.

8. When the estimation means cannot estimate the position of the object and the gain is less than the threshold value, the estimation means estimates that the object is not imaged in the afterimage-containing image information and the non-afterimage image information, and the control means causes the imaging means to acquire the next image information. The detection device according to claim 7.

9. At least one processor using teacher data in which learning afterimage-containing image information including an object and an afterimage of the object is associated with learning afterimage trajectory rectangle information including a trajectory of the afterimage of the object and learning object rectangle information including the object, and calculating the afterimage trajectory rectangle information and the object rectangle information from the afterimage-containing image information using a first learned model that outputs the afterimage trajectory rectangle information and the object rectangle information when the afterimage-containing image information is input, and estimating the position of the object in the afterimage-containing image information based on the calculated afterimage trajectory rectangle information and the object rectangle information; detecting the object based on the estimation result of the estimation; A detection method comprising:

10. A computer Using teacher data in which learning image information with an afterimage including an object and an afterimage of the object is associated with learning afterimage trajectory rectangle information including the trajectory of the afterimage of the object and learning object rectangle information including the object, the first learned model that outputs the afterimage trajectory rectangle information and the object rectangle information when the image information with an afterimage is input is used to calculate the afterimage trajectory rectangle information and the object rectangle information from the image information with an afterimage, and based on the calculated afterimage trajectory rectangle information and the object rectangle information, estimation means for estimating the position of the object in the image information with an afterimage, detection means for detecting the object based on the estimation result by the estimation means, A detection program that causes it to function.

Images