Tracking device, tracking method, and program

Abstract

The present invention provides a tracking device that includes: a person tracking unit that tracks, in an image, a tracking-target person; a moving body detection unit that detects a moving body which has executed an operation for satisfying a temporal condition based on the timing at which the tracking-target person was lost sight of and a positional condition based on the position at which the tracking-target person was lost sight of; and a moving body tracking unit that tracks the detected moving body.

Description

Tracking device, tracking method, and recording medium

[0001] The present invention relates to a tracking device, a tracking method, and a program.

[0002] Techniques related to the present invention are disclosed in Patent Documents 1 and 2.

[0003] The technology disclosed in Patent Document 1, while tracking a person using image analysis, starts tracking the vehicle when it detects that the person being tracked and the vehicle satisfy a predetermined condition, that is, when the person enters the vehicle or when the person gets on top of the vehicle.

[0004] The technology disclosed in Patent Document 2 detects a person being tracked by image analysis, and if the person being tracked is detected to have entered an elevator, the technology analyzes images from a camera at the elevator's designated floor to detect the person being tracked. This technology detects that the person being tracked has entered an elevator based on the direction of movement of the person being tracked, etc.

[0005] International Publication No. WO 2020 / 194584 International Publication No. WO 2022 / 029860

[0006] A person being tracked by image analysis may start moving using a moving object. If the person gets on the moving object, it may be difficult to detect the person in the image. For this reason, as with the technology disclosed in Patent Document 1, it is preferable to start tracking the moving object when the person being tracked starts moving using the moving object.

[0007] The present inventors have found the following problem in the technology that starts tracking a moving object in response to the person being tracked starting to move using the moving object.

[0008] When a scene in which a person being tracked gets into a moving body is captured in an image, it is possible to identify the moving body in which the person being tracked is riding, for example, using the technology disclosed in Patent Document 1, and start tracking of the moving body. However, there may be cases in which the scene in which the person being tracked gets into the moving body is not captured in the image. For example, there may be cases in which the person being tracked gets into the moving body in a location that is out of the camera's blind spot. In such cases, there is a need for a technology that can estimate the moving body in which the person being tracked is riding and start tracking of the moving body.

[0009] The technology disclosed in Patent Document 1 is based on the premise that a scene in which a person being tracked gets on a moving object is captured in the image, and if a scene in which the person being tracked gets on a moving object is not captured in the image, the technology disclosed in Patent Document 1 cannot estimate the moving object on which the person being tracked is riding.

[0010] The technology disclosed in Patent Document 2 is specialized for elevators and lacks versatility.

[0011] In view of the above-mentioned problems, one example of the objective of the present invention is to provide a tracking device, a tracking method, and a program that can estimate the moving body that a person being tracked is riding on even if the scene of the person being tracked getting on the moving body is not captured in the image.

[0012] According to one aspect of the present invention, there is provided a tracking device having: a person tracking means for tracking a person to be tracked within an image; a moving object detection means for detecting a moving object that has performed an action that satisfies a temporal condition based on the timing at which the person to be tracked was lost and a positional condition based on the position at which the person to be tracked was lost; and a moving object tracking means for tracking the detected moving object.

[0013] According to one aspect of the present invention, there is provided a tracking method in which one or more computers track a person to be tracked within an image, detect a moving object that performs an action that satisfies a temporal condition based on the timing at which the person to be tracked was lost and a positional condition based on the position at which the person to be tracked was lost, and track the detected moving object.

[0014] According to one aspect of the present invention, there is provided a program that causes a computer to function as: person tracking means that tracks a person to be tracked within an image; moving object detection means that detects a moving object that has performed an action that satisfies a temporal condition based on the timing at which the person to be tracked was lost, and a positional condition based on the position at which the person to be tracked was lost; and moving object tracking means that tracks the detected moving object.

[0015] According to one aspect of the present invention, a tracking device, a tracking method, and a program are realized that can estimate a moving body in which a person being tracked is riding, even if the scene of the person being tracked getting on the moving body is not captured in the image.

[0016] The above-mentioned objects and other objects, features and advantages will become more apparent from the following description of the preferred embodiments and the accompanying drawings.

[0017] FIG. 1 is a diagram showing an example of a functional block diagram of a tracking device. FIG. 2 is a diagram showing an example of a hardware configuration of the tracking device. FIG. 3 is a diagram for explaining an example of a process in which the tracking device detects a predetermined moving body. FIG. 4 is a diagram for explaining another example of a process in which the tracking device detects a predetermined moving body. FIG. 5 is a diagram for explaining another example of a process in which the tracking device detects a predetermined moving body. FIG. 6 is a diagram for explaining another example of a process in which the tracking device detects a predetermined moving body. FIG. 7 is a diagram for explaining another example of a process in which the tracking device detects a predetermined moving body. FIG. 8 is a flowchart showing an example of a processing flow in the tracking device. FIG. 9 is a diagram for explaining another example of a processing in which the tracking device detects a predetermined moving body.

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0019] 1 is a functional block diagram showing an overview of a tracking device 10 according to a first embodiment. The tracking device 10 includes a person tracking unit 11, a moving object detection unit 12, and a moving object tracking unit 13.

[0020] The person tracking unit 11 tracks a tracking target person in an image (moving image). The moving object detection unit 12 detects a moving object that has performed an action that satisfies a time condition based on the timing at which the tracking target person was lost and a positional condition based on the position at which the tracking target person was lost. The moving object tracking unit 13 tracks the moving object detected by the moving object detection unit 12.

[0021] According to the tracking device 10 of this embodiment, a moving body that performs an action that satisfies a temporal condition based on the timing at which the tracked person was lost and a positional condition based on the position at which the tracked person was lost is estimated as the moving body in which the tracked person is riding. According to such a tracking device 10, even if the scene in which the tracked person gets on the moving body is not captured in the image, it is possible to estimate the moving body in which the tracked person is riding. Furthermore, by making an estimation using both the temporal condition and the positional condition, it is possible to accurately estimate the moving body in which the tracked person is riding.

[0022] Second Embodiment "Overview" The tracking device 10 of the second embodiment is a specific implementation of the tracking device 10 of the first embodiment. That is, the tracking device 10 estimates that a moving object that has performed an action that satisfies a temporal condition based on the timing at which the tracking target person was lost and a positional condition based on the position at which the tracking target person was lost is the moving object that the tracking target person is riding on. The tracking device 10 then tracks the moving object that is estimated to be the moving object that the tracking target person is riding on. This will be described in detail below.

[0023] "Hardware Configuration" An example of the hardware configuration of the tracking device 10 will be described. Each functional unit of the tracking device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. Software includes programs that are pre-stored in the device before shipping, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0024] FIG. 2 is a block diagram illustrating the hardware configuration of the tracking device 10. As shown in FIG. 2, the tracking device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The tracking device 10 does not have to have the peripheral circuit 4A. Note that the tracking device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices can have the above hardware configuration.

[0025] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a processing unit such as a CPU or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, touch panel, etc. Examples of output devices include a display, speaker, printer, mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0026] "Functional Configuration" Next, the functional configuration of the tracking device 10 of this embodiment will be described in detail. Fig. 1 shows an example of a functional block diagram of the tracking device 10 of this embodiment. As shown in the figure, the tracking device 10 of this embodiment has a person tracking unit 11, a moving object detection unit 12, and a moving object tracking unit 13.

[0027] The person tracking unit 11 tracks a tracking target person within an image. To achieve this processing, the person tracking unit 11 acquires an image. The person tracking unit 11 also acquires information indicating the appearance features of the tracking target person. Then, the person tracking unit 11 detects the tracking target person within the acquired image based on the appearance features of the tracking target person, and tracks the tracking target person within the image.

[0028] "Image" is a concept that includes moving images.

[0029] "Image acquisition" can be achieved by any means. For example, the tracking device 10 can communicate with one or more cameras (such as surveillance cameras) installed on the road or at street corners. The cameras may then transmit the images they generate to the tracking device 10. Alternatively, the images generated by the cameras may be stored in any storage means. The images stored in the storage means may then be input to the tracking device 10 by manual operation by the user. Input of images to the tracking device 10 may be performed by real-time processing or batch processing. The person tracking unit 11 can acquire images input to the tracking device 10 in this way. Note that the person tracking unit 11 may also acquire images by other means.

[0030] "Acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition), and a device inputting data or information output from another device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a reply, and accessing and reading information from another device or storage medium. An example of passive acquisition is receiving information that is distributed (or transmitted, pushed, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

[0031] The "information indicating the appearance feature quantities of the person to be tracked" is input by the user to the tracking device 10. For example, the user may input the appearance feature quantities of the person to be tracked to the tracking device 10. Alternatively, the user may input an image of the person to be tracked to the tracking device 10. The person tracking unit 11 may then analyze the image and extract the appearance feature quantities of the person to be tracked.

[0032] The "appearance feature amount of the person to be tracked" includes, but is not limited to, facial feature amount, body feature amount, clothing feature amount, belongings feature amount, shoe feature amount, and the like.

[0033] The person tracking unit 11 detects a person to be tracked from images captured by multiple cameras based on the features of the person's appearance. The person tracking unit 11 then tracks the person to be tracked within the images. The detection and tracking of people within images can be achieved using any technology.

[0034] When the person tracking unit 11 loses sight of the person to be tracked, it inputs to the moving body detection unit 12 information indicating the timing at which the person to be tracked was lost and the position at which the person to be tracked was lost.

[0035] "Losing sight of a tracked person" means that a tracked person who was detected in an image and tracked in the image can no longer be detected in the image. A tracked person who was previously detected may be lost due to reasons such as blending into a crowd, entering a building, or entering a blind spot of the camera. While the tracked person is lost, there is a possibility that the tracked person may get on a moving vehicle. If the tracked person gets on a moving vehicle while the tracked person has been lost, the tracking device 10 of this embodiment can estimate the moving vehicle that the tracked person is riding on.

[0036] The "information indicating the timing at which the tracking target person was lost" is indicated by a date and time. For example, the date and time at which the frame image at which the tracking target person was lost is identified based on a timestamp attached to the image.

[0037] The "information indicating the position where the tracked person was lost" can include information indicating the camera that generated the image in which the tracked person was detected immediately before the tracked person was lost, or the installation location of that camera. Also, the information indicating the position where the tracked person was lost can include information indicating the position in the image in which the tracked person was detected immediately before the tracked person was lost.

[0038] When the person tracking unit 11 loses sight of the person to be tracked, it may input the above information to the moving object detection unit 12. Alternatively, the person tracking unit 11 may input the above information to the moving object detection unit 12 when the person to be tracked is not detected from any of the images even after a predetermined time has elapsed since the person tracking unit 11 lost sight of the person to be tracked. The predetermined time is a time that is set in advance.

[0039] The moving object detection unit 12 detects a moving object that has performed an action that satisfies a time condition based on the timing at which the tracking target person was lost and a positional condition based on the position at which the tracking target person was lost. In response to receiving the information from the person tracking unit 11, the moving object detection unit 12 can start the process of detecting the moving object as described above.

[0040] A "mobile body" is an object that a person rides on and moves, and examples include, but are not limited to, vehicles that move on land such as automobiles, motorcycles, buses, taxis, and trains, objects that move on the sea such as ships and boats, and objects that move in the air such as airplanes and helicopters.

[0041] The moving body detection unit 12 can execute at least one of the following detection processing examples 1 to 3.

[0042] "Detection Process Example 1" The "action that satisfies the time condition and the position condition" in Detection Process Example 1 is an action of starting after the timing at which the tracking target person is lost, at a position within a reference distance from the position at which the tracking target person is lost. Note that "after the timing at which the tracking target person is lost" may be replaced with "after the timing at which the tracking target person is lost and before the timing at which a predetermined time has elapsed since the tracking target person was lost." The reference distance is a predetermined distance. The predetermined time is a predetermined period of time.

[0043] The "starting action" is the action of starting to move from a stopped state.

[0044] It is unlikely that the person being tracked got into a moving vehicle that departed before the person being tracked was lost. Also, the longer the time that has passed since the person being tracked was lost, the higher the possibility that the person being tracked has left the location, and the lower the possibility that the person being tracked will get into a moving vehicle at that location. By detecting a moving vehicle that has performed an action that satisfies the above-mentioned time and position conditions, it is possible to detect a moving vehicle that the person being tracked may have gotten into.

[0045] Here, a specific example of the processing of the moving object detection unit 12 will be described. In one example, the moving object detection unit 12 analyzes an image and detects, from the image, a moving object that has performed an action that satisfies a time condition and a position condition. The image to be analyzed is, for example, an image acquired by the person tracking unit 11 (an image from a surveillance camera). The image to be analyzed here is an image generated (taken) after the image in which the tracking target person is lost.

[0046] For example, as shown in FIG. 3 , assume that position D where the tracked person was lost is shown as a position on the image. In this case, the moving object detection unit 12 identifies an area within the image that is within a reference distance L1 from position D where the tracked person was lost. The moving object detection unit 12 then detects a moving object that starts moving within the identified area after the timing at which the tracked person was lost (or after that but before a predetermined time has elapsed since the tracking person was lost). The distance used by the moving object detection unit 12 may be the distance within the image or the actual distance (the same applies below). The moving object detection unit 12 may calculate the actual distance from the distance within the image by any means.

[0047] Another specific example of the processing of the moving body detection unit 12 will be described. For example, as shown in FIG. １ ~C ５ As shown in FIG. 5, the cameras C １ ~C ５ Position information indicating the installation position of each device is registered in advance in the tracking device 10 .

[0048] In this processing example, the position where the tracking target person is lost is shown as the installation position of the camera. In this case, the moving body detection unit 12 detects the camera where the tracking target person is lost (in the example of FIG. 4, camera C １ ) from the installation position to the reference distance L ２ In the example of FIG. 4, the camera C １ , C ２ and C ４ The above three cameras can be identified based on the position information of a plurality of cameras, for example, as shown in FIG.

[0049] Then, the moving body detection unit 12 detects the position of the person to be tracked after the timing at which the person to be tracked is lost (or after the timing at which a predetermined time has elapsed since the person to be tracked was lost), and detects the position of the person to be tracked after the timing at which the person to be tracked is lost). １ , C ２ and C ４ Detects moving objects within the generated image.

[0050] Hereinafter, another specific example of the processing of the moving object detection unit 12 will be described. In another example, the moving object detection unit 12 detects a moving object that has performed an action that satisfies a time condition and a location condition, based on the boarding and alighting positions of public transportation and a timetable.

[0051] The boarding and alighting locations and timetables of public transport are predetermined, and information indicating the boarding and alighting locations of public transport and the timetables is registered in the tracking device 10 in advance.

[0052] An example of public transportation is a bus. In this case, the boarding and disembarking location is a bus stop. Another example of public transportation is a train. In this case, the boarding and disembarking location is a station. Another example of public transportation is a boat. In this case, the boarding and disembarking location is a pier. Another example of public transportation is an airplane. In this case, the boarding and disembarking location is an airport.

[0053] Based on this information, the mobile object detection unit 12 identifies a public transportation boarding / alighting location within a predetermined distance from the location where the tracked person was lost. Then, based on the timetable of the identified boarding / alighting location, the mobile object detection unit 12 detects a mobile object that departs from the boarding / alighting location after the time when the tracked person was lost (or after that but before the predetermined time has elapsed since the time when the tracked person was lost).

[0054] "Detection Processing Example 2" In the case assumed in this example, as shown in FIG. 6, a camera C is installed at a position to capture images of the vicinity of the entrance Ep where people walk into facility F and the vicinity of the exit Ev of the parking lot of facility F. １ and C ２ Camera C is installed. １ and C ２ Therefore, camera C cannot capture the parking lot of facility F. １ and C ２ In the image generated by the tracking device 10, there is no image of a person getting on a moving object in the parking lot. In addition, no camera is installed in the facility F. Alternatively, a camera is installed in the facility F, but the tracking device 10 cannot acquire an image from the camera installed in the facility F.

[0055] The "facility" includes a parking lot. The facility may be a department store, supermarket, amusement park, or the like, or may be other facilities.

[0056] When the person to be tracked gets on a moving body in the parking lot of such a facility, the moving body detection unit 12 can estimate the moving body in which the person to be tracked gets on.

[0057] The "action satisfying the time condition and the positional condition" in detection processing example 2 is an action of exiting a parking lot of a facility that is within a predetermined distance from the location where the tracking target person was lost, after a first predetermined time has elapsed since the tracking target person was lost. Note that "after a first predetermined time has elapsed since the tracking target person was lost" may be replaced with "after a first predetermined time has elapsed since the tracking target person was lost and before a second predetermined time has elapsed." The first predetermined time is the travel time required to travel from the entrance of the facility to the parking lot of the facility. The second predetermined time is a predetermined time. The predetermined distance is a predetermined distance.

[0058] As shown in Fig. 7, information about each of a plurality of facilities is registered in advance in the tracking device 10. The information about the facilities includes location information indicating the location of each facility and information indicating the distance from the entrance of each facility to the parking lot. If each facility has multiple entrances, the distance from each entrance to the parking lot may be indicated for each entrance. The information about the facilities may further include information indicating the location of each entrance and the exit of the parking lot.

[0059] Based on the information about the facility, the mobile object detection unit 12 identifies a facility within a predetermined distance from the location where the tracked person was lost. The mobile object detection unit 12 then calculates the travel time that corresponds to the first predetermined time based on the distance from the entrance of the identified facility to the parking lot and the estimated travel speed of the tracked person. If the identified facility has multiple entrances, the mobile object detection unit 12 identifies the entrance closest to the location where the tracked person was lost. The mobile object detection unit 12 then calculates the travel time based on the distance from the identified entrance to the parking lot. Next, the mobile object detection unit 12 detects a mobile object exiting the parking lot of the identified facility after the first predetermined time (calculated travel time) from the time the tracked person was lost (or after that but before the time when a second predetermined time has elapsed since the tracked person was lost). The image to be analyzed is an image generated by a camera capturing an image near the exit of the parking lot of the identified facility. For example, a camera capturing an image near the exit of the parking lot of the identified facility is identified based on location information indicating the installation locations of each of the multiple cameras, as shown in FIG. 5 .

[0060] Here, the process of determining the estimated moving speed of the person to be tracked will be described.

[0061] The moving object detection unit 12 determines an estimated moving speed of the person to be tracked based on person information about the person to be tracked, and calculates the moving time based on the estimated moving speed. The person information indicates at least one of the age, sex, whether or not the person to be tracked is carrying luggage, the size of the luggage, whether or not the person is injured, and the moving speed up to that point.

[0062] The age, gender, whether or not the person to be tracked has luggage, the size of the luggage, and whether or not the person is injured may be identified by image analysis. Alternatively, a user may input the age, gender, whether or not the person to be tracked has luggage, the size of the luggage, and whether or not the person to be tracked has injuries to the tracking device 10. The moving object detection unit 12 can calculate the estimated movement speed of the tracked object based on, for example, this information. A speed calculation model is generated in advance, using this person information as input and outputting an estimated movement speed calculated based on the input person information. The speed calculation model may be a function, a learning model generated by machine learning, or other models. The moving object detection unit 12 inputs person information about the person to be tracked into such a speed calculation model and obtains the estimated movement speed output from the speed calculation model.

[0063] Alternatively, the moving object detection unit 12 may calculate the moving speed of the person to be tracked until the person is lost based on the image, and use the calculated result as the estimated moving speed of the person to be tracked. Calculation of the moving speed of the person detected in the image can be realized using any technology.

[0064] The first predetermined time may be a concept that includes not only the travel time required to travel from the facility entrance to the facility parking lot, but also at least one of the time required for a moving object to move within the parking lot and the time required for a person to move within the parking lot. For example, the facility information as shown in FIG. 7 may include an estimated time required for a moving object to move within the parking lot and an estimated time required for a person to move within the parking lot. The moving object detection unit 12 may then calculate the first predetermined time by adding that information to the calculated travel time required to travel from the facility entrance to the facility parking lot.

[0065] "Detection Processing Example 3" In the case assumed in this example, an object in a blind spot of the camera is present in the image generated by the camera, as shown in Fig. 8 and Fig. 9. In Fig. 8, the inside of a tunnel T is in the shadow. In Fig. 9, the part hidden by a building G is in the shadow.

[0066] When the person to be tracked gets on a moving body behind such an object, the moving body detection unit 12 can estimate the moving body on which the person to be tracked is riding.

[0067] The "motion that satisfies the time and position conditions" in detection processing example 3 is a motion that appears from behind an object within a predetermined distance from the position where the tracked person was lost after the timing at which the tracked person was lost. Note that "after the timing at which the tracked person was lost" may be replaced with "after the timing at which the tracked person was lost and before the timing at which a predetermined time has elapsed since the tracked person was lost." The predetermined distance is a distance that is set in advance. The predetermined time is a time that is set in advance.

[0068] Here, a specific example of the processing of the moving object detection unit 12 will be described. In one example, the moving object detection unit 12 analyzes images generated by a camera that generated an image in which the tracked person was lost, and detects from the image a moving object that performed an action that satisfies a time condition and a position condition. The image to be analyzed here is an image generated (taken) after the image in which the tracked person was lost.

[0069] The moving object detection unit 12 identifies an object area (an area behind an object) within a predetermined distance from the position where the tracked person was lost within the image to be analyzed. The moving object detection unit 12 then detects a moving object that appears from the identified object area after the timing at which the tracked person was lost (or after that but before a predetermined time has passed since the tracking person was lost). Note that information indicating one or more object areas within the image may be registered in the tracking device 10 in advance.

[0070] As shown in FIG. 8, when the object is a tunnel T, a plurality of cameras C installed at each of a plurality of exits of the tunnel T are used as shown in FIG. １ and C ２ In other words, the moving object detection unit 12 may analyze images generated by such a plurality of cameras and detect, from among the images, a moving object that has performed an action that satisfies the above-mentioned time condition and position condition.

[0071] The moving object detection unit 12 can identify cameras installed at multiple tunnel exits within a specified distance from the location where the person being tracked was lost, based on location information indicating the installation location of each of the multiple cameras as shown in Figure 5 and map information indicating the shape and location of the tunnel.

[0072] 1 , the moving object tracking unit 13 tracks the moving object (hereinafter referred to as the “tracked moving object”) detected by the moving object detection unit 12. The moving object tracking unit 13 can execute at least one of the following moving object tracking examples 1 and 2.

[0073] "Tracking Process Example 1" In this example, the moving object tracking unit 13 acquires an image. The moving object tracking unit 13 also acquires information indicating the external features of the moving object to be tracked. Then, the moving object tracking unit 13 detects the moving object to be tracked in the acquired image based on the external features of the moving object to be tracked, and tracks the moving object to be tracked in the image.

[0074] The "acquisition of an image" is realized by the same means as the acquisition of an image by the person tracking unit 11.

[0075] The "information indicating the external features of the moving object to be tracked" is input by the user to the tracking device 10. For example, the tracking device 10 may display an image of the moving object to be tracked detected by the moving object detection unit 12 on a display. Then, the user may identify the external features of the moving object to be tracked based on the image and input the identified external features to the tracking device 10. Alternatively, the moving object tracking unit 13 may analyze the image of the moving object to be tracked detected by the moving object detection unit 12 and extract the external features of the moving object to be tracked.

[0076] The "feature values ​​of the appearance of the moving object to be tracked" include the information written on the license plate (such as the number), the vehicle type, the color of the moving object, the design of the moving object, etc. If the moving object to be tracked is a bus, the feature values ​​of the appearance of the moving object to be tracked include the route name, route number, destination, etc., displayed on a bulletin board on the moving object to be tracked.

[0077] The moving object tracking unit 13 detects the moving object to be tracked from images captured by multiple surveillance cameras based on the external features of the moving object to be tracked. The moving object tracking unit 13 then tracks the moving object to be tracked within the images. The detection and tracking of the moving object within the images can be achieved using any technology.

[0078] "Tracking Process Example 2" When the moving object to be tracked is a public vehicle, as a tracking process, the moving object tracking unit 13 can identify one or more destinations of the moving object to be tracked and the estimated time of arrival at each destination based on the timetable of the moving object to be tracked.

[0079] The moving object tracking unit 13 can output the tracking results. For example, the moving object tracking unit 13 may output information indicating the current location of the moving object to be tracked. This information may be information in which the current location of the moving object to be tracked is mapped on a map, or may be other information. In addition, if the moving object to be tracked is a public vehicle, the moving object tracking unit 13 may output the destinations to which the moving object to be tracked is heading and the estimated times at which the moving object to be tracked will arrive at each destination. The output of this information is realized via any output device, such as a display, a projection device, a printer, or the like.

[0080] Next, an example of the processing flow of the tracking device 10 will be described with reference to the flowchart of FIG.

[0081] The tracking device 10 tracks the tracking target person in the image (S10). The tracking device 10 continues tracking the tracking target person while the tracking target person can be detected in the image (No in S11).

[0082] When the tracking device 10 loses sight of the tracked person (Yes in S11), it identifies the location and timing at which the tracked person was lost (S12).The tracking device 10 then executes a process to detect a moving object that has performed an action that satisfies a temporal condition based on the timing at which the tracked person was lost and a positional condition based on the location at which the tracked person was lost (S13).The tracking device 10 then begins tracking the moving object detected in S13 (S14).Although not shown, if the tracking device 10 cannot detect any moving object that has performed an action that satisfies the above-mentioned temporal and positional conditions even after a predetermined time has elapsed since the tracking device 10 lost sight of the tracked person, it may output information indicating this and end the process of detecting the moving object.

[0083] "Operational Effects" According to the tracking device 10 of this embodiment, the same operational effects as those of the tracking device 10 of the first embodiment are realized.

[0084] Furthermore, the tracking device 10 of this embodiment detects a moving object that has performed the characteristic "action that satisfies the time and position conditions" described above. Such a tracking device 10 can accurately detect a moving object that may have been ridden by a tracking target person. Furthermore, such a tracking device 10 can accurately estimate the moving object that the tracking target person is ridden by, even if the tracking target person is riding in a moving object that is in a blind spot of the camera, such as a parking lot of a facility or behind an object.

[0085] <Third embodiment> The tracking device 10 of this embodiment has a function of estimating the moving body carrying the person to be tracked from among multiple moving bodies that have performed "an action that satisfies a time condition and a position condition," as will be described in detail below.

[0086] 12 shows an example of a functional block diagram of the tracking device 10 of this embodiment. As shown in the figure, the tracking device 10 of this embodiment has a person tracking unit 11, a moving object detection unit 12, a moving object tracking unit 13, and an estimation unit 14. The configurations of the person tracking unit 11 and the moving object detection unit 12 are the same as those of the first and second embodiments.

[0087] When multiple moving objects that have performed actions that satisfy the time and location conditions are detected, the moving object tracking unit 13 tracks the multiple moving objects. The tracking method is as described in the second embodiment. Other configurations of the moving object tracking unit 13 are the same as those in the first and second embodiments.

[0088] The estimation unit 14 estimates the moving body in which the person to be tracked is riding from among the moving bodies based on the operation history of the moving bodies after tracking has started.

[0089] The operation history indicates at least one of whether or not a traffic light was run, the number of times a traffic light was run, whether or not the legal speed limit was exceeded, the degree to which the legal speed limit was exceeded, the number of lane changes, and the travel route. The degree to which the legal speed limit was exceeded indicates how much the speed limit was exceeded (the difference from the legal speed limit). Such an operation history can be generated based on images containing the moving object to be tracked. For example, the moving object tracking unit 13 may track the moving object to be tracked and generate the operation history by analyzing images containing the moving object to be tracked.

[0090] If the person being tracked is a fugitive, it is likely that they will frequently ignore traffic lights, exceed the legal speed limit, change lanes, etc. Also, it is likely that they will exceed the legal speed limit to a greater extent. From this perspective, it is possible to estimate the moving vehicle that the person being tracked is riding in from among multiple moving vehicles.

[0091] Furthermore, once the movement path of the moving object to be tracked is identified, the cameras installed on that movement path can be identified. Then, by performing face recognition processing based on the images generated by the identified cameras, the facial similarity between the person riding on the moving object to be tracked and the person to be tracked is calculated. Based on the results of such face recognition processing, the moving object on which the person to be tracked is riding can be estimated.

[0092] The estimation unit 14 estimates the moving body in which the person to be tracked is riding from among the plurality of moving bodies to be tracked, based on the operation history as described above.

[0093] For example, a possibility calculation model is generated in advance, which inputs the above-described motion history and similarity in face recognition processing and outputs the possibility that the tracked person has been riding (hereinafter, sometimes referred to as the "first possibility") calculated based on the input. The possibility calculation model may be a function, a learning model generated by machine learning, or other models. The estimation unit 14 inputs the motion history and similarity in face recognition processing of each tracked moving object into such a possibility calculation model, and obtains the first possibility output from the possibility calculation model.

[0094] The possibility calculation model is configured so that the first possibility becomes higher when a red light is run. The possibility calculation model is also configured so that the first possibility becomes higher the more frequently red light run-offs occur. The possibility calculation model is also configured so that the first possibility becomes higher when the legal speed limit is exceeded. The possibility calculation model is also configured so that the first possibility becomes higher the greater the degree to which the legal speed limit is exceeded. The possibility calculation model is also configured so that the first possibility becomes higher the more frequently lane changes occur. The possibility calculation model is also configured so that the first possibility becomes higher the higher the similarity obtained in the face recognition process.

[0095] Here, an example of a possibility calculation model will be described, but the present invention is not limited to this.

[0096] First, the likelihood calculation model calculates an evaluation point for each moving object to be tracked. For example, additional points are defined in advance for predetermined actions such as ignoring a traffic light, exceeding the legal speed limit, or changing lanes. Additional points may be defined according to the number of times or the degree of exceeding the legal speed limit. Furthermore, additional points may be defined according to the similarity obtained by the face recognition process.

[0097] Based on this definition, the possibility calculation model calculates the sum of added points as the evaluation points for each moving object to be tracked.

[0098] The likelihood calculation model then calculates a first likelihood for each tracked moving object based on the evaluation points for each tracked moving object. There are various methods for calculating the first likelihood from the evaluation points, and no particular limitation is imposed.

[0099] For example, in advance, "Evaluation points P ０ ~P １ : First possibility 0-10%" and "Evaluation points P １ ~P ２ Information that associates the evaluation points with the first possibilities, such as ": first possibility 10-20%", may be generated and stored in the tracking device 10. Then, based on such information, the possibility calculation model may identify the first possibility that corresponds to the evaluation points of each tracked moving object.

[0100] Alternatively, the first possibility of each tracked moving object may be calculated by dividing the evaluation point of each tracked moving object by the total evaluation points of the multiple tracked moving objects.

[0101] The estimation unit 14 can output the estimation result. For example, the estimation unit 14 may output the moving object with the highest first possibility calculated as described above as the estimation result of the moving object carried by the tracked person. Alternatively, the estimation unit 14 may output the moving object with the first possibility calculated as described above that is equal to or greater than a predetermined threshold as the estimation result of the moving object carried by the tracked person. Alternatively, the estimation unit 14 may display a list of the multiple moving objects to be tracked in order of the first possibility.

[0102] The estimation unit 14 may display the estimation result in real time and update the content of the estimation result in real time.

[0103] Next, an example of the processing flow of the tracking device 10 will be described with reference to the flowchart of FIG.

[0104] The tracking device 10 tracks the tracking target person in the image (S20). The tracking device 10 continues tracking the tracking target person while the tracking target person can be detected in the image (No in S21).

[0105] When the tracking device 10 loses sight of the tracked person (Yes in S21), it identifies the position and timing at which the tracked person was lost (S22).The tracking device 10 then executes a process to detect a moving object that has performed an action that satisfies a time condition based on the timing at which the tracked person was lost and a position condition based on the position at which the tracked person was lost (S23).

[0106] If only one moving object is detected in S23 (No in S24), the tracking device 10 starts tracking the moving object (S25).

[0107] On the other hand, if multiple moving objects are detected in S23 (Yes in S24), the tracking device 10 starts tracking the multiple detected moving objects (S26). Then, the tracking device 10 estimates which moving object the tracked person is riding on from among the multiple moving objects based on the movement history of the multiple moving objects after tracking began, and outputs the estimation result (S27). While tracking the multiple moving objects, the tracking device 10 can continue to collect the movement history of the multiple moving objects, estimate which moving object the tracked person is riding on, and output the estimation result. Note that, although not shown, if the tracking device 10 cannot detect any moving object that has performed an action that satisfies the above-mentioned time and position conditions even after a predetermined time has elapsed since the tracking device 10 lost sight of the tracked person, it may output information indicating this and terminate the process of detecting the moving object.

[0108] Other configurations of the tracking device 10 of this embodiment are similar to those of the first and second embodiments.

[0109] The tracking device 10 of this embodiment can provide the same operational effects as those of the first and second embodiments. Furthermore, when multiple moving objects that have performed "motions that satisfy the time and position conditions" are detected, the tracking device 10 of this embodiment can track the multiple moving objects and estimate the moving object carrying the person to be tracked based on the motion history of each moving object after tracking begins.

[0110] <Fourth embodiment> The tracking device 10 of this embodiment has a function of, when multiple moving bodies that depart from multiple predetermined boarding and alighting positions are detected as moving bodies that have performed "an action that satisfies a time condition and a location condition," estimating the moving body carrying the person to be tracked from among the detected moving bodies. The multiple moving bodies that depart from multiple predetermined boarding and alighting positions include a train departing from a station, a bus departing from a bus stop, a ship departing from a dock, an airplane departing from an airport, etc. This will be described in detail below.

[0111] 12 shows an example of a functional block diagram of the tracking device 10 of this embodiment. As shown in the figure, the tracking device 10 of this embodiment has a person tracking unit 11, a moving object detection unit 12, a moving object tracking unit 13, and an estimation unit 14. The configurations of the person tracking unit 11, the moving object detection unit 12, and the moving object tracking unit 13 are the same as those of the first to third embodiments.

[0112] When a plurality of moving bodies that depart from each of a plurality of predetermined boarding and alighting positions are detected as moving bodies that have performed an action that satisfies the time condition and the position condition, the estimation unit 14 estimates from among them the moving body that the person to be tracked is riding in. As described above, the plurality of moving bodies that depart from each of a plurality of predetermined boarding and alighting positions are trains that depart from stations, buses that depart from bus stops, ships that depart from docks, airplanes that depart from airports, etc.

[0113] The estimation unit 14 estimates the moving body that the tracked person is riding in from among the multiple moving bodies based on the movement path of the tracked person up to the time when sighting was lost and the positional relationship between the person and each of the multiple boarding and disembarking points. The movement path of the tracked person up to the time when sighting was lost can be identified based on the tracking result by the person tracking unit 11, for example.

[0114] A specific example of the estimation process will be described with reference to Fig. 14. In Fig. 14, a moving path Q of the person to be tracked up until the time when the person was lost and a plurality of boarding and alighting positions S ａ and S ｂ The position D where the person to be tracked is lost is shown. ａ If the person to be tracked gets on the moving vehicle from ａOn the other hand, if the person to be tracked arrives at the boarding and alighting position S ｂ If the person to be tracked gets on the moving vehicle from ｂ This means that we have reached this level.

[0115] It is unlikely that a fugitive will take a detour, and it is likely that he or she will head to the target boarding or alighting location via the shortest route. ｂ That is, the estimation unit 14 identifies a boarding / alighting position that the tracked person will reach via the shortest route or a boarding / alighting position that will be reached with the smallest deviation from the shortest route, based on the path of the tracked person's movement up to the time of loss of sight and the positional relationship between each of the multiple boarding / alighting positions. Then, the estimation unit 14 estimates that the tracked person got on the mobile body from the identified boarding / alighting position.

[0116] There are various means for determining whether the route to each boarding or alighting location is the shortest route or a detour route. For example, the shortest route can be determined by a route search that sets an arbitrary position on the path Q of the person being tracked up until the time of loss as the starting point and each boarding or alighting location as the destination point. If the route is the same as the shortest route calculated by the route search, it may be determined to be the shortest route, and if it is different, it may be determined to be a detour route. If the routes to all boarding or alighting locations are determined to be the shortest route, the starting point may be changed to another position on the path Q, and the same process may be repeated until it is determined that one of the routes is not the shortest route.

[0117] Furthermore, the deviation from the shortest route is indicated by the difference in distance between the first route and the second route, or the difference in the time required for travel. The larger the difference, the greater the deviation from the shortest route. The first route is, for example, "the shortest route calculated by route search, with an arbitrary position on the movement path Q of the tracked person up to the time of loss as the starting point, and each boarding and disembarking position as the destination point." The second route is, for example, "a route with the same starting point and destination point as the first route, traveling from the starting point to the point of loss along the movement path of the tracked person, and from there to the destination point as the shortest route calculated by route search."

[0118] In addition, when multiple moving bodies that depart from each of multiple predetermined boarding and disembarking locations are detected as moving bodies that have performed ``actions that satisfy the time and location conditions,'' the estimation unit 14 may calculate and output the probability of having boarded each moving body.

[0119] There are various specific methods for calculating the probability. For example, a pre-generated probability calculation model for calculating the probability may be used. The probability calculation model may be a function, a learning model generated by machine learning, or other models. The estimation unit 14 may calculate the probability using such a probability calculation model.

[0120] The probability calculation model may be configured to receive, for example, the input of the determination result of whether the route is the shortest or not. In this case, the probability calculation model is configured to calculate a relatively high probability of getting on a moving vehicle that started from a boarding / alighting location determined to be the shortest route. The probability calculation model is also configured to calculate a relatively low probability of getting on a moving vehicle that started from a boarding / alighting location determined not to be the shortest route.

[0121] The probability calculation model may be configured to receive, for example, the input of the determination result of the deviation from the shortest route. In this case, the probability calculation model is configured to calculate a relatively high probability of having boarded a moving object that departs from a boarding / alighting location that results in a route with a smaller deviation from the shortest route. The probability calculation model is also configured to calculate a relatively low probability of having boarded a moving object that departs from a boarding / alighting location that results in a route with a larger deviation from the shortest route.

[0122] Furthermore, the probability calculation model may calculate the probability using a weighting that is preset for a mobile body departing from each boarding / alighting position. The probability calculation model is configured to calculate a higher probability for a mobile body with a higher weighting. The probability calculation model is configured to calculate a lower probability for a mobile body with a lower weighting. There are various weighting rules. For example, the more passengers a mobile body has, the higher the weighting may be. Furthermore, the more trains a mobile body has per hour, the higher the weighting may be. The user sets a weight for each boarding / alighting position in advance and registers it in the tracking device 10. The probability calculation model calculates the probability using the registered weighting.

[0123] The estimation unit 14 may have the same configuration as that of the third embodiment.

[0124] Other configurations of the tracking device 10 of this embodiment are similar to those of the first to third embodiments.

[0125] The tracking device 10 of this embodiment can achieve the same effects as those of the first to third embodiments. Furthermore, when multiple moving bodies that depart from multiple predetermined boarding and disembarking locations are detected as moving bodies that have performed "an action that satisfies the time and location conditions," the tracking device 10 of this embodiment estimates the moving body carrying the tracked person from among the multiple moving bodies. The tracking device 10 estimates the moving body carrying the tracked person from among the multiple moving bodies based on the movement path of the tracked person up until the time the tracked person was lost and the positional relationship between the path and each of the multiple boarding and disembarking locations. Such a tracking device 10 can accurately estimate the moving body carrying the tracked person.

[0126] <Modification> In the example described in the second embodiment, the tracking device 10 identifies "the action of leaving a parking lot exit of a facility that is within a predetermined distance from the location where the tracked person was lost" through image analysis. As a modification, the tracking device 10 may detect a moving object that performed such an action based on the operation history of the parking lot exit gate. The operation history indicates the timing when the exit gate opened.

[0127] In this modified example, the tracking device 10 and the parking lot exit gate control system are configured to be able to communicate with each other. The tracking device 10 acquires the operation history of the parking lot exit gate from the system, and detects the mobile object that has performed the exit operation based on the acquired operation history.

[0128] A camera may be installed near the exit gate of the parking lot to capture an image of the moving object exiting the parking lot. The tracking device 10 may then acquire the external features of the moving object to be tracked based on the image generated by the camera.

[0129] In this modification, the same effects as those of the above embodiment are achieved.

[0130] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations may be adopted. The configurations of the above-described embodiments may be combined with each other, or some of the configurations may be replaced with other configurations. Furthermore, various modifications may be made to the configurations of the above-described embodiments without departing from the spirit of the invention. Furthermore, the configurations and processes disclosed in the above-described embodiments and modified examples may be combined with each other.

[0131] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-described embodiments can be combined to the extent that the content is not contradictory.

[0132] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes: 1. A tracking device comprising: person tracking means for tracking a tracked person within an image; moving body detection means for detecting a moving body that has performed an action that satisfies a temporal condition based on the timing at which the tracked person was lost and a positional condition based on the position at which the tracked person was lost; and moving body tracking means for tracking the detected moving body. 2. The tracking device described in 1, wherein the action that satisfies the temporal and positional conditions is an action of starting at a position within a reference distance from the position at which the tracked person was lost, at a timing after the timing at which the tracked person was lost. 3. The tracking device described in 1 or 2, wherein the action that satisfies the temporal and positional conditions is an action of exiting from an exit of a parking lot of a facility that is within a predetermined distance from the position at which the tracked person was lost, at a timing when a predetermined time has elapsed since the tracked person was lost, and the predetermined time is the travel time required to travel from the entrance of the facility to the parking lot of the facility. 4. The tracking device according to any one of 1 to 5, wherein the moving object detection means determines a moving speed of the tracked person based on person information about the tracked person, and calculates the moving time based on the determined moving speed. 5. The tracking device according to 4, wherein the person information indicates at least one of the age, sex, whether or not the tracked person has luggage, the size of the luggage, whether or not the tracked person is injured, and the moving speed up to that point. 6. The tracking device according to any one of 1 to 5, wherein the action that satisfies the temporal and positional conditions is an action of appearing from behind an object within a predetermined distance from the position where the tracked person was lost, at a timing after the tracked person was lost. 7. The tracking device according to any one of 1 to 6, wherein the moving object tracking means, when multiple moving objects that have performed actions that satisfy the temporal and positional conditions are detected, tracks the multiple moving objects, and further includes estimation means for estimating the moving object carrying the tracked person from among the multiple moving objects based on the movement history of the multiple moving objects after tracking began.8. The tracking device according to 7, wherein the operation history indicates at least one of whether or not a traffic light was run, the number of times a traffic light was run, whether or not the speed limit was exceeded, the degree to which the speed limit was exceeded, the number of lane changes, and a travel route. 9. The tracking device according to 8, wherein the operation history indicates a travel route, and the estimation means estimates the moving body in which the tracked person is riding by face recognition processing based on images from a camera installed on the travel route of the moving body. 10. The tracking device according to any one of 1 to 9, further comprising estimation means, when a plurality of moving bodies that departed from a plurality of predetermined boarding and alighting positions are detected as a plurality of moving bodies that performed operations that satisfy the time condition and the positional condition, for estimating the moving body in which the tracked person is riding from among the plurality of moving bodies based on the travel route of the tracked person up to the time when sighting was lost and the positional relationship between the plurality of boarding and alighting positions. 11. A tracking method in which one or more computers track a target person within an image, detect a moving object that has performed an action that satisfies a temporal condition based on when sight of the target person was lost and a positional condition based on the position at which sight of the target person was lost, and track the detected moving object. 12. A program that causes a computer to function as: person tracking means that tracks the target person within an image, moving object detection means that detects a moving object that has performed an action that satisfies a temporal condition based on when sight of the target person was lost and a positional condition based on the position at which sight of the target person was lost, and moving object tracking means that tracks the detected moving object.

[0133] This application claims priority based on Japanese Patent Application No. 2023-041790, filed March 16, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0134] REFERENCE SIGNS LIST 10 Tracking device 11 Person tracking unit 12 Moving object detection unit 13 Moving object tracking unit 14 Estimation unit 1A Processor 2A Memory 3A Input / output I / F 4A Peripheral circuit 5A Bus

Claims

1. a person tracking means for tracking a person to be tracked within an image; a moving object detection means for detecting a moving object that has performed an action that satisfies a time condition based on the timing at which the tracking target person was lost and a positional condition based on the position at which the tracking target person was lost; a moving object tracking means for tracking the detected moving object; A tracking device having:

2. The tracking device according to claim 1, wherein the action that satisfies the time condition and the positional condition is an action of starting at a time after the time when the tracked person is lost, at a position within a reference distance from the position where the tracked person is lost.

3. the action that satisfies the time condition and the positional condition is an action of exiting a parking lot of a facility that is located within a predetermined distance from a position where the tracking target person was lost after a predetermined time has elapsed since the tracking target person was lost, 2. The tracking device according to claim 1, wherein the predetermined time is a travel time required to travel from an entrance of the facility to a parking lot of the facility.

4. The moving object detection means 4. The tracking device according to claim 3, wherein a moving speed of the person to be tracked is determined based on person information relating to the person to be tracked, and the moving time is calculated based on the determined moving speed.

5. The tracking device according to claim 4 , wherein the person information indicates at least one of the age, sex, whether or not the person is carrying luggage, the size of the luggage, whether or not the person is injured, and the speed of movement up to that point.

6. The tracking device according to any one of claims 1 to 5, wherein the action that satisfies the time condition and the positional condition is an action of appearing from behind an object within a predetermined distance from the position where the tracked person was lost at a time after the time when the tracked person was lost.

7. The moving object tracking means When a plurality of moving objects that have performed an action that satisfies the time condition and the position condition are detected, the plurality of moving objects are tracked; The tracking device according to any one of claims 1 to 5, further comprising an estimation means for estimating the moving body carrying the person to be tracked from among the plurality of moving bodies based on the operation history of the plurality of moving bodies after tracking has begun.

8. A tracking device as described in any one of claims 1 to 5, further comprising an estimation means for estimating the moving body carrying the person to be tracked from among the moving bodies based on the movement path of the person to be tracked up to the time when the person to be tracked was lost and the positional relationship between the person to be tracked and each of the plurality of boarding and disembarking locations when the moving bodies are detected as the moving bodies that have performed an action that satisfies the temporal conditions and the positional conditions.

9. One or more computers Track the person in the image, detecting a moving object that has performed an action that satisfies a time condition based on the timing at which the tracking target person was lost and a positional condition based on the position at which the tracking target person was lost; A tracking method for tracking the detected moving object.

10. Computer, a person tracking means for tracking a person to be tracked within an image; a moving object detection means for detecting a moving object that has performed an action that satisfies a time condition based on the timing at which the tracking target person was lost and a positional condition based on the position at which the tracking target person was lost; a moving object tracking means for tracking the detected moving object; A program that functions as a