Information processing apparatus and information processing method

Abstract

To improve the follow-up accuracy of a follow-up object.SOLUTION: An information processing apparatus according to an embodiment includes a following unit and a switching unit. The tracking unit tracks the tracking target by the first method in the first environment. The switching unit switches the following method by the following unit from the first method to the second method when the environment to be followed is switched from the first environment to the second environment.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Conventionally, services have been provided that use images captured by a camera, etc. For example, a technology has been proposed in which an image is captured by a camera, etc., attached to a lighting device, and a target to be tracked, such as a person, is tracked using the captured image (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-49772 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, switching between tracking methods is not taken into consideration, and there is room for improvement in terms of improving the tracking accuracy of the target to be tracked.

[0005] The present invention has been made in view of the above, and has an object to provide an information processing device and an information processing method that can improve the tracking accuracy of a tracking target. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, an information processing device according to the present invention includes a tracking unit and a switching unit. The tracking unit tracks a target object in a first environment using a first method. When the environment of the target object switches from the first environment to a second environment, the switching unit switches the tracking method used by the tracking unit from the first method to the second method. [Effects of the Invention]

[0007] According to the present invention, it is possible to improve the tracking accuracy of the target to be tracked. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an overview of the information processing method according to the embodiment. [Figure 3] FIG. 3 is a block diagram of the camera according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of information stored in the image data storage unit according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of information stored in a time period data storage unit according to the embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of a process for setting time period data according to the embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of the following process according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The camera 50 (an example of an information processing device) according to the embodiment described below includes a tracking unit 543 that tracks the position of a target to be tracked using a first method in a first environment, and a switching unit 544 that switches the tracking method used by the tracking unit 543 from the first method to the second method when the environment of the target to be tracked switches from the first environment to a second environment.

[0010] Furthermore, the first method in the embodiment described below is a method that is more effective than the second method at tracking a target in a first environment, and the second method is a method that is more effective than the first method at tracking a target at the boundary between the first environment and the second environment.

[0011] Furthermore, the first environment in the embodiment described below is an area with higher illumination than the second environment, and the switching unit 544 switches the tracking method used by the tracking unit 543 from the first method to the second method when the target to be tracked passes the boundary between the first environment and the second environment.

[0012] Furthermore, the first method in the embodiment described below is a method of tracking a target based on color information of an image taken outdoors, and the second method is a method of tracking a target based on a target area of ​​the target detected from an image taken outdoors.

[0013] Furthermore, the switching unit 544 according to the embodiment described below switches the tracking method used by the tracking unit 543 from the second method to the first method after the target to be tracked moves from the boundary to the second environment.

[0014] Furthermore, the camera 50 (an example of an information processing device) according to the embodiment described below includes a setting unit 542 that sets boundary time zone data based on an image captured by a camera installed outdoors, and a switching unit 544 switches the tracking method based on the time zone data set by the setting unit 542.

[0015] Furthermore, the information processing method according to the embodiment described below is an information processing method executed by a computer, and includes a tracking process for tracking the position of a target to be tracked in a first environment using a first method, and a switching process for switching the tracking method used in the tracking process from the first method to the second method when the environment of the target to be tracked switches from the first environment to a second environment.

[0016] (Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the following embodiments do not limit the technology disclosed by the present invention. Furthermore, the same components in each embodiment are designated by the same reference numerals, and redundant explanations will be omitted.

[0017] First, an information processing system will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an information processing system according to an embodiment. The information processing system S according to the embodiment is a system that is introduced into various facilities, including public facilities and private facilities, and performs people flow analysis based on images captured by each camera 50. Note that the following description will be given of a case where the tracking targets are all people appearing in the images.

[0018] 1, the information processing system S includes a server device 1 and a plurality of cameras 50. Also, as shown in FIG. 1, the server device 1 and the plurality of cameras 50 are connected by a predetermined network N.

[0019] The server device 1 is an information processing device that aggregates people flow data analyzed by each camera 50. For example, the server device 1 provides the aggregated people flow data to a client or the like who will be introducing the information processing system S. Note that, although the present disclosure describes a case where the processing of the information processing device according to the embodiment is performed by the camera 50, a configuration may be adopted in which the server device 1 performs some or all of the processing.

[0020] The camera 50 is installed in various facilities and captures images. The camera 50 corresponds to an example of an information processing device according to an embodiment. In the present disclosure, the camera 50 analyzes people flow data based on the captured images. For this reason, the camera 50 is installed at a relatively high position, where it can capture images from a bird's-eye view. The camera 50 may be configured as an integral part of a lighting device installed on the ceiling of the facility, for example.

[0021] In the present disclosure, the camera 50 is installed outdoors and captures the outside of the facility. When the camera 50 is installed outdoors, the environment of the person to be tracked may change.

[0022] Specifically, there are cases where the environment of the person to be tracked changes, such as when the person moves from sunlight to shade, or from shade to sunlight, etc. When tracking a person using recent AI (Artificial Intelligence), the person is tracked based on the similarity of the color information of the person in each image.

[0023] However, when a person to be tracked moves from sunlight to shade or from shade to sunlight, the color information of the person changes suddenly. Therefore, tracking based on color information may not recognize the same person even if it is actually the same person. In other words, in such cases, when the person to be tracked moves from sunlight to shade or from shade to sunlight, there is a risk that the person to be tracked will be lost.

[0024] Therefore, in the information processing method according to the embodiment, the tracking method is switched depending on the environment of the person to be tracked. Fig. 2 is a diagram showing an overview of the information processing method according to the embodiment. Note that this information processing method is executed by a camera 50.

[0025] 2 schematically shows a first area A1, a second area A2, a boundary area Ab between the first area A1 and the second area A2, and a tracking method for each corresponding area. The first area A1 is an area of ​​the first environment, and the second area A2 is an area of ​​the second environment.

[0026] The first environment is an environment with higher illuminance than the second environment. Specifically, the first environment is in the sunshine, and the second environment is in the shade. A boundary area Ab between the first area A1 and the second area A2 is an area that is the boundary between the sunshine and the shade. Note that although the boundary area Ab is strip-shaped in FIG. 2, it may also be linear.

[0027] As shown in FIG. 2, in the information processing method according to the embodiment, the first area A1 and the second area A2 are tracked by the first method, and the boundary area Ab is tracked by the second method.

[0028] That is, in the information processing method according to the embodiment, when the target moves from the first region A1 to the second region A2, the target is tracked using the first method in the first region A1, and the tracking method is switched from the first method to the second method in the boundary region Ab. After that, after the target moves to the second region A2, the information processing method according to the embodiment switches the tracking method from the second method to the first method and tracks the target.

[0029] Here, the first method is a method of tracking a target based on color information of a person, and the second method is a method of tracking a target based on a person area detected from an image. More specifically, in the first method, a person in an image is detected using neural network technology or the like.

[0030] Next, in the first method, characteristic information such as the clothing and background of the detected person is stored, and if the similarity of the characteristic information of the person detected in the previous frame is above a threshold, the people are considered to be the same person and the person is tracked.

[0031] In the second method, neural network technology or the like is used to detect a person area (e.g., a rectangular area) in which a person appears in an image, and this area is compared with the person area detected in the previous frame. If the similarity of the vectors and size of the person area is above a threshold, the people are considered to be the same person and the person is tracked.

[0032] In other words, the first method tracks people based on the similarity of color information such as the person's clothing and background in each frame, while the second method tracks people based on mathematical techniques such as the vector and size of the person's area in each frame.

[0033] In this way, in the information processing method of the embodiment, when the environment of the person to be tracked changes, such as when the person to be tracked moves from the first area A1 to the second area A2, tracking is performed by switching from the first method to the second method.

[0034] In other words, in the information processing method of the embodiment, when a person to be tracked moves from sunlight (first environment) to shade (second environment), tracking is performed at the boundary by switching to the second method that does not depend on changes in color information.

[0035] Therefore, according to the information processing method according to the embodiment, when the first environment and the second environment are switched, the target to be tracked can be tracked appropriately, and therefore tracking accuracy can be improved.

[0036] For example, the first method and the second method may be performed in parallel for each target, and the period in which the target was lost by the first method may be interpolated using the processing results of the second method.

[0037] Next, an example of the configuration of a camera 50 according to an embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram of the camera 50 according to the embodiment. As shown in Fig. 3, the camera 50 has a communication unit 51, an imaging unit 52, a storage unit 53, and a control unit 54.

[0038] The communication unit 51 is implemented by a predetermined communication circuit such as a NIC (Network Interface Card), and performs data communication with the server device 1 and the like via a communication network such as Ethernet (registered trademark) or LAN.

[0039] The imaging unit 52 is implemented by a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, etc. The imaging unit 52 captures an image of an imaging area to generate an image, and outputs the image to the control unit .

[0040] The storage unit 53 is implemented by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. In the example shown in FIG. 3, the storage unit 53 has an image data storage unit 531, a time period data storage unit 532, and a people flow data storage unit 533.

[0041] The image data storage unit 531 stores image data. The image data is data of an image captured by the imaging unit 52. Fig. 4 is a diagram showing an example of information stored in the image data storage unit 531 according to the embodiment.

[0042] 4, the image data storage unit 531 stores information such as "imaging date and time," "image data," and "flag data" in association with each other. The "imaging date and time" item stores the imaging date and time of the corresponding image data.

[0043] The "image data" field stores data of the image captured on the corresponding capture date and time. The "flag data" field stores data of the flag set for the image captured on the corresponding capture date and time. Flags will be described later.

[0044] Returning to the explanation of Figure 3, the time zone data storage unit 532 will now be described. The time zone data storage unit 532 stores time zone data. The time zone data is time zone data for the boundary between the first area A1 and the second area A2. Specifically, the boundary between sunshine and shade changes depending on the direction and altitude of the sun, and therefore the boundary changes depending on the season and time of day.

[0045] The time period data is data that defines the coordinates of the boundaries of each time period. Fig. 5 is a diagram showing an example of information stored in the time period data storage unit 532 according to the embodiment. As shown in Fig. 5, the time period data storage unit 532 stores information items such as "time period" and "coordinate data" in association with each other.

[0046] The "Time Zone" item stores information about the time zone to which the corresponding coordinate data is applicable. The time zone includes information about the date and time, such as 10:00 to 11:00 on July 1st. The "Coordinate Data" stores information about the coordinates of the churches in the first area A1 and the second area A2 during the corresponding time zone.

[0047] Returning to the explanation of Fig. 3, the following will explain the people flow data storage unit 533. The people flow data storage unit 533 stores people flow data. The people flow data is data obtained by tracking each person and aggregating the trajectory of each person.

[0048] Next, the control unit 54 will be described. The control unit 54 is a controller that controls the entire camera 50. For example, the control unit 54 can be implemented by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). Alternatively, the control unit 54 may be implemented by an integrated circuit such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0049] 3, the control unit 54 has an acquisition unit 541, a setting unit 542, a tracking unit 543, and a switching unit 544. The acquisition unit 541 acquires an image captured by the imaging unit 52. The acquisition unit 541 also stores the acquired image in the image data storage unit 531.

[0050] The setting unit 542 sets the time period data of the boundary based on an image captured by the outdoor camera 50. The processing by the setting unit 542 is performed, for example, when the camera 50 is initially set up.

[0051] For example, once the camera 50 is installed, the imaging unit 52 captures images at predetermined intervals for several days when no people are in the image. The setting unit 542 analyzes these images and sets a weather flag for the image. The weather flag may be one of four types: "sunny," "cloudy," "rainy," and "snowy." The weather flag may be set based on meteorological information.

[0052] The setting unit 542 performs further analysis on images for which a flag corresponding to "sunny" has been set. The setting unit 542 sets an environmental flag for each coordinate within the image. The environmental flags are of three types: "sunny (corresponding to the first environment)," "shade (corresponding to the second environment)," and "unknown."

[0053] Then, the setting unit 542 sets data on the boundary between the first environment and the second environment for each time period, and stores the data as time period data in the time period data storage unit 532. This completes the initial setting by the setting unit 542. Thereafter, the setting unit 542 sets time period data at an arbitrary cycle, and interpolates the time period data for each time period.

[0054] In this way, by setting the time period data in advance, it becomes possible to smoothly perform switching processing by the switching unit 544, which will be described later. Furthermore, by setting the time period data based on an image taken on a "sunny" day, the setting unit 542 can accurately detect sunny and shady areas, and therefore can set highly accurate time period data.

[0055] The tracking unit 543 tracks the position of the target to be tracked using a first method or a second method. The first method tracks the target to be tracked based on color information of the person, and the second method tracks the target to be tracked based on a person area detected from an image.

[0056] That is, the first method is more effective than the second method in an environment where color information does not change, and the second method is more effective than the first method in an environment where color information changes.

[0057] The tracking unit 543 switches the tracking method to either the first method or the second method and performs tracking based on a switching instruction from the switching unit 544. Specifically, for example, assume that a person to be tracked moves from sunny (first environment) to shaded (second environment).

[0058] In such a case, first, the tracking unit 543 performs tracking using the first method in the first environment, and switches from the first method to the second method at the boundary between the first environment and the second environment.

[0059] Thereafter, when the person to be tracked has completely moved into the second environment, the tracking unit 543 switches from the second method to the first method to perform tracking. In this way, the tracking unit 543 performs tracking using the first method in the first environment and the second environment, and performs tracking using the second method at the boundary between them.

[0060] That is, at a boundary where color information changes suddenly, the tracking unit 543 switches from the first method to the second method to perform tracking. In this way, the tracking unit 543 switches the tracking method to perform tracking at the boundary between the first environment and the second environment.

[0061] This allows the target to be tracked appropriately even at the boundary between the first environment and the second environment, thereby improving tracking accuracy.

[0062] When the environment of the tracking target switches from the first environment to the second environment, the switching unit 544 switches the tracking method used by the tracking unit 543 from the first method to the second method. Specifically, the switching unit 544 refers to time period data corresponding to the current time period and determines whether the coordinates of the person in the image are on the boundary between the first environment and the second environment.

[0063] Then, the switching unit 544 sets the tracking method for a person that is determined as not being on the boundary between the first environment and the second environment as the first method, and sets the tracking method for a person that is determined as being on the boundary between the first environment and the second environment as the second method.

[0064] That is, when the person to be tracked moves from the first environment to the second environment, or when the person moves from the second environment to the first environment, the switching unit 544 switches the tracking method of the tracking unit 543 from the first method to the second method. This makes it possible to prevent the person to be tracked from being lost at the boundary between the first environment and the second environment.

[0065] Next, a processing flow of information processing according to the embodiment will be described with reference to Fig. 6 and Fig. 7. Fig. 6 is a flowchart showing an example of a setting process of time zone data according to the embodiment.

[0066] 6, the camera 50 captures images at predetermined intervals (step S101). Then, the camera 50 sets a flag for each image (step S102). Specifically, the camera 50 sets a weather flag for each image, and then sets an environment flag for images with a weather flag corresponding to sunny weather.

[0067] Next, the camera 50 sets time zone data for the boundary area between the first environment and the second environment based on the environment flag (step S103), and ends the process.

[0068] Next, the tracking process according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the tracking process according to the embodiment. As shown in Fig. 7, first, the camera 50 sets a boundary area based on time zone data corresponding to the current time (step S111).

[0069] Next, the camera 50 acquires an image captured by the imaging unit 52 (step S112). Next, the camera 50 tracks the person in the first environment and the person in the second environment using the first method (step S113).

[0070] Next, the camera 50 determines whether or not a person is present in the boundary area between the first environment and the second environment (step S114). If the camera 50 determines that a person is present in the boundary area (step S114; Yes), it tracks the person using the second method (step S115) and ends the process.

[0071] That is, the camera 50 tracks a person in the boundary area by switching from the first method to the second method. If the camera 50 determines in step S114 that there is no person in the boundary area (step S114; No), it omits the processing of step S115.

[0072] As described above, the camera 50 (an example of an information processing device) according to the embodiment includes a tracking unit 543 that tracks the position of a target to be tracked in a first environment using a first method, and a switching unit 544 that switches the tracking method used by the tracking unit 534 from the first method to the second method when the environment of the target to be tracked switches from the first environment to the second environment.

[0073] Therefore, according to the camera 50 of the embodiment, when the environment switches between the first environment and the second environment, the tracking target can be appropriately tracked, thereby improving the tracking accuracy.

[0074] Incidentally, in the above-described embodiment, the first method is a tracking method based on color information, and the second method is a tracking method based on a person area, but the present invention is not limited to this.

[0075] For example, the first method may be a tracking method using skeleton detection, and in an environment where a person is cut off by an obstacle, etc., the second method based on color information may be used. In such a case, a thermosensor may be used as the second method.

[0076] Also, suppose the first environment is a dark place and the second environment is a bright place. In such a case, the first method may be a tracking method using an infrared camera, and the second method may be a tracking method based on color information. In this way, switching between the first method and the second method is not limited to the boundary between the first environment and the second environment, and the first method and the second method may be switched for each environment.

[0077] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as the inventions described in the claims and their equivalents. [Explanation of symbols]

[0078] 1. Server device 50 cameras 51 Communications Department 52 Imaging unit 53 Memory section 54 Control Unit 531 Image data storage unit 532 Time zone data storage unit 533 Human flow data storage section 541 Acquisition Department 542 Settings 543 Follower 544 Switching section A1 1st area A2 2nd area Ab boundary area N Network S Information Processing System

Claims

1. a tracking unit that tracks a target object in a first environment using a first method; a switching unit that switches a tracking method used by the tracking unit from the first method to the second method when the environment of the tracking target switches from the first environment to the second environment; An information processing device comprising:

2. The first method is a method for tracking the target in the first environment that is more effective than the second method; The second method is A method for tracking the target at the boundary between the first environment and the second environment is more effective than the first method. The information processing device according to claim 1 .

3. The first environment is An area with a higher illuminance than the second environment, The switching unit is When the object to be tracked passes through a boundary between the first environment and the second environment, the tracking method by the tracking unit is switched from the first method to the second method. The information processing device according to claim 2 .

4. The first method is A method for tracking the target based on color information of an image captured outdoors, The second method is This is a method for tracking a target based on a target area of ​​the target detected from an image taken outdoors. The information processing device according to claim 2 .

5. The switching unit is After the target object moves from the boundary to the second environment, the tracking method by the tracking unit is switched from the second method to the first method. The information processing device according to claim 2 .

6. a setting unit that sets time zone data of the boundary based on an image captured by a camera installed outdoors; Equipped with The switching unit is The tracking method is switched based on the time period data set by the setting unit. The information processing device according to claim 2 .

7. 1. A computer-implemented information processing method, comprising: a tracking step of tracking a target object in a first environment using a first technique; a switching step of switching a tracking method used in the tracking step from the first method to the second method when the environment of the tracking target is switched from the first environment to the second environment; An information processing method, including:

Images