Management system

Mobile cameras with self-locating capabilities enhance area management by flexibly setting target areas, improving accuracy and reducing costs by covering areas not covered by static cameras.

JP2024168498AInactive Publication Date: 2024-12-05TOYOTA JIDOSHA KK
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2023085221
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-24
Publication Date
2024-12-05
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing area management systems using static cameras have limited and fixed target areas, leading to insufficient accuracy and requiring significant effort and cost to expand these areas by adding more cameras.

Method used

Incorporating mobile cameras mounted on movable objects with self-locating capabilities, allowing flexible setting of target areas by using the mobile camera's position to extract images of interest for area management processing.

Benefits of technology

Enhances area management accuracy and enables cost-effective expansion of target areas without the need for additional static cameras, covering interests areas not covered by static cameras.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024168498000001_ABST
    Figure 2024168498000001_ABST
Patent Text Reader

Abstract

To flexibly set an object area in managing the object area on the basis of an image photographed by a camera.SOLUTION: A management system communicates with a moving body with a self-position specification function. The management system acquires images photographed by a mobile camera mounted on the moving body, and information on mobile camera positions which are positions of the moving body when the images are photographed. The management system extracts, as object images, images of object areas photographed by the mobile camera on the basis of the mobile camera positions. The management system executes area management processing for managing the object areas on the basis of the object images.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a technique for managing a target area based on images captured by a camera. [Background technology]

[0002] Patent Document 1 discloses an image recognition system that tracks a person using images captured by multiple cameras. The multiple cameras capture images of a specific person from different orientations. The specific person is tracked by using the multiple images captured from different orientations. The position information of each of the multiple cameras is used to calculate the relative angle between the multiple cameras. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-001447 A Summary of the Invention [Problem to be solved by the invention]

[0004] Consider area management processing that manages a target area based on images captured by a camera. When only still cameras (fixed cameras) are used, the capture area is fixed. In other words, the target area that is the subject of the area management processing is fixed and limited. In this case, the accuracy of the area management processing is not necessarily sufficient. Increasing the number of still cameras to expand the target area requires enormous effort and cost.

[0005] An object of the present disclosure is to provide a technology that allows for flexible setting of a target area when managing the target area based on an image captured by a camera. [Means for solving the problem]

[0006] One aspect of the present disclosure relates to a management system. The management system comprises one or more processors. The one or more processors are in communication with a mobile unit having self-location capability. The one or more processors acquire information on an image captured by a moving camera mounted on a moving object, and the position of the moving camera, which is the position of the moving object when the image was captured. The one or more processors extract, based on the moving camera position, an image of the target area captured by the moving camera as a target image. The one or more processors execute an area management process for managing a target area based on the target image. Effect of the Invention

[0007] According to the present disclosure, a moving camera is used for the area management process. More specifically, in addition to an image captured by the moving camera, the position of the moving camera at the time of image capture is obtained. Based on the moving camera position, an image of a target area captured by the moving camera is extracted as a target image. By using the extracted target image, area management process for the target area can be performed.

[0008] Since the shooting area of ​​the moving camera used in the area management process is not fixed, it is possible to set the target area to be the subject of the area management process more flexibly. Even if the target area is not covered by a stationary camera, it is possible to cover the target area with a moving camera. Therefore, the accuracy of the area management process can be improved.

[0009] Furthermore, according to the present disclosure, there is no need to increase the number of still cameras in order to expand the target area. Since the shooting area of ​​the moving camera is not fixed but can be flexibly set, it is possible to greatly expand the target area even when only a small number of moving cameras are used. In other words, according to the present disclosure, it is possible to expand the target area of ​​the area management process at low cost and simply. [Brief description of the drawings]

[0010] [Figure 1] FIG. 13 is a conceptual diagram for explaining a comparative example. [Diagram 2] FIG. 1 is a conceptual diagram for explaining an overview of an embodiment. [Diagram 3] 1 is a block diagram showing an example of a configuration of a management system according to an embodiment; [Figure 4] FIG. 2 is a conceptual diagram illustrating an example of an image database according to the embodiment. [Diagram 5] FIG. 2 is a functional block diagram for explaining a first example of an area management process according to an embodiment. [Figure 6] FIG. 11 is a functional block diagram for explaining a second example of the area management process according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0012] 1. Overview Consider the "area management process" that manages a target area based on images captured by a camera. For example, the area management process includes monitoring the target area based on images. Monitoring includes detection of anomalies (e.g., accidents, troubles, crimes, suspicious people, sick people, etc.). As another example, the area management process includes detection or search for targets (e.g., people, events) within the target area based on images. Person re-identification, which identifies and tracks the same person from multiple images captured by multiple cameras, is also included in the area management process. In either case, anomalies and targets can be detected from images by using a machine learning model.

[0013] First, a comparative example will be described with reference to FIG. 1. In the comparative example, only still cameras 10 (fixed cameras) installed at fixed positions are used for area management processing. The management system communicates with the still cameras 10 and acquires images SIMG captured by the still cameras 10. The management system then performs area management processing based on the images SIMG. For example, the management system accumulates the images SIMG and searches for a specific person P from among the accumulated images SIMG. As another example, the management system may perform person re-identification based on multiple images captured by multiple still cameras 10.

[0014] In the example shown in FIG. 1, the area management process is performed based on images SIMG-1 and SIMG-2 captured by still cameras 10-1 and 10-2. The image capturing areas AR-1 and AR-2 are areas that can be captured by the still cameras 10-1 and 10-2, respectively, and are fixed. That is, the target areas that are the subject of the area management process are only the fixed image capturing areas AR-1 and AR-2. Since the target areas are fixed and limited, the accuracy of the area management process is not necessarily sufficient. For example, if a person P goes outside the image capturing areas AR-1 and AR-2, the person P cannot be detected.

[0015] In this way, when only still cameras 10 are used, the target area to be subjected to the area management process is fixed and limited. In this case, the accuracy of the area management process is not necessarily sufficient. Increasing the number of still cameras 10 to expand the target area requires a huge amount of work and cost. Therefore, this embodiment proposes a technology that allows the target area to be set more flexibly without increasing the number of still cameras 10.

[0016] FIG. 2 is a conceptual diagram for explaining an outline of this embodiment. According to this embodiment, in addition to the still camera 10, or instead of the still camera 10, a moving camera 20 mounted on a moving object 2 is used. The moving object 2 can move around freely. Examples of such a moving object 2 include a vehicle, a robot, a flying object, and the like. The vehicle may be an autonomous vehicle or a vehicle driven by a driver. Examples of the robot include a logistics robot, and the like. Examples of the flying object include an airplane, a drone, and the like.

[0017] Furthermore, the moving object 2 has a self-localization function and can acquire its own position information. For example, the moving object 2 acquires its position information using a Global Navigation Satellite System (GNSS). In this embodiment, the position of the moving object 2 is regarded as the position of the moving camera 20. The position of the moving object 2, that is, the position of the moving camera 20, is hereinafter referred to as the "moving camera position MPOS." The moving camera position MPOS is a position in an absolute coordinate system and can be represented on a map.

[0018] Unlike the still camera 10, the position of the moving camera 20 mounted on the moving body 2 is not fixed. Therefore, the shooting area of ​​the moving camera 20 is also not fixed and can be set flexibly.

[0019] The moving object 2 transmits not only the image MIMG captured by the moving camera 20 but also information on the moving camera position MPOS to the management system 100. The management system 100 communicates with the moving object 2. This allows the management system 100 to acquire the image MIMG captured by the moving camera 20 and information on the moving camera position MPOS. The management system 100 may store the acquired information on the image MIMG and the moving camera position MPOS in a database.

[0020] The management system 100 may communicate with the still camera 10 and acquire images SIMG captured by the still camera 10. The images SIMG may also be stored in a database in the same manner.

[0021] The management system 100 also holds information on the target area of ​​the area management process. Typically, the target area is designated in advance by an administrator. The management system 100 executes area management processing to manage the target area based on information acquired from a group of cameras (still cameras 10, mobile cameras 20).

[0022] In the example shown in Fig. 2, the target area includes at least an area AR-X that is not covered by the stationary cameras 10-1 and 10-2. Such an area AR-X (target area) is not photographed by the stationary camera 10, but may be photographed by the mobile camera 20. The management system 100 can determine whether the area AR-X has been photographed by the mobile camera 20 by comparing the mobile camera position MPOS with the area AR-X. If the area AR-X has been photographed by the mobile camera 20, the management system 100 can obtain an image MIMG of the area AR-X.

[0023] The image MIMG of the target area captured by the mobile camera 20 is hereinafter referred to as the "target image TIMG." The target image TIMG can also be said to be the image MIMG captured when the mobile camera 20 is capturing the target area. As described above, the management system 100 can extract the target image TIMG based on a comparison between the moving camera position MPOS and the target area. Then, the management system 100 executes an area management process for managing the target area based on the target image TIMG of the target area.

[0024] <Effects> As described above, according to this embodiment, the moving camera 20 is used for the area management process. More specifically, in addition to the image MIMG captured by the moving camera 20, the moving camera position MPOS at the time of image capture is acquired. Based on the moving camera position MPOS, the image MIMG of the target area captured by the moving camera 20 is extracted as a target image TIMG. By using the extracted target image TIMG, it is possible to perform area management process for the target area.

[0025] Since the shooting area of ​​the moving camera 20 used in the area management process is not fixed, it is possible to set the target area to be the subject of the area management process more flexibly. Even if the target area is not covered by the still camera 10, it is possible to cover the target area with the moving camera 20. Therefore, the accuracy of the area management process can be improved.

[0026] Furthermore, according to this embodiment, there is no need to increase the number of still cameras 10 in order to expand the target area. Since the shooting area of ​​the moving cameras 20 is not fixed but can be set flexibly, it is possible to greatly expand the target area even when only a small number of moving cameras 20 are used. In other words, according to this embodiment, it is possible to expand the target area of ​​the area management process easily and at low cost.

[0027] The management system 100 according to this embodiment will be described in more detail below.

[0028] 2. Management system configuration example FIG. 3 is a block diagram showing a configuration example of a management system 100 according to the present embodiment. The management system 100 includes one or more processors 110 (hereinafter simply referred to as "processor 110"), one or more storage devices 120 (hereinafter simply referred to as "storage device 120"), and an interface 130. The processor 110 executes various processes. For example, the processor 110 includes a CPU (Central Processing Unit). The storage device 120 stores various information required for the processes. Examples of the storage device 120 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, and a non-volatile memory. The interface 130 includes a network interface and a user interface. Examples of the user interface include a display device, a touch panel, a keyboard, and a button.

[0029] The management program 200 is a computer program that performs area management processing. The management program 200 is stored in the storage device 120. The management program 200 may be recorded in a computer-readable recording medium. The management program 200 is executed by the processor 110. The functions of the management system 100 are realized by cooperation between the processor 110 that executes the management program 200 and the storage device 120.

[0030] The processor 110 communicates with the camera group via the interface 130. The camera group includes still cameras 10-i (i=1 to N) and moving cameras 20-j (j=1 to M), where N is an integer equal to or greater than 1, and M is an integer equal to or greater than 1. "i" is an identifier of the still camera 10-i and takes a value from 1 to N. "j" is an identifier of the moving camera 20-j and takes a value from 1 to M.

[0031] The processor 110 acquires an image SIMG-i captured by the still camera 10-i and a time stamp STS-i. The time stamp STS-i is the time when the image SIMG-i was captured, and is associated with the image SIMG-i. The processor 110 may acquire a still camera ID that is identification information of the still camera 10-i. The processor 110 may acquire information on a still camera position SPOS-i that is an installation position (fixed position) of the still camera 10-i.

[0032] The processor 110 also acquires an image MIMG-j captured by the moving camera 20-j and a timestamp MTS-j. The timestamp MTS-j is the time when the image MIMG-j was captured, and is associated with the image MIMG-j. The processor 110 also acquires information on a moving camera position MPOS-j, which is the position of the moving object 2-j when the image MIMG-j was captured. The moving camera position MPOS-j is associated with the image MIMG-j. The processor 110 may acquire a moving camera ID, which is identification information of the moving camera 20-j.

[0033] The processor 110 accumulates the acquired images and information in an image database 300. The image database 300 is stored in the storage device 120.

[0034] 4 is a conceptual diagram showing an example of the image database 300. In the example shown in FIG. 4, the image database 300 includes still camera image data 310 and moving camera image data 320.

[0035] The still camera image data 310 indicates the correspondence between the still camera ID, the image SIMG-i, and the time stamp STS-i. The still camera image data 310 may further include the still camera position SPOS-i. The still camera position SPOS-i may be registered in advance by the user, or may be notified from the moving camera 20-j. Preferably, the still camera position SPOS-i is a position in an absolute coordinate system, and is a position that can be represented on a map. In this case, it becomes possible to treat the still camera 10 and the moving camera 20 without distinction.

[0036] The moving camera image data 320 indicates the correspondence between the moving camera ID, the moving camera position MPOS-j, the image MIMG-j, and the time stamp MTS-j. Note that the moving camera position MPOS-j is a position in an absolute coordinate system, and is a position that can be expressed on a map.

[0037] The processor 110 executes an area management process for managing a target area by using the image database 300. An example of the area management process will be described below.

[0038] 3. Example of area management processing 3-1. First Example 5 is a functional block diagram for explaining a first example of the area management process. The processor 110 includes a target image extraction unit 111, an area management processing unit 112, and an output unit 113 as functional blocks.

[0039] The target image extraction unit 111 acquires information on the target area TAR. Typically, the target area TAR is designated in advance by an administrator. The processor 110 extracts the image MIMG of the target area TAR captured by the moving camera 20 as the target image TIMG. More specifically, the processor 110 accesses the image database 300 and extracts the image MIMG of the target area TAR from the image database 300 (moving camera image data 320) as the target image TIMG. As described above, the moving camera image data 320 of the image database 300 includes the moving camera position MPOS-j when the image MIMG-j was captured. Thus, the processor 110 can extract the target image TIMG based on a comparison between the moving camera position MPOS-j and the target area.

[0040] Note that the target image extraction unit 111 does not extract non-target images from the image database 300. Non-target images are images MIMG other than the target images TIMG. By not extracting non-target images, it is possible to reduce the processing load and memory consumption.

[0041] The area management processing unit 112 executes the area management process based on the target image TIMG. For example, the area management process includes monitoring the target area TAR based on the target image TIMG. The monitoring includes detection of anomalies (e.g., accidents, troubles, crimes, suspicious people, sick people, etc.). As another example, the area management process includes detection or search for targets (e.g., people, events) within the target area TAR based on the target image TIMG. In either case, anomalies or targets can be detected from the target image TIMG by utilizing a machine learning model.

[0042] The output unit 113 presents the results of the area management processing performed by the area management processing unit 112 to the user via a user interface.

[0043] 3-2. Second Example 6 is a functional block diagram for explaining a second example of the area management process. Descriptions that overlap with the first example shown in FIG. 5 will be omitted as appropriate.

[0044] The target image extraction unit 111 acquires information on the target time zone THR in addition to information on the target area TAR. Typically, the target time zone THR is designated in advance by an administrator. The processor 110 extracts an image MIMG of the target area TAR captured by the mobile camera 20 in the target time zone THR as the target image TIMG. More specifically, the processor 110 accesses the image database 300 and extracts an image MIMG of the target area TAR in the target time zone THR from the image database 300 as the target image TIMG. As described above, the mobile camera image data 320 of the image database 300 includes the mobile camera position MPOS-j and the time stamp MTS-j when the image MIMG-j was captured. Thus, the processor 110 can extract the target image TIMG based on a comparison between the mobile camera position MPOS-j and the target area, and a comparison between the time stamp MTS-j and the target time zone THR.

[0045] The rest is the same as in the first example shown in FIG. [Explanation of symbols]

[0046] 10. Still Camera 20 Moving Camera 100 Management Systems 200 Management Program 300 Image Database 310 Still camera image data 320 Moving camera image data MIMG Images MPOS moving camera position

Claims

1. One or more processors; The one or more processors: Communicating with a mobile object having a self-location function, Acquire information on an image captured by a moving camera mounted on the moving body and a position of the moving camera, which is the position of the moving body when the image was captured; extracting the image of the target area captured by the moving camera as a target image based on the moving camera position; An area management process is performed to manage the target area based on the target image. It was configured as Management system.

2. The management system according to claim 1, and one or more storage devices for storing the image database; The one or more processors further include: storing, in the image database, moving camera image data indicating a correspondence between the images and the moving camera positions; Extracting the image of the target area from the image database as the target image based on the moving camera position. It was configured as Management system.

3. The management system according to claim 1, The one or more processors further include: obtaining the image, the information of the moving camera position, and the time stamp information of the image; Based on the moving camera position and the time stamp, the image of the target area captured by the moving camera during the target time period is extracted as the target image. It was configured as Management system.

4. The management system according to claim 3, and one or more storage devices for storing the image database; The one or more processors further include: storing, in the image database, moving camera image data indicating a correspondence between the image, the moving camera position, and the time stamp; Based on the moving camera position and the time stamp, the image of the target area in the target time period is extracted as the target image from the image database. It was configured as Management system.

5. The management system according to claim 2 or 4, a non-target image is an image other than the target image; The one or more processors extract only the target images from the image database without extracting the non-target images, and execute the area management process. Management system.

Citation Information

Patent Citations

  • Method and device for retrieving moving object image

    JP2001195565A

  • Visual observer for unmanned aerial vehicles

    US20210350162A1

  • Railway vehicle monitoring system

    WO2020153483A1

  • Audit assistance device, audit assistance system, audit assistance method, and audit assistance program

    WO2022070533A1

  • Image recognition system, image recognition device, image recognition method and computer program

    JP2016001447A