Management system
Through the image and position information captured by the mobile camera, the object image is extracted for area management, which solves the problem of insufficient management accuracy of fixed area management of the still camera, and achieves flexible and efficient area coverage, reducing the cost of expanding the object area.
Patent Information
- Application Number
- CN202410436532.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-24
- Filing Date
- 2024-04-11
- Publication Date
- 2024-11-26
AI Technical Summary
When using a still camera for area management, the object area is fixed and limited, resulting in insufficient accuracy of area management processing, and expanding the object area requires increasing the number of still cameras, which consumes huge costs.
The mobile camera is used for area management, and the image and position information captured by the mobile camera are extracted to perform area management processing, and the area that is not covered by the still camera is used to cover the flexibility of the mobile camera.
The accuracy of area management processing is improved, and the target area can be greatly expanded without increasing the number of still cameras, achieving low-cost and flexible area management.
Smart Images

Figure CN119027463A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a technology for managing an object area based on an image captured by a camera. Background Art
[0002] Patent document 1 discloses an image recognition system for tracking a person using images taken by multiple cameras. Multiple cameras take pictures of a specific person from different directions. The specific person is tracked by using multiple images taken from different directions. The position information of each of the multiple cameras is used to calculate the relative angles between the multiple cameras. Prior art literature Patent Literature
[0003] Patent Document 1: Japanese Patent Application Publication No. 2016-001447 Summary of the invention Technical problem to be solved by the invention
[0004] Consider the area management process for managing the target area based on the image captured by the camera. When only a stationary camera (fixed camera) is used as the camera, the shooting area is fixed. That is, the target area to be the target of the area management process is fixed and limited. In this case, the area management process may not be able to obtain sufficient accuracy. Increasing the number of stationary cameras in order to expand the target area requires a huge amount of effort and cost.
[0005] An object of the present disclosure is to provide a technology capable of flexibly setting a target area when managing the target area based on an image captured by a camera. Means for solving technical problems
[0006] One aspect of the present disclosure relates to a management system. The management system has one or more processors. One or more processors communicate with a mobile object that has its own position determination capability. One or more processors obtain an image captured by a moving camera mounted on a moving object and information on the position of the moving camera, which is the position of the moving object when the image was captured. The one or more processors extract an image of the object area captured by the mobile camera as the object image based on the mobile camera position. The one or more processors perform an area management process for managing an object area based on the object image. Effects of the Invention
[0007] According to the present disclosure, a mobile camera is used for region management processing. More specifically, in addition to obtaining an image captured by the mobile camera, the position of the mobile camera when the image was captured is also obtained. Based on the mobile camera position, an image of a target region captured by the mobile camera is extracted as a target image. By using the extracted target image, region management processing related to the target region can be performed.
[0008] Since the shooting area of the mobile camera used in the area management process is not fixed, the target area to be the subject of the area management process can be set more flexibly. Even if the target area is not covered by the stationary camera, the target area can be covered by the mobile camera. Therefore, the accuracy of the area management process can be improved.
[0009] In addition, according to the present disclosure, it is not necessary to increase the number of stationary cameras in order to expand the target area. Since the shooting area of the mobile camera is not fixed but can be flexibly set, the target area can be greatly expanded even when only a few mobile cameras are used. That is, according to the present disclosure, the target area of the area management process can be expanded at a low cost and simply. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 This is a conceptual diagram for explaining a comparative example. Figure 2 This is a conceptual diagram for explaining the outline of the embodiment. Figure 3 This is a block diagram showing a configuration example of a management system according to the embodiment. Figure 4 This is a conceptual diagram showing an example of an image database according to the embodiment. Figure 5 This is a functional block diagram for explaining a first example of the area management process according to the embodiment. Figure 6 This is a functional block diagram for explaining a second example of the area management process according to the embodiment. DETAILED DESCRIPTION
[0011] Embodiments of the present disclosure will be described with reference to the drawings.
[0012] 1. Summary Consider "region management processing" for managing an object area based on images captured by a camera. For example, the region management processing includes monitoring the object area based on images. Monitoring includes detection of anomalies (e.g., accidents, disputes, crimes, suspicious persons, patients, etc.). As another example, the region management processing includes detecting or retrieving targets (e.g., people, events) within the object area based on images. Person re-identification (Person Re-identification) that identifies and tracks the same person from multiple images captured by multiple cameras is also included in the region management processing. In either case, anomalies and targets can be detected from images by using machine learning models.
[0013] First, refer to Figure 1 A comparative example is described. In the comparative example, only a still camera 10 (fixed camera) whose installation position is fixed is used for the area management process. The management system communicates with the still camera 10 and obtains the image SIMG taken by the still camera 10. Then, the management system performs the area management process based on the image SIMG. For example, the management system stores the image SIMG and retrieves a specific person P from the stored image SIMG. As another example, the management system can also perform person re-identification based on multiple images taken by multiple still cameras 10.
[0014] exist Figure 1 In the example shown, the area management process is performed based on the images SIMG-1 and SIMG-2 captured by the stationary cameras 10-1 and 10-2. The shooting areas AR-1 and AR-2 are fixed areas that can be captured by the stationary cameras 10-1 and 10-2, respectively. That is, the object areas that become the object of the area management process are only the fixed shooting areas AR-1 and AR-2. Since the object areas are fixed and limited, the area management process may not be able to obtain sufficient accuracy. For example, when a person P leaves the shooting areas AR-1 and AR-2, the person P cannot be detected.
[0015] In this way, when only the still camera 10 is used, the target area to be the target of the area management process is fixed and limited. In this case, the area management process may not be able to obtain sufficient accuracy. In order to expand the target area, increasing the number of still cameras 10 requires a huge amount of effort and cost. Therefore, this embodiment proposes a technology that can set the target area more flexibly without increasing the number of still cameras 10.
[0016] Figure 2This is a conceptual diagram for explaining the outline of the present embodiment. According to the present embodiment, a mobile camera 20 mounted on a mobile body 2 is used in addition to or instead of the still camera 10. The mobile body 2 can move freely around. Examples of such a mobile body 2 include vehicles, robots, flying bodies, etc. The vehicle may be an autonomous vehicle or a vehicle driven by a driver. Examples of robots include logistics robots, etc. Examples of flying bodies include airplanes, drones, etc.
[0017] In addition, the mobile body 2 has a self-position determination function and can obtain its own position information. For example, the mobile body 2 obtains position information using GNSS (Global Navigation Satellite System). In the present embodiment, the position of the mobile body 2 is regarded as the position of the mobile camera 20. Hereinafter, the position of the mobile body 2, that is, the position of the mobile camera 20 is referred to as the "mobile camera position MPOS". The mobile camera position MPOS is a position in an absolute coordinate system and is a position that can be represented on a map.
[0018] Unlike the stationary camera 10, the position of the moving camera 20 mounted on the moving object 2 is not fixed. Therefore, the imaging area of the moving camera 20 is also not fixed and can be flexibly set.
[0019] The mobile body 2 not only sends the image MIMG taken by the mobile camera 20 to the management system 100, but also sends the information of the mobile camera position MPOS to the management system 100. The management system 100 communicates with the mobile body 2. Thus, the management system 100 can obtain the image MIMG taken by the mobile camera 20 and the information of the mobile camera position MPOS. The management system 100 can also store the obtained information of the image MIMG and the mobile camera position MPOS in a database.
[0020] The management system 100 may communicate with the still camera 10 to acquire the image SIMG photographed by the still camera 10. The image SIMG may also be stored in the database in the same manner.
[0021] The management system 100 also holds information on a target area for area management processing. Typically, the target area is specified in advance by a manager. The management system 100 executes area management processing for managing the target area based on information obtained from the camera group (stationary camera 10, mobile camera 20).
[0022] exist Figure 2In the example shown, the target area includes at least an area AR-X that is not covered by the stationary cameras 10-1 and 10-2. Such an area AR-X (target area) is not photographed by the stationary camera 10, but may be photographed by the mobile camera 20. The management system 100 can determine whether the area AR-X is photographed by the mobile camera 20 by comparing the mobile camera position MPOS with the area AR-X. In the case where the area AR-X is photographed by the mobile camera 20, the management system 100 can obtain the image MIMG of the area AR-X.
[0023] Hereinafter, the image MIMG of the object area photographed by the mobile camera 20 is referred to as the "object image TIMG". The object image TIMG can also be said to be the image MIMG when the mobile camera 20 is photographing the object area. As described above, the management system 100 can extract the object image TIMG based on the comparison between the mobile camera position MPOS and the object area. Then, the management system 100 performs the area management processing for managing the object area based on the object image TIMG of the object area.
[0024] <Effect> As described above, according to the present embodiment, the mobile camera 20 is used for the area management process. More specifically, in addition to acquiring the image MIMG captured by the mobile camera 20, the mobile camera position MPOS when the image was captured is also acquired. Based on the mobile camera position MPOS, the image MIMG of the target area captured by the mobile camera 20 is extracted as the target image TIMG. By using the extracted target image TIMG, the area management process related to the target area can be performed.
[0025] Since the shooting area of the mobile camera 20 used in the area management process is not fixed, the target area to be the target of the area management process can be set more flexibly. Even if the target area is not covered by the stationary camera 10, the target area can be covered by the mobile camera 20. Therefore, the accuracy of the area management process can be improved.
[0026] In addition, according to the present embodiment, it is not necessary to increase the number of stationary cameras 10 in order to expand the target area. Since the shooting area of the mobile camera 20 is not fixed but can be flexibly set, the target area can be greatly expanded even when only a small number of mobile cameras 20 are used. That is, according to the present embodiment, the target area of the area management process can be expanded at low cost and simply.
[0027] Hereinafter, the management system 100 according to the present embodiment will be described in more detail.
[0028] 2. Example of management system configuration Figure 31 is a block diagram showing an example of the configuration of the management system 100 involved in the present embodiment. The management system 100 includes one or more processors 110 (hereinafter referred to as "processor 110"), one or more storage devices 120 (hereinafter referred to as "storage device 120"), and an interface 130. The processor 110 performs various processes. For example, the processor 110 includes a CPU (Central Processing Unit). The storage device 120 stores various information required for the processing. Examples of the storage device 120 include a HDD (Hard Disk Drive), an SSD (Solid State Drive), a volatile memory, a non-volatile memory, and the like. The interface 130 includes a network interface and a user interface. As a user interface, a display device, a touch panel, a keyboard, a button, and the like are exemplified.
[0029] The management program 200 is a computer program for performing area management processing. The management program 200 is stored in the storage device 120. The management program 200 may also be recorded in a computer-readable recording medium. The management program 200 is executed by the processor 110. The functions of the management system 100 are realized through the cooperation between the processor 110 executing the management program 200 and the storage device 120.
[0030] The processor 110 communicates with the camera group via the interface 130. The camera group includes a stationary camera 10-i (i=1 to N) and a mobile camera 20-j (j=1 to M). Here, N is an integer greater than 1, and M is an integer greater than 1. "i" is an identifier of the stationary camera 10-i, and takes a value from 1 to N. j is an identifier of the mobile camera 20-j, and takes a value from 1 to M.
[0031] The processor 110 obtains the image SIMG-i and the time stamp STS-i taken by the still camera 10-i. The time stamp STS-i is the time when the image SIMG-i is taken and is associated with the image SIMG-i. The processor 110 may also obtain the still camera ID as identification information of the still camera 10-i. The processor 110 may also obtain information of the still camera position SPOS-i as the installation position (fixed position) of the still camera 10-i.
[0032] In addition, the processor 110 obtains the image MIMG-j and the timestamp MTS-j taken by the mobile camera 20-j. The timestamp MTS-j is the time when the image MIMG-j was taken, and is associated with the image MIMG-j. In addition, the processor 110 obtains information on the mobile camera position MPOS-j, which is the position of the mobile body 2-j when the image MIMG-j was taken. The mobile camera position MPOS-j is associated with the image MIMG-j. The processor 110 may also obtain the mobile camera ID, which is identification information of the mobile camera 20-j.
[0033] The processor 110 stores the acquired images and information in the image database 300 . The image database 300 is stored in the storage device 120 .
[0034] Figure 4 3 is a conceptual diagram showing an example of the image database 300. Figure 4 In the example shown, image database 300 includes still camera image data 310 and moving camera image data 320 .
[0035] The still camera image data 310 shows the correspondence between the still camera ID, the image SIMG-i, and the time stamp STS-i. The still camera image data 310 may also include the still camera position SPOS-i. The still camera position SPOS-i may be pre-registered by the user or may be notified from the mobile camera 20-j. Preferably, the still camera position SPOS-i is a position in an absolute coordinate system and is a position that can be represented on a map. In this case, the still camera 10 and the mobile camera 20 can be processed without distinction.
[0036] The mobile camera image data 320 shows the correspondence between the mobile camera ID, the mobile camera position MPOS-j, the image MIMG-j, and the time stamp MTS-j. The mobile camera position MPOS-j is a position in an absolute coordinate system and can be represented on a map.
[0037] The processor 110 executes a region management process for managing the target region by using the image database 300. An example of the region management process will be described below.
[0038] 3. Example of regional management processing 3-1. The first example Figure 5 1 is a functional block diagram for explaining a first example of the area management process. The processor 110 includes a target image extraction unit 111, an area management processing unit 112, and an output unit 113 as functional blocks.
[0039] The object image extraction unit 111 obtains information of the object area TAR. Typically, the object area TAR is pre-specified by the administrator. The processor 110 extracts the image MIMG of the object area TAR captured by the mobile camera 20 as the object image TIMG. In more detail, the processor 110 accesses the image database 300 and extracts the image MIMG of the object area TAR from the image database 300 (mobile camera image data 320) as the object image TIMG. As described above, the mobile camera image data 320 of the image database 300 includes the mobile camera position MPOS-j when the image MIMG-j was captured. Therefore, the processor 110 can extract the object image TIMG based on the comparison between the mobile camera position MPOS-j and the object area.
[0040] In addition, the target image extraction unit 111 does not extract non-target images from the image database 300. Non-target images are images MIMG other than the target image TIMG. By not extracting non-target images, the processing load can be reduced and the memory consumption can be reduced.
[0041] The area management processing unit 112 performs area management processing based on the object image TIMG. For example, the area management processing includes monitoring the object area TAR based on the object image TIMG. Monitoring includes detection of abnormalities (e.g., accidents, disputes, crimes, suspicious persons, patients, etc.). As another example, the area management processing includes detecting or retrieving targets (e.g., people, events) within the object area TAR based on the object image TIMG. In either case, abnormalities and targets can be detected from the object image TIMG by using a machine learning model.
[0042] The output section 113 presents the result of the area management process performed by the area management processing section 112 to the user via the user interface.
[0043] 3-2. Second example Figure 6 This is a functional block diagram for explaining a second example of area management processing. Figure 5 The instructions for the first example shown are repeated.
[0044] In addition to obtaining information about the object area TAR, the object image extraction unit 111 also obtains information about the object time period THR. Typically, the object time period THR is pre-specified by an administrator. The processor 110 extracts the image MIMG of the object area TAR captured by the mobile camera 20 in the object time period THR as the object image TIMG. In more detail, the processor 110 accesses the image database 300 and extracts the image MIMG of the object area TAR in the object time period THR from the image database 300 as the object image TIMG. As described above, the mobile camera image data 320 of the image database 300 includes the mobile camera position MPOS-j and the timestamp MTS-j when the image MIMG-j was captured. Therefore, the processor 110 can extract the object image TIMG based on the comparison of the mobile camera position MPOS-j with the object area, and the comparison of the timestamp MTS-j with the object time period THR.
[0045] Other Figure 5 The situation is the same as in the first example shown. [Explanation of Reference Numerals]
[0046] 10. Still Camera 20. Move the camera 100 Management System 200 Management Program 300 Image Database 310 Still camera image data 320 mobile camera image data MIMG Image MPOS Mobile Camera Position
Claims
1. A management system, Having one or more processors, The one or more processors are configured to: Communicate with mobile objects that have their own location determination capabilities, acquiring an image captured by a mobile camera mounted on the mobile object and information on the position of the mobile camera, which is the position of the mobile object when the image was captured; extracting the image of the object area photographed by the mobile camera as an object image based on the mobile camera position, An area management process of managing the object area is performed based on the object image.
2. The management system according to claim 1, wherein: It also has one or more storage devices for storing the image database, The one or more processors are further configured to: storing mobile camera image data showing a correspondence between the image and the position of the mobile camera in the image database, Based on the mobile camera position, the image of the object area is extracted from the image database as the object image.
3. The management system according to claim 1, wherein: The one or more processors are further configured to: obtaining the image, the information of the position of the mobile camera, and the information of the timestamp of the image, Based on the mobile camera position and the time stamp, the image of the object area photographed by the mobile camera in an object time period is extracted as the object image.
4. The management system according to claim 3, wherein: It also has one or more storage devices for storing the image database, The one or more processors are further configured to: storing mobile camera image data showing a corresponding relationship between the image, the mobile camera position and the time stamp in the image database, The image of the object area in the object time period is extracted from the image database as the object image based on the mobile camera position and the time stamp.
5. The management system according to claim 2 or 4, wherein: The non-target image is the image other than the target image. The one or more processors perform the area management process by extracting only the object image without extracting the non-object image from the image database.
