Information processing deice, image analysis system, control method and program for information processing device

Abstract

To provide a technique for determining whether a vehicle can be detected satisfactorily at a current imaging position of an imaging apparatus.SOLUTION: An information processing device comprises: setting means for setting one or more vehicle space regions indicating parking frames in a picked-up image of a parking lot; determination means for determining each vehicle region indicating a region of a vehicle, which can be parked, with respect to each vehicle space region based on information of the vehicle space regions; superposition means for superposing the vehicle region on the vehicle space region; region estimation means for estimating a multistorey vehicle region indicating the vehicle on the basis of the vehicle region; estimation means for estimating a size of each multistorey vehicle region and a superposing degree of the multistorey vehicle regions; and determination means for determining, on the basis of the size and the superposing degree, in a case where a vehicle is parked in each vehicle space region, whether the vehicle can be detected or not.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an image analysis system, a control method for an information processing device, and a program. [Background technology]

[0002] There has long been a need to detect whether a vehicle (e.g., an automobile) is parked in each parking area of ​​a parking lot in order to determine the parking lot's congestion level and vacant parking status. One method for detecting the presence or absence of a vehicle in each parking area is to install a vehicle detection device such as an infrared sensor to detect objects. However, these vehicle detection devices generally have a limited detection range, and in large parking lots, the number of vehicle detection devices installed may become enormous. Another method for detecting the presence or absence of a vehicle in each parking area is to capture an image of the parking area with a camera and apply image processing to the captured image to detect whether a vehicle is parked there. This method has the advantage that by capturing images of multiple parking areas, the presence or absence of vehicles in multiple parking areas can be detected using an image captured by a single camera.

[0003] Patent Document 1 describes a technique for evaluating camera placement based on the camera position, angle of view, and imaging direction, as well as the probability of the presence of a moving obstacle in the captured image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-164080 [Non-patent literature]

[0005] [Non-Patent Document 1] J. Redmon, A. Farhadi, "YOLO9000:Better Faster Stronger", Computer Vision and Pattern Recognition (CVPR) 2016. Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology described in Patent Document 1 does not take into consideration the detection accuracy of the detection target, which is affected by the size and overlapping of the detection target. Therefore, the technology described in Patent Document 1 has a problem in that it is difficult to determine whether the current imaging position of the imaging device can satisfactorily detect vehicles parked in each parking area of ​​a parking lot, for example.

[0007] The present invention has been made in view of the above-mentioned problems, and has an object to provide a technique for determining whether a vehicle can be detected satisfactorily at the current imaging position of an imaging device. [Means for solving the problem]

[0008] To achieve the above object, an information processing device according to one aspect of the present invention comprises: A setting means for setting one or more vehicle compartment areas representing parking spaces in a captured image of the parking lot; A determination means for determining each vehicle area indicating an area where a vehicle can be parked for each vehicle area based on the information on the vehicle area; an overlapping means for overlapping the vehicle area with the passenger compartment area; an area estimation means for estimating a three-dimensional vehicle area showing the vehicle based on the vehicle area; an estimation means for estimating the size of each three-dimensional vehicle area and the degree of overlap between the three-dimensional vehicle areas; a determination means for determining whether a vehicle can be detected when the vehicle is parked in each vehicle compartment area based on the size and the overlapping state; The present invention is characterized by comprising: [Effects of the Invention]

[0009] According to the present invention, it is possible to determine whether a vehicle can be detected satisfactorily at the current imaging position of an imaging device. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example of a system configuration. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of an imaging apparatus. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of an imaging apparatus. [Figure 4] FIG. 2 illustrates an example of a hardware configuration of a server. [Figure 5] FIG. 2 illustrates an example of a functional configuration of a server. [Figure 6] FIG. 10 is a diagram illustrating an example of a setting screen. [Figure 7] FIG. 10 is a diagram illustrating an example of a vehicle interior setting screen. [Figure 8] FIG. 4 is a diagram showing an example of vehicle interior area information. [Figure 9] 10A and 10B are diagrams illustrating an example of an imaging device coordinate system and a bird's-eye coordinate system of a parking lot. [Figure 10] FIG. 10 is a diagram showing an example of an availability status display screen. [Figure 11] FIG. 10 is a diagram illustrating a flow of displaying an optimum imaging device position proposal screen. [Figure 12] FIG. 2 is a diagram illustrating a vehicle position. [Figure 13] FIG. 10 is a diagram illustrating a detection result estimation processing flow. [Figure 14] FIG. 10 is a diagram illustrating a detection result estimation process. [Figure 15] FIG. 10 is a diagram illustrating an example of a table for managing vehicle sizes and vehicle overlapping degrees. [Figure 16] 10A and 10B are diagrams illustrating the results of a detection result estimation process. [Figure 17] FIG. 10 is a diagram illustrating an example of a detection accuracy confirmation screen. [Figure 18] FIG. 10 is a diagram illustrating a flow of processing for determining an optimum imaging device position. [Figure 19]FIG. 10 is a diagram illustrating an optimum imaging device position proposal screen. [Figure 20] FIG. 10 is a diagram illustrating a second optimum imaging device position proposal screen. [Figure 21] FIG. 10 is a diagram illustrating a second detection result estimation processing flow. [Figure 22] FIG. 10 is a diagram illustrating a second detection result estimation process. [Figure 23] FIG. 10 is a diagram illustrating a second optimum imaging device position proposal screen display flow. [Figure 24] FIG. 10 is a diagram illustrating an example of a second detection accuracy confirmation screen. [Figure 25] FIG. 10 is a diagram illustrating a third optimum imaging device position proposal screen. [Figure 26] FIG. 10 is a diagram illustrating a fourth optimum imaging device position proposal screen. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] [First embodiment] <System configuration> FIG. 1 shows an example of the configuration of an image analysis system 101 according to this embodiment. In the following, a case where this system is a parking situation determination system will be described as an example. However, this is not limited to this, and this embodiment can be applied to any system that analyzes images and outputs predetermined information. The image analysis system 101 is configured to include imaging devices 110a to 110d, a network 120, and a server 130. Note that, in the following, the imaging devices 110a to 110d will be referred to as "imaging devices 110."

[0013] The imaging device 110 is an imaging device such as a network camera. In this embodiment, the imaging device 110 is assumed to have a built-in arithmetic device capable of processing video, but is not limited to this. For example, there may be an external computer such as a PC (personal computer) connected to the imaging device 110, and a combination of these may be treated as the imaging device 110. The server 130 is a computer such as a PC and has an image analysis processing function. Furthermore, the server 130 is a device that receives input from a user and outputs information to the user (for example, displays information).

[0014] The imaging device 110 and the server 130 are communicatively connected via a network 120. The network 120 is configured to include a plurality of routers, switches, cables, etc. that comply with a communication standard such as Ethernet (registered trademark). In this embodiment, the network 120 may be any network that enables communication between the imaging device 110 and the server 130, and can be constructed with any scale, configuration, and conforming communication standard.

[0015] For example, the network 120 may be the Internet, a wired local area network (LAN), a wireless LAN, a wide area network (WAN), etc. The network 120 may be configured to enable communication using a communication protocol conforming to the ONVIF (Open Network Video Interface Forum) standard, for example. However, the network 120 is not limited to this, and may be configured to enable communication using other communication protocols, such as a proprietary communication protocol.

[0016] <Device configuration> (Configuration of imaging device) Next, a description will be given of the configuration of the imaging device 110. Fig. 2 is a diagram showing an example of the hardware configuration of the imaging device 110. The imaging device 110 includes, as its hardware configuration, an imaging unit 201, an image processing unit 202, an arithmetic processing unit 203, and a distribution unit 204, for example.

[0017] The imaging unit 201 includes a lens unit for focusing light into an image, and an imaging element for converting the focused light into an analog signal. The lens unit has a zoom function for adjusting the angle of view, an aperture function for adjusting the amount of light, and the imaging element has a gain function for adjusting the sensitivity when converting light into an analog signal. These functions are adjusted based on setting values ​​notified by the image processing unit 202. The analog signal acquired by the imaging unit 201 is converted into a digital signal by an analog-to-digital conversion circuit, and transferred to the image processing unit 202 as an image signal.

[0018] The image processing unit 202 is configured to include an image processing engine and its peripheral devices. The peripheral devices include, for example, a RAM (Random Access Memory) and drivers for each I / F. Note that I / F is an abbreviation for interface. The image processing unit 202 performs image processing, such as development processing, filtering, sensor correction, and noise removal, on the image signal acquired from the imaging unit 201 to generate image data. The image processing unit 202 can also transmit setting values ​​to the lens unit and the imaging element and perform exposure adjustment so that an appropriately exposed image can be acquired. The image data generated in the image processing unit 202 is transferred to the arithmetic processing unit 203.

[0019] The arithmetic processing unit 203 is composed of one or more processors such as a CPU or MPU, memories such as RAM or ROM, drivers for each I / F, etc. Note that CPU is an acronym for Central Processing Unit, MPU for Micro Processing Unit, RAM for Random Access Memory, and ROM for Read Only Memory.

[0020] The distribution unit 204 includes a network distribution engine and peripheral devices such as a RAM and an ETH PHY module. The ETH PHY module is a module that executes processing at the physical (PHY) layer of Ethernet. The distribution unit 204 converts the image data and processing result data acquired from the arithmetic processing unit 203 into a format that can be distributed to the network 120, and outputs the converted data to the network 120.

[0021] 3 shows an example of the functional configuration of the imaging device 110. The imaging device 110 includes an imaging control unit 301, a signal processing unit 302, a storage unit 303, a control unit 304, an analysis unit 305, a device communication unit 306, and a network communication unit 307.

[0022] The imaging control unit 301 executes control to capture an image of the surrounding environment via the imaging unit 201. The signal processing unit 302 performs predetermined processing on the image captured by the imaging control unit 301 to generate captured image data. Hereinafter, this captured image data will be simply referred to as a "captured image." The signal processing unit 302 encodes, for example, the image captured by the imaging control unit 301. The signal processing unit 302 encodes the still image using an encoding method such as JPEG (Joint Photographic Experts Group).

[0023] Furthermore, the signal processing unit 302 encodes the moving image using an encoding method such as H.264 / MPEG-4 AVC (hereinafter referred to as "H.264"), HEVC (High Efficiency Video Coding), etc. Furthermore, the signal processing unit 302 may encode the image using an encoding method selected by a user from among a plurality of preset encoding methods via, for example, an operation unit (not shown) of the image capturing device 110.

[0024] The storage unit 303 stores temporary data for various processes. The control unit 304 controls the signal processing unit 302, storage unit 303, analysis unit 305, and network communication unit 306 so that each performs a predetermined process. The analysis unit 305 performs image analysis processing on the captured image. The network communication unit 306 communicates with the server 130 via the network 120.

[0025] (Server configuration) Fig. 4 shows an example of the hardware configuration of the server 130. The server 130 is configured as a computer such as a general PC, and, for example, as shown in Fig. 4, includes a processor 401 such as a CPU, memory such as a RAM 402 and a ROM 403, a storage device such as an HDD 404, and a communication I / F 405. The server 130 can perform various functions by the processor 401 executing programs stored in the memory or storage device.

[0026] 5 shows an example of the functional configuration of the server 130 according to this embodiment. The server 130 includes, as its functional configuration, for example, a network communication unit 501, a control unit 502, a display unit 503, an operation unit 504, an analysis unit 505, a storage unit 506, and a setting processing unit 507. The network communication unit 501 is connected to, for example, the network 120, and executes communication with an external device such as the image capture device 110 via the network 120. Note that this is merely an example, and for example, the network communication unit 501 may be configured to establish a direct connection with the image capture device 110 and communicate with the image capture device 110 without going through the network 120 or another device.

[0027] The control unit 502 controls the network communication unit 501, display unit 503, operation unit 504, analysis unit 505, storage unit 506, and setting processing unit 507 to execute their respective processes. The display unit 503 presents information to the user via, for example, a display. In this embodiment, the information is presented to the user by displaying the results of rendering by the browser on the display. Note that information may also be presented by a method other than a screen display, such as sound or vibration.

[0028] The operation unit 504 accepts operations from the user. In this embodiment, the operation unit 504 is a mouse or a keyboard, which the user operates to input user operations into the browser. However, the operation unit 504 is not limited to this, and may be any device capable of detecting the intention of other users, such as a touch panel or a microphone.

[0029] The analysis unit 505 executes vehicle detection processing on the captured image. In this embodiment, as an example, the vehicle detector used for the vehicle detection processing uses an object detection model trained by a technique called machine learning, which finds hidden patterns in a large amount of data through repeated calculations. For example, the technology described in Non-Patent Document 1 can be applied as the object detection model.

[0030] The analysis unit 505 receives an image as input and outputs the area in the image where the vehicle is located, its class, and the reliability of the detection. The area where the vehicle is located is represented by a rectangular area surrounding the vehicle called a bounding box. The storage unit 506 stores setting information and a vehicle detector, which will be described later. The setting processing unit 507 executes the following processes: cabin setting processing, displaying a screen proposing the optimal imaging device position, and displaying the availability status.

[0031] (Settings screen) FIG. 6 shows an example of the configuration of a setting screen for setting vehicle cabin settings, which are realized by the setting processing unit 507 of the server 130 according to this embodiment, a screen display proposing the optimum image capture device position, and settings related to vacancy status display.

[0032] The setting screen 600 includes a vehicle interior setting button 601, an optimum imaging device position display button 602, and an availability display button 603. Pressing each button transitions to a screen for performing each setting. The setting screen 600 and screens described below are displayed on the display unit 503 (display) of the server 130. Furthermore, each setting information set by the user via the operation unit 504 is saved in the saving unit 506 of the server 130.

[0033] The vehicle interior setting button 601 is a button for transitioning to a vehicle interior setting screen 700. Here, FIG. 7 is a diagram showing an example of the vehicle interior setting screen 700. In the vehicle interior setting screen 700, the position and size of each vehicle interior (parking space) relative to the captured image of the imaging device 110 are set and saved as a set of vertex coordinates (hereinafter referred to as a vehicle interior area). The captured image 701 is an image captured by the imaging device 110. In this embodiment, the captured image 701 is assumed to be an image of 1080 × 720 pixels. The vehicle interior area 702 is an example of a vehicle interior area. The vehicle interior area 702 is composed of vehicle interior vertex coordinates 703, 704, 705, and 706.

[0034] In this embodiment, the user sets the vertex coordinates by manually specifying four vertices of the vehicle interior area while viewing the captured image 701 displayed on the screen. Note that, although the vehicle interior area is set manually in this embodiment, a configuration may also be adopted in which the white lines of the parking area on the captured image 701 are detected using a Hough transform (known), and the vehicle interior area is detected with the intersections of the white lines as vertices.

[0035] FIG. 8 is a diagram showing an example of setting values ​​related to a vehicle compartment area (hereinafter referred to as vehicle compartment area information). The vehicle compartment area information 800 includes a vehicle compartment ID 801 and a vacant vehicle status 802. The vehicle compartment ID 801 is an identifier for identifying the vehicle compartment, and in this embodiment, vehicle compartment IDs 1 to 32 are set as values ​​for 32 vehicle compartment areas set by the user while viewing the captured image 701. The vacant vehicle status 802 stores the determination result of whether or not a vehicle is vacant, as described below, as information indicating whether each vehicle compartment is vacant or parked. Each vehicle compartment includes a vertex ID 803, an X coordinate 804, and a Y coordinate 805. The information on the vehicle compartment vertex coordinates 703 to 706 corresponds to records 806 to 809.

[0036] The optimum imaging device position display button 602 transitions to a detection accuracy confirmation screen and an optimum imaging device position proposal screen, which will be described later. Pressing this button executes the optimum imaging device position proposal screen display flow, which will be described later, and transitions to a detection accuracy confirmation screen and an optimum imaging device position proposal screen, which will be described later.

[0037] The availability display button 603 is a button for transitioning to an availability display screen. Here, Fig. 10 is a diagram showing an example of the availability display screen. The availability display screen 1000 is a screen that displays the availability of the vehicle compartment by displaying the results of vehicle detection processing by the analysis unit 505 of the server 130 from the captured images captured by the imaging devices 110a to 110d. A parking status 1001 indicates a state in which a vehicle is parked in the vehicle compartment. An empty status 1002 indicates a state in which no vehicle is parked in the vehicle compartment.

[0038] [Imaging device coordinate system and bird's-eye coordinate system] Next, the imaging device coordinate system and the bird's-eye coordinate system will be described. The plane coordinate system (X, Y) of an image captured by imaging device 110 (for example, captured image 701) is the imaging device coordinate system. The plane coordinate system (X, Y) when the parking lot is viewed from directly above is the bird's-eye coordinate system. The imaging device coordinate system is a different coordinate system for each of the images captured by imaging devices 110a to 110d.

[0039] An example of the image capture device coordinate system and the overhead coordinate system will be described using Figure 9. The image capture device coordinate system 900 is a planar coordinate system of the image captured by the image capture device 110a. The image capture device image 901 is an example in which the vehicle interior area information 800 is superimposed on the image capture device coordinate system. The overhead coordinate system 902 is a planar coordinate system when the parking lot is viewed from directly above. The overhead image 903 is an example of an image in which the vehicle interior area information 800 is converted into the overhead coordinate system 902 and superimposed.

[0040] When converted to the overhead coordinate system, the vehicle interior area 702 in the image capture device coordinate system corresponds to a vehicle interior area 904. Furthermore, vehicle interior vertex coordinates 703, 704, 705, and 706 in the image capture device coordinate system correspond to vehicle interior vertex coordinates 905, 906, 907, and 908 in the overhead coordinate system.

[0041] The image capture device coordinate system and the overhead coordinate system are aligned by calculating a homography transformation matrix (hereafter referred to as a transformation matrix) for performing a homography transform (planar projection transformation) between feature points on an image in the image capture device coordinate system and feature points on the overhead coordinate system. Homography transformation can transform and transfer any point on a planar coordinate system onto a different planar coordinate system. The homography transformation can be expressed as shown in the following equation (1) using the coordinate point (x1, y1) on the original image, the coordinate point (x2, y2) on the transformed image, a transformation matrix H, and a constant s.

[0042]

number

[0043] If similar objects exist between the two images, the parameters of the transformation matrix H are calculated based on the calculated coordinates of corresponding points between the images.

[0044] Next, an example of a method for calculating the transformation matrix H in this embodiment will be described. First, the transformation matrix H is calculated using feature points in the image capture device coordinate system and feature points in the overhead image system. Vehicle interior vertex coordinates 703, 704, 705, and 706 are values ​​set by the user on the vehicle interior setting screen 700. An arbitrary reference point (vertex 907 in this embodiment) is determined for vertex coordinates 905, 906, 907, and 908 in the overhead coordinate system. Then, based on this reference point, the other vertices are calculated from the size of a typical vehicle interior (length 5.5 m, width 3 m).

[0045] In this embodiment, the transformation matrix H is calculated using the vertex coordinates of both coordinate systems determined in this manner. Then, by performing homography transformation using the calculated transformation matrix H, it is possible to associate any point in the image capture device coordinate system with the overhead coordinate system. Furthermore, by using an inverse matrix, it is also possible to associate any point in the overhead coordinate system with the image capture device coordinate system.

[0046] The method of calculating the transformation matrix H is not limited to a method of having the user specify four points, since the calculation can be performed by specifying four points in the coordinate system of the original image and the coordinate system of the transformed image. For example, a Hough transform (known) or the like is used to detect the white lines in the vehicle interior of the captured image. Then, multiple points are calculated as feature points, which are edges where the white lines intersect. From the multiple points, four likely points are selected using RANSAC (known). Then, the transformation matrix H may be calculated based on the four points.

[0047] <Processing flow> Next, an example of a processing flow executed by the image analysis system 101 will be described. Each of the following processes is realized by the processor 401 of the server 130 executing a program stored in the RAM 402. However, this is merely an example, and some or all of the processes described below may be realized not only by the server 130 but also by the imaging device 110 or dedicated hardware.

[0048] (Screen display flow for suggesting optimal imaging device position) 11 is a diagram illustrating a flow of displaying a screen for proposing an optimal imaging device position, which is executed by the image analysis system 101. This processing flow is triggered when the user presses an optimal imaging device position display button 602 on the setting screen 600.

[0049] In step S1101, the setting processing unit 507 of the server 130 controls the network communication unit 501 via the control unit 502 to acquire the captured image 701 from the imaging device 110. In step S1102, the setting processing unit 507 sets one or more vehicle compartment areas indicating parking spaces in the captured image 701 of the parking lot, and stores the set areas in the storage unit 506 as vehicle compartment area information (e.g., vehicle compartment area information 800).

[0050] In step S1103, setting processing unit 507 transforms vehicle interior area information 800 in image capture device coordinate system 900 into a coordinate system of overhead-view coordinate system 902. This coordinate transformation is performed using a transformation matrix H between image capture device coordinate system 900 and overhead-view coordinate system 902. Image capture device coordinate system image 901 shown in Fig. 9 is an example in which each vehicle interior of vehicle interior area information 800 is superimposed on the image capture device coordinate system. Moreover, overhead-view coordinate system image 903 is an example of an image in which each vertex coordinate of vehicle interior area information 800 is transformed into bird's-eye view coordinate system 902 using transformation matrix H, and the transformed vehicle interiors are superimposed.

[0051] In step S1104, the setting processing unit 507 determines a vehicle area indicating the area of ​​vehicles that can be parked for each vehicle area based on the vehicle area information converted into the overhead coordinate system 902, and superimposes the vehicle area on the vehicle area. The vehicle area has fixed vehicle size (e.g., length 5.0 m, width 2.7 m, height 2.0 m) relative to the vehicle compartment size (length 5.5 m, width 3 m), and allocates coordinates for each vehicle compartment.

[0052] 12 is a diagram illustrating a vehicle area. A vehicle interior 1200 is an area in which a vehicle area 1201 is arranged and superimposed on the vehicle interior area 904. An overhead coordinate system image 1202 is an example of an image in which the vehicle area 1201 is superimposed on each vehicle interior of the overhead coordinate system image 903.

[0053] In step S1105, setting processing unit 507 transforms the vehicle coordinates in bird's-eye coordinate system 902 into image capture device coordinate system 900 using the inverse matrix of transformation matrix H. As shown in Fig. 12, image capture device coordinate system image 1203 is an example of an image in which the result of transforming the vehicle coordinates in bird's-eye coordinate system 902 into image capture device coordinate system 900 using the inverse matrix of transformation matrix H is superimposed on image capture device coordinate system image 901. Vehicle area 1204 is an area in which vehicle area 1201 is transformed into the image capture device coordinate system.

[0054] In step S1106, the setting processing unit 507 executes a process of determining whether or not a vehicle can be detected. Details of this process will be described with reference to FIG.

[0055] (Detection result estimation processing flow) Fig. 13 is a diagram illustrating in detail the processing flow for determining whether or not a vehicle can be detected in step S1106, and Fig. 14 is a diagram illustrating the detection result estimation processing.

[0056] In step S1301, the setting processing unit 507 of the server 130 performs vehicle position estimation processing to estimate a three-dimensional vehicle area indicating the vehicle. Vehicle position estimation result 1400 in FIG. 14 is a diagram illustrating the estimation result of the vehicle position relative to the vehicle cabin area 702. The vehicle position estimation area 1401 is a three-dimensional vehicle area (e.g., a rectangular parallelepiped area) where the vehicle is expected to be parked. The vehicle position estimation area 1401 is calculated as a three-dimensional vehicle area with the vehicle area 1204 as the base and connecting vertices equivalent to 2.0 m, which is the height of the vehicle, to each vertex. The vehicle position estimation area 1401 is the estimated area of ​​the vehicle position relative to the vehicle cabin area 702 of the image capture device 110a. In this step, the vehicle position estimation area of ​​each vehicle cabin is output as a result of performing the vehicle position estimation processing for all vehicle cabin areas. The vehicle position estimation processing result 1402 is an example in which the results of performing the vehicle position estimation processing for all vehicle cabin areas are superimposed on the image capture device coordinate system image 1203.

[0057] In step S1302, the setting processing unit 507 performs vehicle size estimation processing. The estimated vehicle size 1403 shown in Fig. 14 indicates the estimated vehicle size of the vehicle area 1204 (size of the three-dimensional vehicle area). In this embodiment, the circumscribing rectangle of the vehicle position estimation area 1401 is calculated, and the estimated vehicle size 1403 is expressed as the area of ​​the circumscribing rectangle. The circumscribing rectangle 1404 indicates the circumscribing rectangle of the vehicle position estimation area 1401. In this step, the estimated vehicle size of each vehicle compartment is output as a result of performing the vehicle size estimation processing for all vehicle compartments.

[0058] In step S1303, the setting processing unit 507 performs vehicle overlap estimation processing. In the vehicle overlap estimation processing, the degree of overlap of the circumscribing rectangles of two adjacent three-dimensional vehicle areas is estimated as the degree of overlap between the three-dimensional vehicle areas. The vehicle position estimation area 1405 shown in FIG. 14 is the vehicle position estimation area of ​​the vehicle cabin area adjacent to the vehicle position estimation area 1401. The vehicle circumscribing rectangle 1406 represents the circumscribing rectangle of the vehicle position estimation area 1405. The degree of vehicle overlap is represented by the overlap rate between the vehicle circumscribing rectangle 1404 and the vehicle circumscribing rectangle 1406. In this embodiment, the overlap rate is calculated using the IoU (Intersection over Union) with a vehicle area in front of the target vehicle area (a vehicle area that hides the target area). Whether a vehicle area is in front of or behind the target area is determined by the positional relationship of the coordinates of the vehicle areas. The IoU is 1 if all areas overlap, and 0 if there is no overlap at all. As a result of performing the vehicle overlap estimation processing for all vehicle cabins, the overlap rate between each vehicle cabin is output. It should be noted that this processing is not limited to circumscribing rectangles, and other shapes of figures may also be used.

[0059] In step S1304, the setting processing unit 507 executes a detection result estimation process based on the estimated vehicle state for each vehicle compartment (the size of each three-dimensional vehicle area and the degree of overlap between the three-dimensional vehicle areas) and the detection capability of the detector. The detection result estimation process is a process for determining whether a vehicle can be detected when parked in each vehicle compartment area, and can be determined by comparing two vehicle states, the vehicle size and the degree of overlap between the three-dimensional vehicle areas, with the corresponding table information.

[0060] FIG. 15 shows an example of a table for managing vehicle size and vehicle overlap. Vehicle size correspondence table 1500 is a table for managing the correspondence between vehicle size and detection accuracy. Size column 1501 indicates vehicle size in terms of area. Detectability column 1502 indicates whether or not a vehicle is detectable for that size. Reliability degradation column 1503 indicates the degree of deterioration in detection reliability. Record 1504 indicates that detection is possible for sizes 400 or larger, and the detection reliability degradation is 0. Record 1505 indicates that detection is possible for sizes 300, and the detection reliability degradation is 10. Record 1506 indicates that detection is possible for sizes 200, and the detection reliability degradation is -20. Record 1507 indicates that detection is possible for sizes 180, and the detection reliability degradation is -30. Record 1508 indicates that detection is difficult for sizes 150 or smaller.

[0061] It may be configured to determine that a vehicle is detectable when the size is equal to or larger than a predetermined size and the degree of overlap is equal to or smaller than a predetermined ratio. For example, in the example of Fig. 15, it can be determined that a vehicle is detectable when the size is equal to or larger than 180 and the degree of overlap is equal to or smaller than 0.3.

[0062] Next, vehicle overlap correspondence table 1509 is a table that manages the correspondence between vehicle overlap and detection accuracy. Overlap column 1510 indicates the overlap using the IoU between vehicles. Detectability column 1502 indicates whether detection is possible for that overlap. Reliability degradation column 1503 indicates the deterioration of detection reliability. Record 1513 indicates that detection is possible when the overlap is 0, and the detection reliability degradation is 0. Record 1514 indicates that detection is possible when the overlap is 0.1, and the detection reliability degradation is 10. Record 1515 indicates that detection is possible when the overlap is 0.2, and the detection reliability degradation is -20. Record 1516 indicates that detection is possible when the overlap is 0.3, and the detection reliability degradation is -30. Record 1517 indicates that detection is difficult when the overlap is 0.4 or less. For example, if the size is 400 or more and the overlap level is 0, it is detectable and the degradation of detection reliability is 0. Also, even if the size is 400 or more, if the overlap level is 0.4 or less, it is difficult to detect. Similarly, if the size is 180 and the overlap level is 0.3, both are detectable, but the sum of the degradation of detection reliability for both is -60, so even if it is detectable, it is expected that the accuracy will be significantly reduced.

[0063] Next, Fig. 16 is a diagram illustrating the results of the detection result estimation process. Vehicle size comparison result 1600 is an example of the result determined by comparing the vehicle size of each vehicle compartment with vehicle size correspondence table 1500, superimposed on each vehicle compartment in bird's-eye coordinate system image 903. Vehicle size detection difficulty result 1601 represents vehicle compartment areas where detection is difficult in terms of vehicle size. Vehicle size detection possible result 1602 represents vehicle compartment areas where detection is possible in terms of vehicle size.

[0064] The vehicle overlap degree comparison result 1603 is an example of the result determined by comparing the vehicle overlap degree of each vehicle compartment with the vehicle overlap degree correspondence table 1509 and superimposing it on each vehicle compartment of the overhead coordinate system image 903. The vehicle overlap degree detection difficulty result 1604 represents a vehicle compartment area where detection is difficult in terms of the vehicle overlap degree. The vehicle overlap degree detection possibility result 1605 represents a vehicle compartment area where detection is possible in terms of the vehicle overlap degree.

[0065] The detection result estimation processing result 1606 is an example in which the results of vehicle size and vehicle overlapping degree are superimposed on each vehicle interior of the overhead coordinate system image 903. The overall detection difficulty result 1607 represents a vehicle interior area that is overall difficult to detect. The overall detectable result 1608 represents a vehicle interior area that is overall detectable. The overall low detection reliability result 1609 represents a vehicle interior area that was detectable from the perspectives of both vehicle size and vehicle overlapping degree, but where the detection reliability is expected to be low overall even if it was detectable.

[0066] The integrated result area 1610 is an example in which the results of the detection result estimation process are used to divide the area into two, a difficult-to-detect area 1611 and a detectable area 1612, and the results are superimposed on each vehicle interior of the overhead coordinate system image 903.

[0067] Returning to FIG. 11 again, the flow of displaying the optimum imaging device position proposal screen will be described below.

[0068] In step S1107, the setting processing unit 507 displays a detection accuracy confirmation screen to present the determination result of step S1106. Here, FIG. 17 is a diagram illustrating an example of the detection accuracy confirmation screen. An overhead plan view 1700 is an example in which the detection result estimation processing result 1606 and the integrated result area 1610 are superimposed on an overhead coordinate system. An imaging plan view 1701 is an example in which the detection result estimation processing result 1606 is superimposed on an imaging device coordinate system. A captured image overlay view 1702 is an example in which the detection result estimation processing result 1606 is superimposed on a captured image. For example, as shown in FIG. 17, the positions of detectable vehicles and the positions of vehicles that are difficult to detect may be presented in different display forms (e.g., 1607 and 1609). Furthermore, the positions of detectable vehicles whose detection reliability is equal to or greater than a predetermined value and the positions of detectable vehicles whose detection reliability is less than the predetermined value may be presented in different display forms (for example, 1608 and 1609).

[0069] In step S1108, the setting processing unit 507 performs an optimum image capturing device position determination process to find the optimum image capturing device position. Details of this process will be described with reference to FIG.

[0070] (Optimal imaging device position determination process flow) FIG. 18 is a diagram illustrating in detail the optimum imaging device position determination process flow in step S1108.

[0071] In step S1801, the setting control unit 507 scores the total accuracy of the image capture device coordinate system 900. For example, the overall detection reliability (total accuracy) for all vehicle interior areas is estimated based on the detection reliability at the position of each vehicle interior area in the image capture device coordinate system 900. In this embodiment, when detection is possible and the detection reliability degradation is 0, the detection reliability is set to 100. Then, an evaluation value is determined by assuming that a detection reliability degradation of -10 is set to 90, and a detection reliability degradation of difficult detection is set to 0, and the total accuracy is scored based on the sum of the evaluation values ​​of all vehicle interior areas.

[0072] For example, suppose there is a parking lot with 10 spaces, designated spaces 1 to 10, and the evaluation values ​​(detection reliability) of spaces 1 to 8 are 100, the evaluation value of space 9 is 80, and the evaluation value of space 10 is 0. In this case, the total accuracy (overall detection reliability) can be calculated as follows: = average value of spaces 1 to 10 = (100 x 8 + 80 x 1 + 0 x 1) / 10 = 88. However, the total accuracy (overall detection reliability) is not limited to the average value, and the median or other values ​​may be used.

[0073] In step S1802, the setting control unit 507 estimates the imaging position (position, orientation, angle of view) of the imaging device using a (publicly known) technique for estimating the camera position and orientation from corresponding points in the imaging coordinate system and the bird's-eye coordinate system.

[0074] Thereafter, the imaging device position is virtually moved in the vertical and horizontal directions relative to the estimated imaging position of the imaging device (hereinafter referred to as the imaging device position), and the process of scoring the total accuracy (overall detection reliability) at each imaging device position is repeatedly performed.

[0075] In step S1803, the setting control unit 507 calculates a homography transformation matrix H based on the image capturing device coordinates corresponding to the image capturing device position and the coordinates when the image capturing device is virtually moved, and converts the vehicle interior area information of the image capturing device coordinates 900 into the vehicle interior area information of the moved image capturing device. After performing the processes of S1103 to S1801, the image capturing device position is virtually moved in the height direction and left / right direction relative to the image capturing position of the image capturing device, and the processes from S1803 are repeated.

[0076] Returning to FIG. 11 again, the next step in the optimum imaging device position proposal screen display flow will be described.

[0077] In step S1109, if there is an imaging device position that exceeds the initial total accuracy, the setting control unit 507 displays an optimal imaging device position proposal screen. That is, an imaging position that improves the total accuracy (overall detection reliability) compared to the current imaging position is proposed. Here, FIG. 19 is a diagram illustrating the optimal imaging device position proposal screen. A total accuracy display 1900 is an example of a display of total accuracy. In the illustrated example, the total accuracy is 80, with 10 vehicles having a accuracy of 90% or higher, 11 vehicles having an accuracy of 70-89%, and 10 vehicles having an accuracy of 50-69%.

[0078] Display screen 1901 is an example of a screen suggesting optimal imager position. Suggestion 1902 indicates that moving the imager upward will improve total accuracy from 80 to 85. Suggestion 1903 indicates that moving the imager left will improve total accuracy from 80 to 90.

[0079] As described above, in this embodiment, the detection accuracy of each detection target is estimated based on the vehicle interior area information in the captured image, according to the captured image state of the detection target and the detection capability of the system. Then, the detection accuracy for the captured image is scored based on the detection accuracy of each detection target. The estimated image capture device position is then moved based on the vehicle interior area information in the captured image to calculate a score value. Then, an installation position with a better score value is proposed. This allows for improved camera installation.

[0080] <Modification 1: Display comparison results> FIG. 20 is a diagram illustrating a second optimum imaging device position proposal screen. Comparative display example 2000 is an example of displaying a comparison between the estimated accuracy of the current position and the estimated accuracy when the imaging device is moved. Comparative display example 2000 displays how much the total accuracy improves when the imaging device is moved in the vertical direction, comparing the current total accuracy with the total accuracy after the movement. Detection result estimation processing result 2001 is an example of superimposing the results of the vehicle size and vehicle overlap state after movement in the vertical direction on each vehicle compartment of the bird's-eye view coordinate system image 903. Total accuracy display 2002 is an example of displaying the total accuracy after movement in the vertical direction. In this example, the total accuracy has improved from 80 to 85.

[0081] A comparative display example 2003 displays how much the total accuracy improves when the imaging device is moved left or right, comparing the current total accuracy with the total accuracy after the movement. A detection result estimation processing result 2004 is an example in which the results of the vehicle size and vehicle overlap after left or right movement are superimposed on each vehicle compartment of the overhead coordinate system image 903. A total accuracy display 2005 is an example display of the total accuracy after left or right movement. In this example, the total accuracy has improved from 80 to 90.

[0082] By comparing and displaying the current total accuracy with that after the move in this way, it is possible to propose an installation location that is easier for the user to understand.

[0083] <Modification 2: Filtering when estimated detection accuracy is low> When a detection process is performed on an input captured image, the detection results in the vehicle interior area where the estimation accuracy (detection reliability) is low are expected to be low in accuracy. Therefore, vehicle detection results in areas where the estimation accuracy is below a predetermined value may be filtered from the detection process. In other words, detection results in the vehicle interior area where the detection reliability is below a predetermined value may be filtered out. This can increase the adaptability (precision) of the detection process.

[0084] <Variation 3: Filtering out recognition results outside the area> Furthermore, when a detection process is performed on an input captured image, if a vehicle is detected outside the vehicle interior area, it is highly likely that the vehicle is not parked in the parking lot. Therefore, vehicle detection results detected outside the vehicle interior area may be filtered from the detection process results. In other words, if a vehicle is detected in an area other than the vehicle interior area, the detection results for that vehicle may be filtered and excluded. This can increase the adaptability (precision) of the detection process.

[0085] [Second embodiment] In the first embodiment, the detection accuracy of each detection target was estimated based on the vehicle interior area information in the captured image, depending on the imaging state of the detection target and the detection capability of the system. In contrast, in this embodiment, a process for estimating the detection accuracy of each detection target by placing a 3D vehicle on the image and performing detection processing based on the imaging state of the detection target will be described. The system configuration and device configuration are the same as in the first embodiment. The following mainly describes the differences from the first embodiment.

[0086] (Second detection result estimation processing flow) Fig. 21 is a diagram illustrating in detail the second processing flow for determining whether or not a vehicle can be detected in step S1106 according to this embodiment. Fig. 22 is a diagram illustrating the second detection result estimation processing. For example, on the setting screen 600 shown in Fig. 6, the optimum imaging device position proposal screen display flow is started when the user presses the optimum imaging device position display button 602, which is a trigger. The processing of steps S1301 to S1303 is the same as that of the first embodiment.

[0087] In step S2101, the control unit 502 performs a vehicle orientation estimation process. First, in the bird's-eye view coordinate system 902 shown in FIG. 9, the two short sides of the vehicle compartment area represent the front and rear of the parked vehicle, and the two long sides represent the sides of the vehicle. The short sides, transformed into the image capture device coordinate system 900, represent the front and rear of the vehicle in the parking position area 1204. The front-to-rear orientation of the vehicle can be determined based on the positional relationship between adjacent vehicle areas. The parking position sides 2200 and 2201 in FIG. 22 represent the front and rear of the vehicle in the parking position area 1204. For example, if one of the two short sides of the vehicle compartment area is included in part of an adjacent vehicle compartment area, it can be determined that the side is the rear and the other is the front. As a result of performing the vehicle orientation estimation process for all vehicle compartments, the estimated vehicle orientation for each vehicle compartment is output.

[0088] In step S2102, the setting processing unit 507 of the server 130 estimates a three-dimensional vehicle model (3D vehicle model) that resembles the vehicle as a three-dimensional vehicle area based on the vehicle orientation estimated in step S2101. Then, the 3D vehicle model is placed on the captured image 701 shown in FIG. 7. The setting processing unit 507 then executes a detection process on the image and sets the detection result as a detection estimate. That is, based on the detection result of the detection process, it is determined whether or not a vehicle can be detected when parked in each vehicle compartment area.

[0089] As shown in Fig. 22, the 3D vehicle arrangement screen 2202 is a screen on which the user can check and modify the vehicle arrangement. On the 3D vehicle arrangement screen 2202, the user can check the placed 3D vehicle models, move, enlarge or reduce, delete or add placed vehicles. Furthermore, the user can also change the orientation (front and rear) of the vehicles. Furthermore, the user can also change the type of vehicle of the 3D vehicle model.

[0090] The 3D vehicle arrangement image 2203 is an image in which a 3D vehicle model is superimposed on the captured image 701. The 3D vehicle 2204 is a 3D vehicle (three-dimensional vehicle), and in this embodiment, is a passenger car type vehicle model. Furthermore, the 3D vehicle 2204 can be changed using a vehicle model selection UI 2205. By selecting a light vehicle 2205 in the vehicle model selection UI 2205, the vehicle model can be changed to a light vehicle, and by selecting a truck 2207, the vehicle model can be changed to a truck. When the user presses the execute button 2208, the setting processing unit 507 executes detection processing on the 3D vehicle arrangement image 2203, and sets the result as a detection estimate value.

[0091] The 3D vehicle detection result view 2209 is an example in which the results of a detection process are superimposed on the 3D vehicle placement image 2203. An undetectable vehicle 2210 is an example in which the placed vehicle was not detected correctly. Here, "not detected correctly" refers to, for example, a case in which the IoU between the circumscribing rectangle of the placed vehicle in the image capture coordinate system and the bounding box of the object detection is below a threshold. A detectable vehicle 2211 is an example in which the placed vehicle was detected correctly. The detection confidence 2212 is an example in which the likelihood of the detection result is displayed.

[0092] As described above, in this embodiment, a 3D vehicle is placed on the image based on the imaging state of the detection target, and the detection process is performed. Furthermore, the detection result is used as an estimation result. This allows the user to understand the detection accuracy of the installation position.

[0093] [Third embodiment] In the first embodiment, an image captured by the imaging device 110a was used for explanation, but in this embodiment, an embodiment using a plurality of imaging devices (for example, 110a, 110b, and 110c) will be explained. The system configuration and device configuration are the same as those in the first embodiment. The following mainly explains the differences from the first embodiment.

[0094] (Second optimal imaging device position proposal screen display flow) Fig. 23 is a diagram illustrating a second optimum imaging device position proposal screen display flow executed by the image analysis system 101 according to this embodiment. For example, this processing flow is triggered by the user pressing the optimum imaging device position display button 602 on the setting screen 600 shown in Fig. 6. The processes of S1101 to S1106 and S1108 are the same as those described with reference to Fig. 11.

[0095] In step S2301, the setting processing unit 507 of the server 130 displays a detection accuracy check screen for checking the detection accuracy of a plurality of imaging devices. FIG. 24 is a diagram illustrating an example of the second detection accuracy check screen, illustrating the detection accuracy estimation results of a plurality of imaging devices. Estimation result image 2400 is an example of the estimation result for imaging device 110a. Imaging device 2401 represents the installation position and orientation of imaging device 110a. Estimation result image 2402 is an example of the estimation result for imaging device 110b. Imaging device 2403 represents the installation position and orientation of imaging device 110b. Estimation result image 2404 is an example of the estimation result for imaging device 110c. Imaging device 2405 represents the installation position and orientation of imaging device 110c. Estimation result summary image 2406 is an example of a result displaying a summary of the estimation results for imaging devices 110a, 110b, and 110c. As shown in the estimation result summary image 2406, the results of the determination of the detectability for each of the multiple imaging devices are summarized and presented. In step S2302, the setting processing unit 507 of the server 130 determines the combination that provides the best aggregation accuracy from the results of the optimum image capturing device position determination process (S1108) for the images of each image capturing device.

[0096] In step S2303, the setting processing unit 507 of the server 130 displays an optimal imaging device position proposal screen. FIG. 25 is a diagram illustrating a third optimal imaging device position proposal screen. The total accuracy display 2500 is a display example of an aggregated total accuracy (integrated detection reliability) obtained by aggregating the estimation results of the imaging devices 110a, 110b, and 110c. That is, the overall detection reliability (total accuracy) is estimated for each of the multiple imaging devices, and the multiple overall reliabilities are integrated to estimate and present an integrated detection reliability (aggregated total accuracy). In the illustrated example, the total accuracy is 95, with 28 devices at 90% or higher and 3 devices at 70-89%.

[0097] Display screen 2501 is an example of a screen that proposes optimal imaging device positions. Suggestion 2502 indicates that moving imaging device 2405 upward will improve the total accuracy from 95 to 97. Suggestion 2503 indicates that moving imaging device 2401 left will improve the total accuracy from 95 to 98. In this way, imaging positions for multiple imaging devices are proposed that will improve the integrated detection reliability (aggregated total accuracy) compared to the current imaging positions of the multiple imaging devices.

[0098] As described above, according to this embodiment, it is possible to propose an optimal position for adjusting the more complicated arrangement positions of a plurality of imaging devices.

[0099] <Modification 4: Display comparison results> 26 is a diagram illustrating a fourth optimum imaging device position proposal screen. A comparative display example 2600 is an example of displaying a comparison between the estimated accuracy of the current position and the estimated accuracy when the imaging devices are moved. The comparative display example 2600 displays how much the aggregated total accuracy (integrated detection reliability) improves when the imaging device 2405 is moved upward and the imaging device 2401 is moved to the left, by comparing the current aggregated total accuracy (integrated detection reliability) with that after the moves.

[0100] Detection result estimation processing result 2601 is an example in which the results after movement are superimposed on each vehicle compartment of the bird's-eye coordinate system image 903. Total accuracy display 2602 is an example of displaying the aggregated total accuracy after movement. By comparing and displaying the current state and the aggregated total accuracy after movement in this way, it is possible to propose an installation location that is easier for the user to understand. In this way, by presenting the integrated detection reliability at the current imaging position and the integrated detection reliability at the proposed imaging position, the user can more easily change the installation of multiple imaging devices.

[0101] <Variation 5: Proposal to increase the number of units> If the total accuracy (integrated detection reliability) for the number of imaging devices specified by the user is equal to or less than a predetermined value (for example, equal to or less than 99), an imaging device may be added to a position where the determination accuracy is low, and the process of determining whether a vehicle can be detected (S1106) may be performed again. Also, if the accuracy is less than the predetermined value (for example, less than 100), an imaging device may be added to a position where the determination accuracy is low, and the process of determining whether a vehicle can be detected (S1106) may be performed again.

[0102] That is, the integrated detection reliability when another imaging device is virtually placed may be further estimated, and the integrated detection reliability before and after the addition may be further displayed. If the total accuracy improves by adding an imaging device, the current total accuracy and the total accuracy after the addition may be compared and displayed, thereby making it possible to suggest installation locations that are easier for the user to understand.

[0103] Regardless of whether the total accuracy (integrated detection reliability) is equal to or less than a predetermined value, an integrated detection reliability may be further estimated in the case where another imaging device is additionally virtually arranged.

[0104] <Variation 6: When the user specifies the number of units> When the user specifies the number of imaging devices by operating the screen using a method for specifying the number of imaging devices (not shown), a process (S1106) may be performed to determine whether a vehicle can be detected with the specified number of devices. That is, the specification of the number of imaging devices may be accepted, and the integrated detection reliability at the current imaging positions of the multiple imaging devices and the integrated detection reliability for the specified number of imaging devices may be estimated and presented. By comparing and displaying the current aggregated total accuracy (integrated detection reliability) with the aggregated total accuracy (integrated detection reliability) for the specified number of devices, it is possible to propose installation locations that are easier for the user to understand.

[0105] <Other variations> In the above embodiment, an example has been described in which the captured image captured by the imaging device 110 is analyzed by the server 130, but the captured image captured by the imaging device 110 may be analyzed by the imaging device 110 itself.

[0106] The disclosure of this specification includes the following information processing device, image analysis system, control method for an information processing device, and program.

[0107] (Item 1) A setting means for setting one or more vehicle compartment areas representing parking spaces in a captured image of the parking lot; A determination means for determining each vehicle area indicating an area where a vehicle can be parked for each vehicle area based on the information on the vehicle area; an overlapping means for overlapping the vehicle area with the passenger compartment area; an area estimation means for estimating a three-dimensional vehicle area showing the vehicle based on the vehicle area; an estimation means for estimating the size of each three-dimensional vehicle area and the degree of overlap between the three-dimensional vehicle areas; a determination means for determining whether a vehicle can be detected when the vehicle is parked in each vehicle compartment area based on the size and the overlapping state; An information processing device comprising:

[0108] (Item 2) 2. The information processing device according to item 1, wherein the determining means determines that the vehicle is detectable when the size is equal to or larger than a predetermined size and the degree of overlap is equal to or smaller than a predetermined ratio.

[0109] (Item 3) 3. The information processing device according to item 1 or 2, wherein the estimation means estimates the size of the circumscribing rectangle of each three-dimensional vehicle area and the degree of overlap between the circumscribing rectangles of each three-dimensional vehicle area as the degree of overlap between the three-dimensional vehicle areas.

[0110] (Item 4) further comprising a presentation means for presenting a determination result by the determination means, 4. The information processing device according to any one of items 1 to 3, wherein the presenting means presents the positions of detectable vehicles and the positions of vehicles that are difficult to detect in different display formats.

[0111] (Item 5) The estimation means further estimates a detection reliability at each vehicle interior region position based on the size and the degree of overlap; Item 5. The information processing device according to item 4, wherein the presentation means presents the position of the detectable vehicle for which the detection reliability is equal to or greater than a predetermined value and the position of the detectable vehicle for which the detection reliability is less than the predetermined value in different display forms.

[0112] (Item 6) The estimation means further estimates a detection reliability at each vehicle interior region position based on the size and the degree of overlap; 5. The information processing device according to any one of items 1 to 4, wherein the estimation means estimates an overall detection reliability for all vehicle interior areas based on the detection reliability at the position of each vehicle interior area.

[0113] (Item 7) further comprising a detection means for detecting a vehicle from the captured image, The estimation means further estimates a detection reliability at each vehicle interior region position based on the size and the degree of overlap; 5. The information processing device according to any one of items 1 to 4, wherein the detection means filters out detection results in a vehicle interior area where the detection reliability is equal to or less than a predetermined value.

[0114] (Item 8) further comprising a detection means for detecting a vehicle from the captured image, The information processing device described in any one of items 1 to 6, characterized in that when a vehicle is detected in an area other than the vehicle interior area, the detection means filters out the detection result of the vehicle.

[0115] (Item 9) further comprising an imaging position estimation means for estimating an imaging position of an imaging device that acquires the captured image; the estimation means further estimates the overall detection reliability when the imaging position is virtually changed, The information processing device includes: 7. The information processing device according to item 6, further comprising a suggestion unit that suggests an imaging position that improves the overall detection reliability more than the current imaging position.

[0116] (Item 10) 10. The information processing device according to item 9, further comprising a presentation unit that presents the overall detection reliability before and after the imaging position is virtually changed.

[0117] (Item 11) The estimation means further estimates the orientation of the vehicle in each of the passenger compartment areas based on the positional relationship between the passenger compartment areas; the area estimation means estimates a three-dimensional vehicle model that imitates the vehicle as the three-dimensional vehicle area based on the orientation of the vehicle; The information processing device includes: further comprising a detection means for detecting the three-dimensional vehicle model from the image in which the three-dimensional vehicle model is estimated, The information processing device described in item 1, characterized in that the determination means determines whether a vehicle can be detected when a vehicle is parked in each vehicle compartment area based on the detection result of the detection means.

[0118] (Item 12) The determination means determines whether a vehicle parked in each vehicle compartment area can be detected from the captured images acquired by each of a plurality of imaging devices that capture images of the parking lot from different imaging positions, and The information processing device includes: 2. The information processing device according to item 1, further comprising a presentation unit that compiles and presents the results of the determination made by the determination unit for each of the plurality of imaging devices.

[0119] (Item 13) The estimation means further estimates a detection reliability at a position of each vehicle interior region, the estimation means estimates an overall detection reliability for all vehicle interior regions based on the detection reliability at each vehicle interior region position; Item 13. The information processing device according to item 12, wherein the estimation means estimates the overall detection reliability for each of the plurality of imaging devices and integrates the plurality of overall detection reliabilities to estimate an integrated detection reliability.

[0120] (Item 14) Item 14. The information processing device according to item 13, further comprising a suggestion unit that suggests imaging positions of the plurality of imaging devices that improve the integrated detection reliability compared to current imaging positions of the plurality of imaging devices.

[0121] (Item 15) Item 15. The information processing device according to item 14, wherein the presenting means presents the integrated detection reliability at the current imaging position and the integrated detection reliability at the imaging position proposed by the proposing means.

[0122] (Item 16) the estimation means further estimates an integrated detection reliability when another imaging device is additionally virtually arranged, Item 14. The information processing device according to item 13, wherein the presentation means presents the integrated detection reliability before and after the addition.

[0123] (Item 17) Item 17. The information processing device according to item 16, wherein the estimation means further estimates the integrated detection reliability when the other imaging device is virtually placed in addition to the other imaging device if the integrated detection reliability of the plurality of imaging devices is equal to or less than a predetermined value.

[0124] (Item 18) further comprising a receiving unit for receiving a designation of the number of imaging devices; The information processing device described in item 13, characterized in that the presentation means presents the integrated detection reliability at the current imaging position of the multiple imaging devices and the integrated detection reliability for the number of imaging devices specified by the reception means.

[0125] (Item 19) one or more imaging devices; An information processing device according to any one of items 1 to 18; An image analysis system comprising:

[0126] (Item 20) A control method for an information processing device, comprising: a setting step of setting one or more vehicle compartment areas representing parking spaces in a captured image of the parking lot; a determining step of determining each vehicle area indicating an area where a vehicle can be parked for each vehicle area based on the information of the vehicle area; an overlapping step of overlapping the vehicle area with the passenger compartment area; an area estimation step of estimating a three-dimensional vehicle area showing the vehicle based on the vehicle area; an estimation step of estimating the size of each three-dimensional vehicle region and the degree of overlap between the three-dimensional vehicle regions; a determination step of determining whether a vehicle can be detected when the vehicle is parked in each vehicle compartment area based on the size and the overlapping state; 1. A method for controlling an information processing device, comprising:

[0127] (Item 21) A program for causing a computer to function as the information processing device according to any one of items 1 to 18.

[0128] (Item 1) (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0129] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0130] 110a to 110d: imaging devices, 120: network, 130: server, 201: imaging unit, 202: image processing unit, 203: arithmetic processing unit, 204: distribution unit, 301: imaging control unit, 302: signal processing unit, 303: storage unit, 304: control unit, 305: analysis unit, 306: network communication unit, 401: processor, 402: RAM, 403: ROM, 404: HDD, 405: communication I / F, 501: network communication unit, 502: control unit, 503: display unit, 504: operation unit

Claims

1. A setting means for setting one or more vehicle compartment areas representing parking spaces in a captured image of the parking lot; A determination means for determining each vehicle area indicating an area where a vehicle can be parked for each vehicle area based on the information of the vehicle area; an overlapping means for overlapping the vehicle area with the passenger compartment area; an area estimation means for estimating a three-dimensional vehicle area showing the vehicle based on the vehicle area; an estimation means for estimating a size of each three-dimensional vehicle area and an overlapping degree between the three-dimensional vehicle areas; a determination means for determining whether a vehicle can be detected when the vehicle is parked in each vehicle compartment area based on the size and the overlapping state; An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein the determining means determines that the vehicle is detectable when the size is equal to or larger than a predetermined size and the degree of overlap is equal to or smaller than a predetermined ratio.

3. 2 . The information processing device according to claim 1 , wherein the estimation means estimates, as the degree of overlap between the three-dimensional vehicle areas, a size of a circumscribing rectangle of each three-dimensional vehicle area and an overlapping degree between the circumscribing rectangles of each three-dimensional vehicle area.

4. A presentation means for presenting a result of the determination by the determination means, 2. The information processing apparatus according to claim 1, wherein the presenting means presents the positions of vehicles that can be detected and the positions of vehicles that are difficult to detect in different display formats.

5. The estimation means further estimates a detection reliability at a position of each vehicle interior region based on the size and the overlapping state, 5. The information processing device according to claim 4, wherein the presentation means presents the position of the detectable vehicle for which the detection reliability is equal to or greater than a predetermined value and the position of the detectable vehicle for which the detection reliability is less than the predetermined value in different display forms.

6. The estimation means further estimates a detection reliability at a position of each vehicle interior region based on the size and the overlapping state, 2. The information processing apparatus according to claim 1, wherein the estimation means estimates an overall detection reliability for all the vehicle interior areas based on the detection reliability at the position of each vehicle interior area.

7. A detection unit for detecting a vehicle from the captured image is further provided. The estimation means further estimates a detection reliability at a position of each vehicle interior region based on the size and the overlapping state, 2. The information processing apparatus according to claim 1, wherein the detection means filters out detection results in the vehicle interior area where the detection reliability is equal to or lower than a predetermined value.

8. A detection unit for detecting a vehicle from the captured image is further provided.

2. The information processing apparatus according to claim 1, wherein, when a vehicle is detected in an area other than the vehicle interior area, the detection means filters out the detection result of the vehicle.

9. The imaging device further includes an imaging position estimation unit for estimating an imaging position of an imaging device that acquires the captured image, The estimation means further estimates the overall detection reliability when the imaging position is virtually changed, The information processing device includes: The information processing apparatus according to claim 6 , further comprising a suggestion unit that suggests an imaging position at which the overall detection reliability is improved compared to a current imaging position.

10. The information processing apparatus according to claim 9 , further comprising a presentation unit that presents the overall detection reliability before and after the imaging position is virtually changed.

11. The estimation means further estimates a direction of the vehicle in each of the vehicle interior areas based on a positional relationship between the vehicle interior areas; The area estimation means estimates a three-dimensional vehicle model that imitates the vehicle as the three-dimensional vehicle area based on a direction of the vehicle, The information processing device includes: a detection unit for detecting the three-dimensional vehicle model from the image in which the three-dimensional vehicle model is estimated, 2. The information processing apparatus according to claim 1, wherein the determining means determines whether or not a vehicle can be detected when the vehicle is parked in each vehicle compartment area, based on a detection result from the detecting means.

12. The determination means determines whether or not a vehicle can be detected when a vehicle is parked in each vehicle compartment area, for an image captured by each of a plurality of imaging devices that capture images of the parking lot from different imaging positions; The information processing device includes: The information processing apparatus according to claim 1 , further comprising a presentation unit that compiles and presents the results of the determination made by the determination unit for each of the plurality of image capture devices.

13. The estimation means further estimates a detection reliability at a position of each vehicle interior region, The estimation means estimates an overall detection reliability for all the vehicle interior regions based on the detection reliability at the position of each vehicle interior region; The information processing apparatus according to claim 12 , wherein the estimation means estimates the overall detection reliability for each of the plurality of image capture devices, and estimates an overall detection reliability by integrating the overall detection reliability.

14. The information processing apparatus according to claim 13 , further comprising a suggestion unit that suggests imaging positions of the plurality of imaging devices that improve the integrated detection reliability more than current imaging positions of the plurality of imaging devices.

15. 15 . The information processing apparatus according to claim 14 , wherein the presenting means presents the integrated detection reliability at the current imaging position and the integrated detection reliability at the imaging position proposed by the proposing means.

16. The estimation means further estimates an integrated detection reliability in a case where another imaging device is additionally virtually arranged, The information processing apparatus according to claim 13 , wherein the presenting means presents the integrated detection reliability before and after the addition.

17. The information processing device according to claim 16 , wherein the estimation means, when the combined detection reliability of the plurality of imaging devices is equal to or less than a predetermined value, further estimates the combined detection reliability in a case where the other imaging device is additionally virtually arranged.

18. The imaging device further includes a reception unit for receiving a designation of the number of imaging devices, 14. The information processing apparatus according to claim 13, wherein the presenting means presents an integrated detection reliability at a current imaging position of the plurality of imaging devices and the integrated detection reliability for a number of imaging devices designated by the accepting means.

19. one or more imaging devices; An information processing device according to any one of claims 1 to 18; An image analysis system comprising:

20. A method for controlling an information processing device, comprising: A setting step of setting one or more vehicle compartment areas representing parking spaces in a captured image of the parking lot; determining each vehicle area indicating an area where a vehicle can be parked for each vehicle area based on the information of the vehicle area; a superimposing step of superimposing the vehicle area onto the passenger compartment area; an area estimation step of estimating a three-dimensional vehicle area showing the vehicle based on the vehicle area; an estimation step of estimating a size of each three-dimensional vehicle area and an overlapping degree between the three-dimensional vehicle areas; a determination step of determining whether a vehicle can be detected when the vehicle is parked in each vehicle compartment area based on the size and the overlapping state; 13. A method for controlling an information processing apparatus comprising the steps of:

21. A program for causing a computer to function as the information processing device according to any one of claims 1 to 18.

Images