Computer program, information processing method, information processing device, and information processing system

Abstract

To provide a computer program and the like that can accurately specify a parking situation of a vehicle in a parking place.SOLUTION: The computer program causes a computer to execute processing of: acquiring an image obtained by imaging a parking place having a plurality of vehicle rooms; detecting a vehicle region corresponding to a vehicle in the acquired image; and specifying a vehicle room in which the vehicle is parked based on positional relationship between the vehicle region in the image and vehicle room regions corresponding to the respective vehicle rooms and a priority order of the respective vehicle rooms.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a computer program, an information processing method, an information processing device, and an information processing system. [Background technology]

[0002] Conventionally, technologies for determining the parking status of vehicles in parking lots have been proposed. For example, Patent Document 1 discloses a vehicle location management system that includes multiple markers on the floor of the parking lot and a surveillance camera that periodically captures images of the parking lot from above, and that identifies the location information of each vehicle within the parking lot by detecting markers whose shapes cannot be properly recognized. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-175026 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the vehicle location management system described in Patent Document 1 has room for improvement in terms of the accuracy of identifying the parking situation of a vehicle.

[0005] An object of the present disclosure is to provide a computer program etc. that can accurately identify the parking status of vehicles in a parking lot. [Means for solving the problem]

[0006] A computer program according to one aspect of the present disclosure acquires an image of a parking lot having multiple vehicle compartments, detects a vehicle area corresponding to a vehicle in the acquired image, and causes a computer to perform a process of identifying the vehicle compartment in which the vehicle is parked based on the positional relationship between the vehicle area in the image and the vehicle compartment areas corresponding to each vehicle compartment and the priority of each vehicle compartment.

[0007] An information processing method according to one aspect of the present disclosure involves a computer acquiring an image of a parking lot having multiple vehicle compartments, detecting a vehicle area corresponding to a vehicle in the acquired image, and identifying the vehicle compartment in which the vehicle is parked based on the positional relationship between the vehicle area in the image and the vehicle compartment areas corresponding to each vehicle compartment and the priority of each vehicle compartment.

[0008] An information processing device according to one aspect of the present disclosure includes an acquisition unit that acquires an image of a parking lot having multiple vehicle compartments, a detection unit that detects a vehicle area corresponding to a vehicle in the acquired image, and an identification unit that identifies the vehicle compartment in which the vehicle is parked based on the positional relationship between the vehicle area in the image and the vehicle compartment areas corresponding to each vehicle compartment and the priority of each vehicle compartment.

[0009] An information processing system according to one aspect of the present disclosure comprises a photographing device installed in a parking lot having a plurality of vehicle compartments, and an information processing device, wherein the information processing device comprises an acquisition unit that acquires an image of the parking lot photographed by the photographing device, a detection unit that detects a vehicle area corresponding to a vehicle in the acquired image, and an identification unit that identifies the vehicle compartment in which the vehicle is parked based on the positional relationship between the vehicle area in the image and the vehicle compartment areas corresponding to each vehicle compartment and the priority of each vehicle compartment. [Effects of the Invention]

[0010] According to the present disclosure, the parking situation of a vehicle in a parking lot can be identified with high accuracy. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram of a parking lot management system. [Figure 2] FIG. 1 is a block diagram showing an example of the configuration of a parking lot management system. [Figure 3] FIG. 4 is a diagram showing an example of information stored in a parking lot DB. [Figure 4] FIG. 1 is an explanatory diagram illustrating an overview of a detection model. [Figure 5] FIG. 10 is an explanatory diagram illustrating a method for identifying a vehicle compartment corresponding to a parked vehicle. [Figure 6] 10 is a flowchart illustrating an example of a processing procedure for acquiring a vehicle interior area and a priority level, which is executed by an information processing device. [Figure 7] 10 is a flowchart illustrating an example of a processing procedure for identifying a parking space executed by an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present disclosure will be specifically described with reference to the drawings showing embodiments thereof.

[0013] 1 is a schematic diagram of a parking lot management system 100. The parking lot management system 100 includes an information processing device 1 and an imaging device 2 provided in a parking lot 3.

[0014] The information processing device 1 is a device capable of various information processing and information transmission and reception, such as a server computer, a personal computer, a quantum computer, etc. Based on a parking lot image of the parking lot 3 captured by the imaging device 2, the information processing device 1 identifies the parking status of the vehicle 4 in the parking lot 3, i.e., in which compartment 31 the vehicle 4 is parked.

[0015] The parking lot 3 is provided with a plurality of vehicle compartments 31. Each vehicle compartment 31 is a generally rectangular area in a plan view. The vehicle compartments 31 are demarcated by demarcation lines 32, such as white lines. Each vehicle compartment 31 can accommodate one vehicle 4. The parking lot 3 may be indoors or outdoors.

[0016] The photographing device 2 is a camera equipped with an imaging element such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The photographing range of the photographing device 2 includes the parking lot 3, and photographs vehicles 4 parked in the vehicle compartment 31 and vehicles 4 moving within the parking lot 3. The photographing device 2 of this embodiment is a fixed camera installed in the parking lot 3. The photographing device 2 is installed at a position higher than the vehicle height of the vehicle 4. The images photographed by the photographing device 2 may be videos or still images. The parking lot images obtained by the photographing device 2 are transmitted to the information processing device 1.

[0017] The camera devices 2 may be installed in any suitable location as long as they are capable of photographing the inside of the parking lot 3. The number of camera devices 2 may be two or more. In the case of a large parking lot 3, it is advisable to divide the parking lot into multiple areas and install multiple camera devices 2 to photograph each area.

[0018] 2 is a block diagram showing an example of the configuration of the parking lot management system 100. As described above, the parking lot management system 100 includes the information processing device 1 and the photographing device 2. The information processing device 1 and the photographing device 2 are communicatively connected via a network N such as the Internet or a LAN (Local Area Network), and are capable of transmitting and receiving data.

[0019] The information processing device 1 includes a control unit 11, a storage unit 12, a communication unit 13, a display unit 14, and an operation unit 15. The information processing device 1 may be a multi-computer consisting of multiple computers, or may be a virtual machine virtually constructed by software.

[0020] The control unit 11 includes one or more arithmetic processing devices such as a central processing unit (CPU) or a graphics processing unit (GPU). The control unit 11 controls each component unit and executes processing using built-in memories such as a read-only memory (ROM) or a random access memory (RAM), a clock, a counter, etc. The functional units of the control unit 11 may be realized by software, or some or all of them may be realized by hardware such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0021] The storage unit 12 includes a non-volatile memory such as a hard disk, a flash memory, or an SSD (Solid State Drive). The storage unit 12 may be an external storage device connected to the information processing device 1. The storage unit 12 stores various computer programs and data referenced by the control unit 11. The storage unit 12 of this embodiment stores a program 1P for causing a computer to execute processing related to identifying a parking space, and a parking lot DB (Data Base) 121 and a detection model 122 used to execute the program 1P. The parking lot DB 121 is a database that stores information about the spaces 31 in the parking lot 3. The detection model 122 is a machine learning model that has learned predetermined training data. The detection model 122 is expected to be used as a program module that constitutes part of artificial intelligence software.

[0022] A computer program (program product) including program 1P may be provided by a non-transitory recording medium 1A on which the computer program is readably recorded. Storage unit 12 stores the computer program read from recording medium 1A by a reading device (not shown). Recording medium 1A may be, for example, a magnetic disk, optical disk, or semiconductor memory. Alternatively, the computer program may be downloaded from an external server connected to a communications network and stored in storage unit 12. Program 1P may be a single computer program or may be composed of multiple computer programs, and may be executed on a single computer or on multiple computers interconnected by a communications network.

[0023] The communication unit 13 includes a communication device that realizes communication with external devices via the network N. The control unit 11 transmits and receives various information to and from external devices including the photographing device 2 via the communication unit 13. The information processing device 1 may realize communication with the photographing device 2 through another computer connected to the network N or a relay device for relaying communication.

[0024] The display unit 14 includes a display device such as a liquid crystal display, an organic EL (Electro Luminescence) display, etc. The display unit 14 displays information to be notified to the user in accordance with instructions from the control unit 11. The display unit 14 may be interpreted as a notification unit and may be configured to notify information by other means such as sound or lighting.

[0025] The operation unit 15 is an interface that accepts user operations. The operation unit 15 includes, for example, a keyboard, a mouse, a touch panel device with a built-in display, a speaker, a microphone, etc. The operation unit 15 accepts operation input from the user and sends a control signal to the control unit 11 according to the operation content.

[0026] Note that the information processing device 1 and the photographing device 2 are not limited to transmitting and receiving data via the network N. The information processing device 1 may, for example, have an input interface for connecting the photographing device 2 and receive image data output from the photographing device 2 via a signal line. The information processing device 1 may also be configured to receive operations via an externally connected computer and output information to be notified to the external computer. In this case, the information processing device 1 does not need to have the display unit 14 and the operation unit 15.

[0027] 3 is a diagram showing an example of the contents of information stored in the parking lot DB 121. For example, the parking lot DB 121 stores records for each of a plurality of parking spaces 31 in the parking lot 3, linking information such as parking lot information, image capturing device information, parking space number, distance, parking space area information, and priority order.

[0028] The parking lot information is identification information for identifying the parking lot 3, and includes, for example, a parking lot ID. The photographing device information is identification information for identifying the photographing device 2 that photographs the parking lot image, and includes, for example, a device ID, an IP address, etc. If multiple photographing devices 2 are installed in the same parking lot 3, the photographing device information may store the photographing device information of any one of the photographing devices 2 that is suitable for photographing the vehicle compartment 31, or the photographing device 2 that is closest to the vehicle compartment 31.

[0029] The vehicle compartment number is an example of identification information for identifying the vehicle compartment 31. A different vehicle compartment number is assigned to each of the vehicle compartments 31 in the parking lot 3. Note that the identification information for the vehicle compartment 31 is not limited to the vehicle compartment number, and may be, for example, letters, symbols, etc.

[0030] The distance is information indicating the actual distance between the camera device 2 installed in the parking lot 3 and each vehicle compartment 31. The distance is obtained by measuring the distance from the camera device 2 to each vehicle compartment 31. The distance is used as an index for determining priority, which will be described later.

[0031] The vehicle compartment area information is information indicating the vehicle compartment area corresponding to the vehicle compartment 31 in the parking lot image. The vehicle compartment area is the area surrounding the vehicle compartment frame indicated by the demarcation line 32. In this embodiment, the vehicle compartment area is a rectangular area on the image and is the smallest area surrounding the vehicle compartment frame. The vehicle compartment area is represented, for example, by a rectangular bounding box indicating the vehicle compartment area. The vehicle compartment area information sequence includes definition information of the bounding box in the image. In the example of Figure 3, the definition information includes the coordinates of the center point, height, and width of the bounding box, but the method of defining the bounding box is not limited.

[0032] The information processing device 1 automatically detects the vehicle compartment area of ​​each vehicle compartment 31, for example, by analyzing a parking lot image obtained by photographing each vehicle compartment 31 in an empty state using the imaging device 2. An empty state means that no vehicle is parked in each vehicle compartment 31, i.e., a state in which the entirety of the partition lines 32 indicating each vehicle compartment 31 can be recognized. The method for detecting the vehicle compartment area from the parking lot image is not particularly limited, and any object detection method can be used. For example, the information processing device 1 detects the vehicle compartment area in the parking lot image using a learning model trained to detect the vehicle compartment area from the parking lot image using a machine learning method. Examples of learning models for object detection include CNN (Convolutional Neural Network), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), R-CNN (Regions with Convolutional Neural Network), and Faster R-CNN. Note that the detection of the vehicle compartment area may be performed by an external device other than the information processing device 1, or may be performed manually.

[0033] The priority order is information indicating the order in which the vehicle compartments 31 corresponding to the vehicle compartment numbers are given priority in the parking lot 3. The priority order is information used to determine which vehicle compartment 31 is given priority when identifying the vehicle compartment number where the vehicle 4 is parked if the vehicle 4 detected from the parking lot image straddles the vehicle compartment areas of multiple vehicle compartments 31. The priority order is, for example, a series of numbers assigned to all vehicle compartments 31 in the parking lot 3. Note that the priority order is not limited to being assigned to all vehicle compartments 31, and it is sufficient if it can identify the priority relationship between at least adjacent vehicle compartments 31. In other words, the priority order is sufficient if it is information that can identify which vehicle compartment 31 is given priority among adjacent vehicle compartments 31 in the parking lot 3.

[0034] The priority order is set based on the distance from the image capture device 2 to each vehicle compartment 31. The vehicle compartments 31 are ranked so that the closer the distance between the image capture device 2 and the vehicle compartment 31, the higher the priority order. The information processing device 1 determines the priority order of each vehicle compartment 31 by comparing the distances between the vehicle compartments 31 located in the parking lot 3. Note that the prioritization may be performed by an external device other than the information processing device 1, or may be performed manually.

[0035] The information processing device 1 acquires information about the parking spaces 31 in advance, including the above-mentioned parking space area information and priority order, for each parking space 3 to be managed, and stores the acquired various information in the parking space DB 121.

[0036] 4 is an explanatory diagram showing an overview of the detection model 122. The detection model 122 is a machine learning model that receives an image of a parking lot as input and outputs information indicating a vehicle area in the image of the parking lot that corresponds to the vehicle 4. The detection model 122 is a model configured by a neural network, such as a CNN (Convolutional Neural Network).

[0037] The vehicle region is a region surrounding a vehicle. In this embodiment, the vehicle region is a rectangular region on the image, and is the smallest region surrounding the vehicle. The vehicle region is represented by, for example, a rectangular bounding box 51 surrounding the detected vehicle.

[0038] The detection model 122 has an input layer to which a parking lot image is input, an intermediate layer (hidden layer) that extracts feature amounts of the parking lot image, and an output layer that outputs information indicating a vehicle area. The intermediate layer has multiple nodes that extract feature amounts of the image, and passes the feature amounts extracted using various parameters to the output layer. When a parking lot image is input to the input layer, calculations are performed in the intermediate layer using the learned parameters, and information indicating a vehicle area is output from the output layer.

[0039] The detection model 122 can be generated by preparing training data in which labels indicating the vehicle's position and range are associated with parking lot images containing vehicles, and then using the training data to train an untrained neural network.

[0040] The information processing device 1 inputs multiple parking lot images included in the training data into the input layer of a pre-training neural network model, undergoes arithmetic processing in the intermediate layer, and obtains the vehicle position and range output from the output layer. The information processing device 1 compares the vehicle position and range output from the output layer with the vehicle position and range included in the training data, and optimizes parameters such as the weights between neurons so that the object position and range output from the output layer approach the correct values. There are no particular limitations on the method for optimizing the parameters, but the information processing device 1, for example, optimizes various parameters using an error backpropagation method.

[0041] Through the above-described processing, a detection model 122 is constructed that has been trained to be able to output vehicle regions corresponding to one or more vehicles in the parking lot image when the parking lot image is input. Note that the detection model 122 may be constructed by an external device and deployed in the information processing device 1.

[0042] The detection model 122 may be configured to detect both the vehicle 4 and the vehicle compartment 31. In this case, the detection model 122 recognizes the location and type of objects in the parking lot image.

[0043] The configuration of the detection model 122 is not limited to the example shown in Fig. 4. The detection model 122 may be any model as long as it can detect a vehicle area in a parking lot image. The detection model 122 may be a model constructed using other learning algorithms, such as YOLO, SSD, R-CNN, Faster R-CNN, a support vector machine, or a regression tree. The detection model 122 may also be a model that detects the vehicle 4 using a method such as pattern matching.

[0044] Instead of storing the detection model 122 in the storage unit 12 of the information processing device 1, the information processing device 1 may access and read out the detection model 122 stored in an external detection server or the like.

[0045] A method for identifying a vehicle compartment corresponding to a parked vehicle, which is executed by the information processing device 1 of this embodiment, will be described. FIG. 5 is an explanatory diagram illustrating the method for identifying a vehicle compartment corresponding to a parked vehicle. FIG. 5 schematically shows a portion of the parking lot image. For ease of explanation, FIG. 5 shows only the vehicle 4 parked in the vehicle compartment 31 with the vehicle compartment number 8.

[0046] When the information processing device 1 acquires a parking lot image captured by the image capturing device 2, it detects a vehicle area corresponding to the vehicle 4 included in the acquired parking lot image using the detection model 122. The vehicle area is represented by a rectangular bounding box 51 that surrounds the vehicle 4, as shown by the thick solid line in FIG.

[0047] The information processing device 1 acquires vehicle compartment area information relating to the vehicle compartment number and vehicle compartment area of ​​each vehicle compartment 31 corresponding to the acquired parking lot image based on the information stored in the parking lot DB 121. The vehicle compartment area is represented by a rectangular bounding box 52 surrounding the vehicle compartment 31, as shown by the thin solid line in FIG.

[0048] The information processing device 1 identifies the compartment number of the compartment 31 in which the vehicle 4 is parked based on the positional relationship between the detected vehicle area and the compartment area of ​​each compartment 31, and the priority of each compartment 31. In detail, one or more compartments 31 that are candidates for the parked compartment are identified based on the overlap state between the vehicle area and the compartment area of ​​each compartment 31 in the parking lot image. By selecting the compartment 31 with the highest priority from the identified candidate compartments 31, the compartment number of the compartment 31 corresponding to the parked vehicle is finally derived.

[0049] A specific example will be described with reference to Figure 5. In the following description, the bounding box 51 indicating the vehicle area will also be referred to as the vehicle bounding box 51, and the bounding box 52 indicating the passenger compartment area will also be referred to as the passenger compartment bounding box 52.

[0050] The information processing device 1 identifies one or more vehicle compartment bounding boxes 52 that partially overlap with the vehicle bounding box 51 based on the positions and sizes of the vehicle bounding box 51 on the parking lot image and the vehicle compartment bounding boxes 52 corresponding to each vehicle compartment 31. In the example shown in Figure 5, vehicle compartment bounding boxes 52 corresponding to seven vehicle compartments 31 corresponding to vehicle compartment numbers 1, 2, 3, 6, 7, 8, and 9 are identified as overlapping with the vehicle bounding box 51.

[0051] The information processing device 1 identifies candidate vehicle compartments that are candidates for the parked vehicle compartment based on the degree of overlap between each of the identified vehicle compartment bounding boxes 52 and the vehicle bounding box 51. The information processing device 1 identifies vehicle compartments 31 whose degree of overlap is equal to or greater than a preset threshold as candidate vehicle compartments. The degree of overlap may be found by calculating the IoU (Intersection over Union) between the vehicle bounding box 51 and the vehicle compartment bounding box 52.

[0052] IoU is a numerical representation of the degree of overlap between two rectangular regions, and the larger the IoU value, the greater the degree of overlap between the rectangles. IoU is calculated by dividing the area of ​​the intersection between vehicle bounding box 51 and cabin bounding box 52 by the area of ​​the combined region.

[0053] The information processing device 1 calculates the IoU between the vehicle bounding box 51 and the cabin bounding box 52 for each cabin 31 and determines whether the calculated IoU is equal to or greater than a threshold. If the IoU for a cabin 31 is equal to or greater than the threshold, the information processing device 1 identifies the cabin 31 as a candidate cabin. In the example shown in Fig. 5, cabin numbers 7 and 8 are selected as candidate cabins based on the IoU calculation results.

[0054] The threshold value can be set appropriately based on the arrangement of the vehicle compartments 31 in each parking lot 3, the position and size of the vehicle compartment area, etc. The threshold value may be set for each vehicle compartment bounding box 52 (each vehicle compartment 31). The information processing device 1 may store the threshold value received from the user in the storage unit 12 in advance. The threshold value may be configured to be changeable at any timing during operation.

[0055] The information processing device 1 finally identifies the compartment number of the compartment 31 in which the vehicle 4 is parked by selecting the compartment 31 with the highest priority from among the candidate compartments based on the priority set for each identified candidate compartment. For example, if the priority of compartment number 7 selected as a candidate compartment is 9 and the priority of compartment number 8 is 8, the compartment number 8 with the highest priority is finally identified as the parked compartment.

[0056] In the above process, if there is no vehicle compartment bounding box 52 with an IoU equal to or greater than a threshold value for the vehicle bounding box 51 corresponding to the vehicle 4, the vehicle 4 may be determined not to be a parked vehicle. If it is determined not to be a parked vehicle, the subsequent process of identifying the parked vehicle compartment may not be performed.

[0057] Furthermore, when multiple image capture devices 2 are installed in the same parking lot 3, it may be possible to determine whether or not it is necessary to identify the vehicle compartment of a vehicle 4 detected from the parking lot image, depending on the position of the vehicle 4 on the parking lot image or the position of the bounding box 51. For example, a detection area may be set in advance within the parking lot image, and only vehicles 4 included within that detection area may be selected as targets for identification processing. By setting the detection area according to the imaging range of each image capture device 2, it is possible to switch the image used for identification processing depending on the position of the vehicle 4 in the parking lot 3, thereby improving identification accuracy.

[0058] The information processing device 1 performs the above-described parking space identification process for each vehicle 4 detected from the parking lot image, and identifies the space number of the parking space for each vehicle 4 included in the parking lot image. Based on the identification result, the information processing device 1 derives the parking status for each space 31 in the parking lot 3, indicating whether it is vacant or full. The information processing device 1 generates a display screen showing the derived parking status for each space 31, and displays the generated display screen on the display unit 14. For example, the display screen shows a list of space numbers associated with vacant / full status for all spaces 31 in the parking lot 3.

[0059] The parking status may be output to an output device other than the display unit 14 of the information processing device 1. The parking status may be displayed on a display device provided at the entrance to the parking lot 3, for example. The parking status may be output individually for each vehicle space 31 through an indicator light provided for each vehicle space 31 in the parking lot 3.

[0060] The information processing device 1 uses the parking lot image captured by the image capturing device 2 to perform the above-mentioned process of identifying the parking space and outputting the parking situation in real time.

[0061] Although the above describes an example in which IoU is used to indicate the overlap state between the vehicle area and the vehicle interior area, the method for identifying the overlap state is not limited to this. The overlap state may be determined, for example, by dividing the area of ​​the intersection between the vehicle area and the vehicle interior area by the area of ​​the vehicle area or the vehicle interior area. The overlap state may also be determined by dividing the distance between the center points of the vehicle bounding box 51 and the vehicle interior bounding box 52 in at least one of the X-axis and Y-axis directions of the parking lot image by the maximum distance between their vertices.

[0062] Furthermore, the vehicle area and the passenger compartment area are not limited to those detected as rectangular areas surrounding the vehicle and the passenger compartment, respectively, and may be areas showing the shapes of the vehicle 4 and the passenger compartment 31 themselves. In this case, the detection model 122 that detects the vehicle and the learning model that detects the passenger compartment may be a model that recognizes an object (vehicle or passenger compartment) on a pixel-by-pixel basis using image recognition technology that uses instance segmentation, for example. When the vehicle area and the passenger compartment area are detected on a pixel-by-pixel basis, the overlap state may be a value obtained by dividing the area of ​​the common part between the vehicle area and the passenger compartment area by the area of ​​the merged area.

[0063] The information processing device 1 may omit the previous process of collecting the vehicle compartment area, and may collectively detect the vehicle 4 and the vehicle compartment 31 from the parking lot image acquired when executing the process of identifying the parked vehicle compartment.

[0064] The information processing device 1 may divide the parking lot image into a plurality of areas according to the arrangement of the parking spaces 31, and perform the process of identifying the parked parking spaces for each of the parking spaces 31 included in the divided image. In this case, the number of divisions, the size of the divided images, the correspondence between the parking spaces 31 and the divided images, etc. may be stored in advance in the parking lot DB 121.

[0065] 6 is a flowchart showing an example of a processing procedure for acquiring the vehicle interior area and priority order executed by the information processing device 1. The processing in each of the following flowcharts is executed by the control unit 11 in accordance with a program 1P stored in the storage unit 12 of the information processing device 1.

[0066] The control unit 11 of the information processing device 1 acquires information about the vehicle compartment 31 of the managed parking lot 3, for example, based on a user's operation of the operation unit 15 or by communication with an external device via the communication unit 13 (step S11). The information about the vehicle compartment 31 includes parking lot information, photographing device information, vehicle compartment number, and distance for the managed parking lot 3.

[0067] The control unit 11 acquires the priority of each vehicle compartment 31, which is set based on the acquired distance to each vehicle compartment 31 (step S12). The control unit 11 determines the priority of each vehicle compartment 31 so that, for example, the closer the distance between the image capturing device 2 and the vehicle compartment 31, the higher the priority.

[0068] The control unit 11 acquires a parking lot image showing an empty parking lot 3 photographed by the photographing device 2 (step S13). The control unit 11 may receive from the photographing device 2 the parking lot image in association with the photographing device information of the photographing device 2 that photographed the parking lot image.

[0069] The control unit 11 acquires a vehicle compartment area corresponding to each vehicle compartment 31 in the parking lot 3, which is detected by analyzing the acquired image of the parking lot in an empty state (step S14). The control unit 11 detects a vehicle compartment area corresponding to each vehicle compartment 31 included in the parking lot image, for example, by inputting the parking lot image into a learning model that detects a vehicle compartment area when the parking lot image is input, and acquiring the vehicle compartment area output from the learning model. The vehicle compartment area is represented by a vehicle compartment bounding box that surrounds the vehicle compartment 31.

[0070] The control unit 11 stores the obtained priority order and various information including the vehicle compartment area in the parking DB 121 (step S15), and ends the process.

[0071] 7 is a flowchart showing an example of a processing procedure for identifying a parked vehicle space, which is executed by the information processing device 1. The information processing device 1 repeatedly executes the following processing at predetermined or appropriate time intervals.

[0072] The control unit 11 of the information processing device 1, by functioning as an acquisition unit, acquires a parking lot image captured by the photographing device 2 (step S21). The control unit 11 may receive the parking lot image from the photographing device 2 in association with the photographing device information of the photographing device 2 that captured the parking lot image.

[0073] The control unit 11, by functioning as a detection unit, detects a vehicle area corresponding to the vehicle 4 included in the acquired parking lot image (step S22). Specifically, the control unit 11 inputs the acquired parking lot image to the detection model 122 and acquires the vehicle area output from the detection model 122, thereby detecting the vehicle area in the parking lot image. The vehicle area is represented by a vehicle bounding box that surrounds the vehicle 4.

[0074] Based on the information stored in the parking lot DB 121, the control unit 11 acquires the vehicle compartment area corresponding to the photographing device 2 or parking lot 3 that captured the acquired parking lot image (step S23).

[0075] The control unit 11, by functioning as an identification unit, identifies the vehicle compartment 31 of the vehicle compartment bounding box that overlaps with the vehicle bounding box based on the positional relationship between the vehicle bounding box on the parking lot image and the vehicle compartment bounding box corresponding to each vehicle compartment 31 (step S24).

[0076] The control unit 11 calculates the IoU between the vehicle bounding box and the vehicle compartment bounding box of each of the identified overlapping compartments 31 (step S25). The control unit 11 derives candidate compartments by identifying one or more compartments 31 whose calculated IoU is equal to or greater than a preset threshold (step S26). Note that the control unit 11 may omit step S24 and perform the IoU calculation in step S25 for each compartment 31.

[0077] The control unit 11 identifies the compartment number of the parking compartment corresponding to the vehicle 4 by identifying the compartment 31 with the highest priority from among the candidate compartments based on the priority set for each of the derived candidate compartments (step S27). The control unit 11 executes the processes of steps S24 to S27 for each vehicle 4 detected from the parking lot image, and identifies the compartment number of the parking compartment for each vehicle 4.

[0078] Based on the result of identifying the parking space number, the control unit 11 derives the parking situation indicating whether each parking space 31 in the parking lot 3 is empty or full (step S28).

[0079] The control unit 11 generates notification information for notifying the derived parking situation for each vehicle compartment 31 (step S29), and outputs the generated notification information via, for example, the display unit 14 (step S30). The control unit 11 ends the series of processes.

[0080] According to this embodiment, the vehicle compartment where the vehicle is parked can be easily and accurately identified based on the positional relationship between the vehicle area and the vehicle compartment area in the parking lot image and the priority of the vehicle compartment. By presetting the priority between the vehicle compartments in the identification process, the parked vehicle compartment can be identified efficiently and accurately even when the vehicle area spans multiple vehicle compartment areas. Even when using parking lot images in which the vehicle compartment is photographed from a relatively low position or an oblique angle, the vehicle parking situation can be identified accurately, which improves the flexibility of installing image capture devices in the parking lot.

[0081] By taking into account the actual distance between the camera and the vehicle compartment in the parking lot, the vehicle compartment layout can be reflected and priority can be set appropriately. By pre-storing the vehicle compartment area accurately detected using images of vacant parking lots, the processing load during operation can be reduced.

[0082] By using the degree of overlap between the vehicle area and the vehicle compartment area as an index, candidate vehicle compartments that are candidates for parked vehicle compartments can be efficiently identified. By removing vehicle compartments with low overlap from the candidate vehicle compartments, it is possible to reduce false positives of vehicle compartments that are not actually parked as parked vehicle compartments.

[0083] By detecting the vehicle area and the vehicle compartment area as rectangular areas, the computational cost of the object detection process can be reduced and the processing speed can be increased. By using the IoU of the bounding boxes corresponding to each area, the overlapping state can be calculated efficiently, and the parking status of each vehicle compartment can be identified quickly even in a parking lot with many vehicles.

[0084] The following additional notes are provided regarding the above-described embodiments. (Appendix 1) Acquire an image of a parking lot having a plurality of parking spaces; Detecting a vehicle area corresponding to a vehicle in the acquired image; Identifying the vehicle compartment in which the vehicle is parked based on a positional relationship between the vehicle area in the image and a vehicle compartment area corresponding to each vehicle compartment and a priority of each vehicle compartment. A computer program that causes a computer to perform a process. (Appendix 2) The priority order is set at least between adjacent vehicle compartments. 2. The computer program of claim 1. (Appendix 3) The priority order is set based on the actual distance between a photographing device installed in the parking lot and each vehicle compartment. 10. The computer program of claim 1 or 2. (Appendix 4) The vehicle compartment area and priority are stored in advance for each vehicle compartment in the parking lot. 4. A computer program according to any one of claims 1 to 3. (Appendix 5) deriving a plurality of vehicle compartments in which the degree of overlap between the vehicle region and the vehicle compartment region in the image is equal to or greater than a predetermined value; The compartment with the highest priority among the derived plurality of compartments is identified as the compartment in which the vehicle is parked. 5. A computer program according to any one of claims 1 to 4. (Appendix 6) deriving a plurality of vehicle compartments for which the IoU between a first bounding box representing the vehicle area and a second bounding box representing the vehicle compartment area is equal to or greater than a predetermined value; The compartment with the highest priority among the derived plurality of compartments is identified as the compartment in which the vehicle is parked. 6. A computer program according to any one of claims 1 to 5. (Appendix 7) Derive a parking situation for each vehicle compartment in the parking lot based on the result of identifying the vehicle compartment. 7. A computer program according to any one of claims 1 to 6.

[0085] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the scope of the claims and the scope equivalent to the claims. The sequences shown in each embodiment are not limited, and the order of each process may be changed within a range consistent with the present invention, and multiple processes may be executed in parallel. The entity that performs each process is not limited, and the process of each device may be executed by another device within a range consistent with the present invention.

[0086] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, although the claims use a format in which a claim references two or more other claims (multiple claim format), this is not limited to this format. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used. [Explanation of symbols]

[0087] 100 Parking lot management system (information processing system) 1. Information processing equipment 11 Control section 12 Storage section 13 Communications Department 14 Display section 15 Control section 1A Recording Media 1P Program 121 Parking DB 122 Detection Model 2. Imaging equipment 3. Parking 31 Cabin 4 vehicles

Claims

1. Acquire an image of a parking lot having a plurality of parking spaces; Detecting a vehicle area corresponding to a vehicle in the acquired image; Identifying the vehicle compartment in which the vehicle is parked based on the positional relationship between the vehicle area in the image and the vehicle compartment area corresponding to each vehicle compartment and the priority of each vehicle compartment indicating which of multiple vehicle compartments is given priority when identifying the vehicle compartment in which the vehicle is parked. A computer program that causes a computer to perform a process.

2. The priority order is set at least between adjacent vehicle compartments.

2. The computer program of claim 1.

3. The priority order is set based on the actual distance between a photographing device installed in the parking lot and each vehicle compartment.

3. A computer program according to claim 1 or claim 2.

4. The vehicle compartment area and the priority order are stored in advance for each vehicle compartment in the parking lot.

3. A computer program according to claim 1 or claim 2.

5. deriving a plurality of vehicle compartments in which the degree of overlap between the vehicle region and the vehicle compartment region in the image is equal to or greater than a predetermined value; The compartment with the highest priority among the derived plurality of compartments is identified as the compartment in which the vehicle is parked.

3. A computer program according to claim 1 or claim 2.

6. deriving a plurality of vehicle compartments for which an IoU between a first bounding box representing the vehicle area and a second bounding box representing the vehicle compartment area is equal to or greater than a predetermined value; The compartment with the highest priority among the derived plurality of compartments is identified as the compartment in which the vehicle is parked.

3. A computer program according to claim 1 or claim 2.

7. Derive a parking situation for each vehicle compartment in the parking lot based on the result of identifying the vehicle compartment.

3. A computer program according to claim 1 or claim 2.

8. Acquire an image of a parking lot having a plurality of parking spaces; Detecting a vehicle area corresponding to a vehicle in the acquired image; Identifying the vehicle compartment in which the vehicle is parked based on the positional relationship between the vehicle area in the image and the vehicle compartment area corresponding to each vehicle compartment and the priority of each vehicle compartment indicating which of multiple vehicle compartments is given priority when identifying the vehicle compartment in which the vehicle is parked. An information processing method in which processing is performed by a computer.

9. an acquisition unit that acquires an image of a parking lot having a plurality of parking spaces; a detection unit that detects a vehicle area corresponding to a vehicle in the acquired image; and an identification unit that identifies the vehicle compartment in which the vehicle is parked based on a positional relationship between the vehicle area in the image and a vehicle compartment area corresponding to each vehicle compartment and a priority order of each vehicle compartment that indicates which of a plurality of vehicle compartments is given priority when identifying the vehicle compartment in which the vehicle is parked. Information processing device.

10. The parking lot includes a photographing device and an information processing device, the photographing device being installed in a parking lot having a plurality of parking spaces; The information processing device includes: an acquisition unit that acquires an image of the parking lot captured by the imaging device; a detection unit that detects a vehicle area corresponding to a vehicle in the acquired image; and an identification unit that identifies the vehicle compartment in which the vehicle is parked based on a positional relationship between the vehicle area in the image and a vehicle compartment area corresponding to each vehicle compartment and a priority order of each vehicle compartment that indicates which of a plurality of vehicle compartments is given priority when identifying the vehicle compartment in which the vehicle is parked. Information processing system.

Images