System and information processing method

Abstract

To provide a new system and information processing method capable of managing an inappropriate parking vehicle with less labor.SOLUTION: A system 1 includes: boundary line identification means 25 for identifying a boundary line of a parking space 92 in an image to be photographed by a photographing device 60; vehicle detection means 46 for detecting a vehicle based on an image received by a reception means 44; determination means 52 for determining whether or not a vehicle detected by the vehicle detection means 46 is being parked in a normal state inside the boundary line to be identified by the boundary line identification means 25; and notification means 28 for transmitting information related to the vehicle determined not to be parked in a normal state inside the boundary line by the determination means 25 to a pre-set terminal.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a system and an information processing method. [Background technology]

[0002] Roadside stations are facilities where drivers can take breaks for meals, naps, etc., or purchase local specialties, and their use has been increasing in recent years. However, there are also occasional cases of so-called shared golfing, where multiple users meet up and then go out for golf or other activities in the same vehicle, or cases where users park their vehicles for long periods of time in the facility's parking facilities (inappropriate parking). When such inappropriate parking occurs frequently, there are no parking spaces available for users who want to use the facility appropriately, resulting in parking lots that are always full but low utilization rates for stores and facilities. Visualizing inappropriately parked vehicles and efforts to optimize parking lot use are beginning to become challenges. For example, Patent Document 1 discloses a parking monitoring system that monitors whether vehicles are parked in a facility's parking lot for fraudulent purposes other than visiting the facility, thereby helping facility operators manage the parking lot more efficiently. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-205035 Summary of the Invention [Problem to be solved by the invention]

[0004] The parking monitoring system described in Patent Document 1 is equipped with multiple monitoring cameras in parking lots, identifies license plates, and determines whether or not a vehicle with the same license plate number has been parked there for a certain period of time. Therefore, there is a need for a new system and method for managing improperly parked vehicles that differs from this system.

[0005] The present disclosure has been made in consideration of these points, and aims to provide a new system and information processing method that can manage improperly parked vehicles with little effort. [Means for solving the problem]

[0006] The system of the present disclosure comprises: a receiving unit for receiving an image captured by the imaging device and transmitted from the imaging device; a boundary line specifying means for specifying a boundary line of the parking space in the image captured by the imaging device; a vehicle detection means for detecting a vehicle based on the image received by the reception means; a determination means for determining whether or not a vehicle detected by the vehicle detection means is parked properly within the boundary line identified by the boundary line identification means; a notification means for transmitting information about a vehicle that is determined by the determination means to be not parked properly within the boundary line to a preset terminal; The present invention is characterized by the following features.

[0007] The system of the present disclosure comprises: a parking start time specifying means for specifying a time when the vehicle detected by the vehicle detection means starts parking within the boundary line, The determination means may determine that a vehicle detected by the vehicle detection means is not parked within the boundary line in a normal manner if the vehicle is still parked within the boundary line even after a preset time has elapsed since the time specified by the parking start time specification means.

[0008] The system of the present disclosure also includes: further comprising an identification information assigning means for assigning identification information to the vehicle detected by the vehicle detection means; The determination means may determine that the vehicle detected by the vehicle detection means has moved in the image captured by the imaging device, based on the assigned identification information, that the vehicle is not parked within the boundary line in a normal manner when a predetermined time has elapsed from the time specified by the parking start time specification means.

[0009] The vehicle detection means also detects the coordinates of the vehicle to which the identification information is assigned based on the image captured by the imaging device, The notification means may transmit to the terminal information relating to a vehicle that has been determined not to be parked within the boundary line in a normal manner, the identification information assigned to the vehicle and information on the vehicle's coordinates linked to this identification information.

[0010] The vehicle detection means defines a square or rectangular bounding box surrounding the vehicle shown in the image received by the reception means, and detects the coordinates of the vehicle based on the defined bounding box; The determination means may determine whether or not the coordinates of the vehicle detected by the vehicle detection means are within the boundary line identified by the boundary line identification means.

[0011] Further, a coordinate correction value is set for each parking space whose boundary line is identified by the boundary line identification means, The vehicle detection means may correct the coordinates of the vehicle relative to the center of a bounding box surrounding the vehicle, which is defined based on the image received by the reception means, based on a coordinate correction value corresponding to the parking space in which the vehicle is parked.

[0012] In addition, the determination means may detect, based on the image received by the reception means, a vehicle parked outside the boundary line as a vehicle that is not parked within the boundary line in a normal manner.

[0013] The system of the present disclosure also includes: The vehicle further includes a second boundary line specifying means for specifying a boundary line of the no-parking space in the image captured by the imaging device, The determination means may determine that a vehicle detected by the vehicle detection means is not parked in a normal manner if it is parked within the boundary line of the no-parking space identified by the second boundary line identification means.

[0014] The system of the present disclosure also includes: The image processing device further includes an image processing unit that processes the image received by the receiving unit so as to surround with a frame any vehicle that is determined by the determining unit to be parked improperly within the boundary line, The notification means may transmit to the terminal an image processed by the image processing means as information relating to the vehicle that has been determined not to be parked properly within the boundary line.

[0015] In addition, the image processing means may be configured to change the manner of the frame surrounding a vehicle that is determined by the determination means to be not parked within the boundary line in a normal manner, depending on the parking time that has elapsed since the time specified by the parking start time specification means.

[0016] The system of the present disclosure also includes: further comprising boundary line information receiving means for receiving information on the boundary line from an external source; The boundary line specifying means may specify the boundary line based on the information about the boundary line received by the boundary line information receiving means.

[0017] The boundary line specifying means may automatically specify the boundary line from an image captured by the imaging device.

[0018] The boundary line specifying means also specifies information about the type of vehicle to be parked within the specified boundary line, The determination means may determine, based on the image received by the reception means, that a vehicle of a type different from the type of vehicle that should be parked inside the boundary line, as identified by the boundary line identification means, is not parked in a normal manner inside the boundary line when the time that the vehicle has been parked inside the boundary line is longer than a predetermined time.

[0019] The boundary line specifying means may specify, as the boundary line, a boundary line that surrounds the parking space in which a plurality of vehicles can be parked.

[0020] In addition, the vehicle detection means may detect a vehicle and its model based on an image captured by the imaging device using learned data generated by machine learning using training data including images of the vehicle and information on the vehicle model received by the reception means.

[0021] Furthermore, the images included in the training data may include images in which a portion of the vehicle is missing.

[0022] The information processing method of the present disclosure includes: An information processing method executed by one or more computers, each having a controller, comprising: the control unit receives an image captured by the imaging device and transmitted from the imaging device; The control unit identifies a boundary line of a parking space in an image captured by the imaging device, The control unit detects a vehicle based on the received image, The control unit determines whether the detected vehicle is parked within the boundary line in a normal manner, The control unit is characterized in that it transmits information about a vehicle that is determined not to be parked properly within the boundary line to a preset terminal. [Effects of the Invention]

[0023] According to the system and information processing method disclosed herein, it is possible to manage improperly parked vehicles with little effort. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a schematic configuration diagram illustrating a system configuration according to an embodiment of the present invention; [Figure 2] 10 is a screen displayed on the display unit of the administrator terminal, showing a parking lot and vehicles parked in the parking lot. [Figure 3] 10 is a screen displayed on the display unit of the administrator terminal, showing a parking lot and vehicles parked in the parking lot. [Figure 4] FIG. 2 is a chart showing transmission and reception of information between the imaging device, the edge AI terminal, the server, and the administrator terminal in the system shown in FIG. [Figure 5] 2 is a flowchart showing the flow of information processing in the system shown in FIG. 1. [Figure 6] 10 is a table showing parking history stored in a memory unit of a server. [Figure 7] 10 is a warning screen displayed on the display unit of the administrator terminal regarding vehicles that have been parked for a long period of time. DETAILED DESCRIPTION OF THE INVENTION

[0025] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present disclosure will now be described with reference to the accompanying drawings, in which: Figures 1 to 7 are diagrams illustrating a system 1 according to the present embodiment and the components of this system 1. As shown in FIG.

[0026] [Overall configuration of System 1] In this embodiment, the system 1 is configured to install multiple imaging devices 60 (described below), such as surveillance cameras, in a parking lot. Based on images captured by each imaging device 60, the system detects improper parking, such as parking outside the boundaries of a parking space 92 (described below) or parking for an extended period of time, and notifies an administrator terminal 70 (described below). In this system 1, the installation of multiple imaging devices 60 in the parking lot covers the entire parking lot. Furthermore, the images captured by the imaging devices 60 are analyzed by an edge AI terminal 40 using AI to detect vehicles within the field of view of the imaging devices 60. The analysis results from each edge AI terminal 40 are transmitted to a server 10, such as a cloud server, at predetermined intervals (e.g., every minute). The server 10 aggregates the analysis results transmitted from each edge AI terminal 40 by area of ​​the parking lot and visualizes the usage status of the entire parking lot. Based on the determination results, the administrator terminal 70 allows users to view the parking status of the parking lot (e.g., full, crowded, empty, etc.).

[0027] The configuration of a parking lot will be described using Figures 2, 3, etc. The parking lot has one or more parking areas 90, and each parking area 90 has one or more parking spaces 92 separated by boundaries. One or more vehicles can be parked in each parking space 92. Each parking area 90 is assigned a set of vehicle types (e.g., motorcycles, cars, large vehicles such as trucks and buses, etc.), and vehicles other than those permitted are prohibited from parking in the parking spaces 92 of the parking area 90. The parking lot may also have no-parking spaces 94. Examples of no-parking spaces 94 include: spaces to ensure the passage of emergency vehicles such as fire engines and ambulances; spaces for ensuring the safety of pedestrians, such as entrances and exits to the parking lot or turning spaces; spaces to ensure smooth vehicle movement; spaces for wheelchair users and access to facilities; and spaces to prohibit parking in specific locations based on the Building Standards Act or local ordinances.

[0028] As shown in FIG. 1, the system 1 of this embodiment includes a server 10, an edge AI terminal 40, an imaging device 60, and an administrator terminal 70. The server 10, the edge AI terminal 40, and the administrator terminal 70 are communicatively connected to each other via a communication network 80, such as the Internet. Multiple edge AI terminals 40 are installed in a parking lot, and each edge AI terminal 40 is communicatively connected to an imaging device 60, such as a camera. Video captured by the imaging device 60 (in this embodiment, a video is formed by the continuous display of still images) is transmitted from the imaging device 60 to the edge AI terminal 40. In other words, a video is composed of multiple frames (still images), each of which represents a specific moment in the video. By displaying the frames continuously and at high speed, the video appears to the human eye as if it were moving. Each component of this system 1 is described in detail below.

[0029] [Configuration of Server 10] The server 10 is a server accessible via a communication network 80, such as the Internet. The server 10 may be provided as a virtual dedicated server built within a physical server accessible via the Internet. The configuration of such a server 10 will be described with reference to FIG. 1. The server 10 includes a control unit 20, a storage unit 34, and a communication unit 36. The control unit 20 is connected to the storage unit 34 and the communication unit 36 ​​via a bus (not shown). The control unit 20 is configured with a circuit including a CPU and semiconductor memory, and controls the operation of the server 10 by executing a program stored in the storage unit 34. Specifically, the control unit 20 functions as a receiving unit 22, an information processing unit 23, a boundary line information receiving unit 24, a boundary line identification unit 25, a second boundary line identification unit 26, and a notification unit 28 by executing the program stored in the storage unit 34.

[0030] The receiving means 22 receives various information from the edge AI terminal 40. The information processing means 23 processes the information received by the receiving means 22 from the edge AI terminal 40. Details of information processing by the information processing means 23 will be described later. The boundary line information receiving means 24 receives information on the boundary lines of the parking areas 90 and parking spaces 92, and information on the boundary lines of the no-parking spaces 94, from the edge AI terminal 40 or the administrator terminal 70. The boundary line identification means 25 identifies the boundary lines of the parking areas 90 and parking spaces 92 based on the information on the boundary lines of the parking areas 90 and parking spaces 92 received by the boundary line information receiving means 24. The second boundary line identification means 26 identifies the boundary lines of the no-parking spaces 94 based on the information on the boundary lines of the no-parking spaces 94 received by the boundary line information receiving means 24. The information on the boundary lines of the parking areas 90 and parking spaces 92 identified by the boundary line identification means 25 and the information on the boundary lines of the no-parking spaces 94 identified by the second boundary line identification means 26 is stored in the memory unit 34 and transmitted from the server 10 to each edge AI terminal 40. The notification means 28 notifies the administrator terminal 70 based on the result of the information processing by the information processing means 23.

[0031] A method by which the boundary line identification means 25 identifies the boundary lines of the parking area 90 and the parking spaces 92 will be described in more detail. An image of the parking lot is captured by the imaging device 60 and transmitted from the imaging device 60 to the edge AI terminal 40, whereupon the edge AI terminal 40 performs image analysis to identify the boundary lines of the parking spaces 92 in the image. Furthermore, based on the boundary lines of the parking spaces 92, the boundary lines of the parking area 90, which has one or more parking spaces 92, are identified. Then, when the identified boundary lines of the parking area 90 and the parking spaces 92 are transmitted from the edge AI terminal 40 to the server 10, the boundary line information receiving means 24 of the server 10 receives the information on the boundary lines of the parking area 90 and the parking spaces 92 identified by the edge AI terminal 40. Then, the boundary line identification means 25 identifies the boundary lines of the parking area 90 and the parking spaces 92 based on the information on the boundary lines of the parking area 90 and the parking spaces 92 received by the boundary line information receiving means 24. Alternatively, an image of the parking lot may be captured by the imaging device 60, and the captured image may be transmitted from the imaging device 60 to the edge AI terminal 40, which may then transmit the image as is to the server 10. Then, when the boundary line information receiving means 24 of the server 10 receives the image from the edge AI terminal 40, the information processing means 23 performs image analysis of the image received by the boundary line information receiving means 24 to identify the boundary lines of the parking area 90 and the parking space 92 in the image. Then, the boundary line identification means 25 identifies the boundary lines of the parking area 90 and the parking space 92 based on the information of the boundary lines of the parking area 90 and the parking space 92 identified by the information processing means 23. Furthermore, information on the boundary lines of the identified parking area 90 and the parking space 92 is stored in the memory unit 34.

[0032] Alternatively, the administrator may input information about the boundary lines of the parking areas 90 and the parking spaces 92 using the administrator terminal 70. At this time, the administrator also inputs the types of vehicles that can be parked in the parking areas 90 and the parking spaces 92 using the administrator terminal 70. The administrator can also set coordinate correction values ​​(described later) for the X and Y coordinates for each parking area 90 or parking space 92. When the administrator inputs information about the boundary lines of the parking areas 90 and the parking spaces 92 using the administrator terminal 70, the information about the boundary lines of the parking areas 90 and the parking spaces 92, information about the types of vehicles that can be parked in the parking areas 90 and the parking spaces 92, and information about the coordinate correction values ​​for each parking area 90 or the parking spaces 92 that have been input by the administrator terminal 70 are transmitted to the administrator terminal 70. When the boundary line information receiving means 24 receives the transmitted information about the boundary lines of the parking areas 90 and the parking spaces 92, the boundary line identification means 25 identifies the boundary lines of the parking areas 90 and the parking spaces 92 based on the information about the boundary lines of the parking areas 90 and the parking spaces 92 that the boundary line information receiving means 24 receives from the administrator terminal 70. In addition, the memory unit 34 stores information on the boundary lines of the identified parking areas 90 or parking spaces 92, information on the types of vehicles that can be parked in the parking areas 90 or parking spaces 92, and information on coordinate correction values ​​for each parking area 90 or parking space 92.

[0033] Next, a method by which the second boundary line identification means 26 identifies the boundary line of the no-parking space 94 will be described in more detail. An image of the parking lot is captured by the imaging device 60 and transmitted from the imaging device 60 to the edge AI terminal 40. The edge AI terminal 40 performs image analysis to identify the boundary line of the no-parking space 94 in the image. The edge AI terminal 40 then transmits the identified boundary line of the no-parking space 94 from the edge AI terminal 40 to the server 10, and the boundary line information receiving means 24 of the server 10 receives information about the boundary line of the no-parking space 94 identified by the edge AI terminal 40. The second boundary line identification means 26 then identifies the boundary line of the no-parking space 94 based on the information about the boundary line of the no-parking space 94 received by the boundary line information receiving means 24. Alternatively, an image of the parking lot may be captured by the imaging device 60 and transmitted from the imaging device 60 to the edge AI terminal 40. The edge AI terminal 40 may then transmit the image as is to the server 10. Then, when the boundary line information receiving means 24 of the server 10 receives an image from the edge AI terminal 40, the information processing means 23 performs image analysis of the image received by the boundary line information receiving means 24 to identify the boundary line of the no-parking space 94 in the image. Then, the second boundary line identification means 26 identifies the boundary line of the no-parking space 94 based on the information on the boundary line of the no-parking space 94 identified by the information processing means 23. In addition, the memory unit 34 stores information on the identified boundary line of the no-parking space 94.

[0034] Alternatively, the administrator may specify the boundary line of the no parking space 94 using the administrator terminal 70. In this case, information about the boundary line of the no parking space 94 specified by the administrator terminal 70 is transmitted to the administrator terminal 70, and when the boundary line information receiving means 24 receives this transmitted information about the boundary line of the no parking space 94, the second boundary line identification means 26 identifies the boundary line of the no parking space 94 based on the information about the boundary line of the no parking space 94 received by the boundary line information receiving means 24 from the administrator terminal 70. In addition, information about the identified boundary line of the no parking space 94 is stored in the memory unit 34.

[0035] The storage unit 34 is configured with a hard disk drive (HDD), random access memory (RAM), read only memory (ROM), or solid state drive (SSD), etc. The storage unit 34 stores programs executed by the control unit 20. The storage unit 34 also stores information on the boundary lines of the parking areas 90 and parking spaces 92 identified by the boundary line identification means 25, information on the types of vehicles that can be parked in the parking areas 90 and parking spaces 92, information on coordinate correction values ​​for each parking area 90 or parking space 92, and information on the boundary lines of the no-parking spaces 94 identified by the second boundary line identification means 26. The storage unit 34 also stores a table showing parking history as shown in FIG. 6.

[0036] The communication unit 36 ​​includes a communication interface that transmits and receives various data between the control unit 20 and each of the control unit 42 (described later) of the edge AI terminal 40 and the control unit 72 (described later) of the administrator terminal 70 via the communication network 80.

[0037] [Configuration of edge AI terminal 40] The configuration of the edge AI terminal 40 will be described with reference to FIG. 1. The edge AI terminal 40 has a control unit 42, a storage unit 54, and a communication unit 56. The control unit 42 is connected to the storage unit 54 and the communication unit 56 via a bus (not shown). The control unit 42 is composed of a circuit including a CPU and semiconductor memory, and controls the operation of the edge AI terminal 40 by executing a program stored in the storage unit 54. Specifically, the control unit 42 functions as a reception unit 44, a vehicle detection unit 46, an identification information assignment unit 48, a parking start time identification unit 50, a determination unit 52, and an image processing unit 53 by executing the program stored in the storage unit 54. In addition, each edge AI terminal 40 is sent from the server 10 information on the boundaries of the parking areas 90 and parking spaces 92 identified by the boundary line identification means 25, information on the types of vehicles that can be parked in the parking areas 90 and parking spaces 92, information on coordinate correction values ​​for each parking area 90 or parking space 92, information on the boundaries of the no-parking spaces 94 identified by the second boundary line identification means 26, etc., and the information sent to each edge AI terminal 40 is stored in the memory unit 54.

[0038] The receiving means 44 receives from the imaging device 60 images captured by the imaging device 60 (specifically, a video consisting of a number of consecutive still images). The vehicle detection means 46 detects vehicles based on the images received by the receiving means 44. Such vehicle detection by the vehicle detection means 46 is performed through AI processing. More specifically, the vehicle detection means 46 detects the type of vehicle contained in the image received by the receiving means 44 using trained data, which is obtained by performing machine learning using training data that includes at least images containing vehicles and vehicle types. The trained data takes images as input and vehicle types as output. The trained data may include images in which a portion of a vehicle is missing. Typically, the imaging device 60 is installed in a parking lot several meters above ground level, and images are captured from the imaging device 60 looking down on the parking lot. Therefore, multiple adjacent vehicles may overlap in the image captured by the imaging device 60, resulting in an image in which some vehicles are partially missing due to other vehicles obstructing the view. Even in this case, by including an image of the vehicle with a portion missing in the images included in the training data, such vehicles can be detected with high accuracy.

[0039] The vehicle detection means 46 also detects information about the coordinates (specifically, X and Y coordinates) of vehicles parked in the parking lot. Specifically, the vehicle detection means 46 defines a square or rectangular bounding box 96 that surrounds the vehicle shown in the image received by the reception means 44, and detects the coordinates of the vehicle based on the defined bounding box 96. Specifically, the center of the defined bounding box 96 is set as the coordinate of the vehicle. At this time, the coordinates of the vehicle may be corrected relative to the center of the bounding box 96 based on a coordinate correction value corresponding to the parking area 90 or parking space 92 in which the vehicle is parked. In FIGS. 2 and 3, the location of the corrected vehicle coordinates is indicated by reference numeral 98. As described above, the imaging device 60 is installed in the parking lot at a location several meters above the ground, and images are captured from the imaging device 60 looking down on the parking lot. Furthermore, vehicles parked in the parking lot have a certain height. For this reason, if the center of a square or rectangular bounding box 96 surrounding the vehicle shown in the image received by the receiving means 44 is used as the coordinates of the vehicle, the center of this bounding box 96 will be misaligned with the center of the actual bottom surface of the vehicle, which could result in an error in determining whether the vehicle is parked within the boundary line of the parking space 92 when the determining means 52, which will be described later, determines whether the vehicle is parked within the boundary line of the parking space 92. For this reason, by setting a coordinate correction value for each parking area 90 or parking space 92 whose boundary line is identified by the boundary line identifying means 25 and correcting the coordinates from the center of the bounding box 96 using the coordinate correction value, it becomes possible to more accurately determine whether the vehicle is parked within the boundary line of the parking space 92.

[0040] The identification information assignment means 48 assigns identification information to vehicles detected by the vehicle detection means 46. The parking start time identification means 50 identifies the time when a vehicle detected by the vehicle detection unit started parking inside the boundary line of a parking space 92 or inside the boundary line of a no-parking space 94. The determination means 52 determines whether a vehicle detected by the vehicle detection means 46 is parked properly within the boundary line of the parking space 92 identified by the boundary line identification means 25. The determination result by the determination means 52 is transmitted from the edge AI terminal 40 to the server 10. The image processing means 53 processes the image received by the reception means 44 so as to surround with a frame any vehicle determined by the determination means 52 to be parked improperly within the boundary line. The processed image is also transmitted from the edge AI terminal 40 to the server 10.

[0041] The storage unit 54 is configured with a hard disk drive (HDD), random access memory (RAM), read only memory (ROM), or solid state drive (SSD), etc. The storage unit 54 stores programs executed by the control unit 42.

[0042] The communication unit 56 includes a communication interface that transmits and receives various data between the control unit 42 and each of the control units 42 of the imaging device 60 and the edge AI terminal 40 via the communication network 80.

[0043] [Configuration of imaging device 60] The imaging devices 60 are installed in various locations in the parking lot, and each imaging device 60 is connected to a corresponding edge AI terminal 40. Images captured by the imaging devices 60 (specifically, videos made up of a number of consecutive still images) are transmitted from the imaging devices 60 to the corresponding edge AI terminal 40. A general-purpose camera or the like is used as the imaging device 60. Note that the imaging device 60 and the edge AI terminal 40 may be integrated into one device.

[0044] [Configuration of administrator terminal 70] The configuration of the administrator terminal 70 will be described with reference to FIG. 1. The administrator terminal 70 is installed, for example, in the office of a business company that manages the server 10, and is operated by an administrator who manages the server 10. The administrator terminal 70 includes, but is not limited to, a personal computer, a smartphone, a PC tablet, etc. As shown in FIG. 1, the administrator terminal 70 has a control unit 72, a display unit 74, an operation unit 76, a storage unit 78, and a communication unit 79. The control unit 72 is connected to each of the display unit 74, the operation unit 76, the storage unit 78, and the communication unit 79 via a bus (not shown).

[0045] The control unit 72 is composed of a circuit including a CPU and semiconductor memory, and controls the operation of the administrator terminal 70 by executing a program stored in the storage unit 78. The display unit 74 is composed of, for example, a liquid crystal display or the like, and functions as a means for displaying various information. The display unit 74 displays various information in response to instructions from the control unit 72. The operation unit 76 functions as a means for inputting various instructions by the user. For example, a keyboard, a mouse, a touch panel, etc. are used as the operation unit 76. When the operation unit 76 is a touch panel, such a touch panel is superimposed on the display unit 74, and an operation signal is input to the control unit 72 when the user touches the touch panel.

[0046] The storage unit 78 is configured with a hard disk drive (HDD), random access memory (RAM), read only memory (ROM), or solid state drive (SSD), etc. The storage unit 78 stores programs executed by the control unit 72. The communication unit 79 includes a communication interface that transmits and receives various data between the control unit 72 and an external device such as the control unit 20 of the server 10 via a communication network 80.

[0047] [System 1 operation] Next, the operation of the system 1 according to this embodiment will be described with reference to Figs. 2 to 7. Figs. 2 and 3 are screens displayed on the display unit 74 of the administrator terminal 70, respectively, showing a parking lot and a vehicle parked therein. Fig. 4 is a chart showing the transmission and reception of information between the imaging device 60, the edge AI terminal 40, the server 10, and the administrator terminal 70 in the system 1 shown in Fig. 1, and Fig. 5 is a flowchart showing the flow of information processing in the system 1 shown in Fig. 1. Fig. 6 is a table showing parking history stored in the memory unit 34 of the server 10, and Fig. 7 is a warning screen displayed on the display unit 74 of the administrator terminal 70 regarding vehicles that have been parked for a long period of time.

[0048] When an image of a parking lot and a vehicle parked therein is captured by an imaging device 60 such as a camera (step ST1 in FIG. 5), the images of the parking lot and the vehicle parked therein captured by the imaging device 60 are transmitted as image information from the imaging device 60 to the edge AI terminal 40 over time. When the reception means 44 of the edge AI terminal 40 receives the image information from the imaging device 60, the vehicle detection means 46 detects a vehicle based on the image received by the reception means 44. Specifically, the vehicle detection means 46 defines a square or rectangular bounding box 96 that surrounds the vehicle shown in the image received by the reception means 44 (see FIGS. 2, 3, etc.), and detects the coordinates of the vehicle based on the defined bounding box 96 (step ST2 in FIG. 5). At this time, as described above, a coordinate correction value is set for each parking area 90 or parking space 92 whose boundary line is identified by the boundary line identification means 25, and the vehicle detection means 46 corrects the coordinates of the vehicle based on the coordinate correction value corresponding to the parking area 90 or parking space 92 in which the vehicle is parked, relative to the center of a bounding box 96 that surrounds the vehicle and is defined based on the image accepted by the acceptance means 44. The location of the corrected coordinates is indicated by reference numeral 98 in Figure 2 etc.

[0049] Furthermore, the identification information assigning means 48 assigns identification information to the vehicle detected by the vehicle detection means 46. Specifically, for example, a five-digit identification number is assigned as identification information to the vehicle detected by the vehicle detection means 46 (step ST3 in FIG. 5). As shown in FIGS. 2 and 3, the identification information (specifically, the five-digit identification number) assigned to the vehicle parked in the parking lot may be displayed on the display unit 74 of the administrator terminal 70 next to the vehicle. The identification information displayed on the display unit 74 is indicated by reference numeral 99. Furthermore, the parking start time identifying means 50 identifies the time when the vehicle detected by the vehicle detection means 46 started parking within the boundary line of the parking space 92 identified by the boundary line identifying means 25. As a method for determining the time when parking within the boundary line of the parking space 92 is started by the parking start time determination means 50, the point at which the vehicle enters within the boundary line of the parking space 92 determined by the boundary line determination means 25 may be determined as the parking start time, or if the vehicle remains within the boundary line of the parking space 92 even after a predetermined time (e.g., 5 minutes) has elapsed since the vehicle entered within the boundary line of the parking space 92 determined by the boundary line determination means 25, this point may be determined as the parking start time.

[0050] Furthermore, the vehicle detection means 46 detects the type of vehicle detected by the vehicle detection means 46 (for example, a motorcycle, an automobile, a large vehicle such as a truck or a bus, etc.) (step ST4 in FIG. 5). Furthermore, if a vehicle detected by the vehicle detection means 46 in an image captured by the imaging device 60 moves, the vehicle detection means 46 tracks the moved vehicle based on the identification information assigned to the vehicle (step ST5 in FIG. 5). Furthermore, information on the type of vehicle may be displayed next to the identification information of the vehicle parked in the parking lot, which is displayed on the display unit 74 of the administrator terminal 70. The information on the type of vehicle displayed on the display unit 74 is indicated by reference numeral 99.

[0051] With respect to a vehicle detected by the vehicle detection means 46, information such as the type of vehicle detected by the vehicle detection means 46, its coordinates, the identification information (specifically, the identification number) assigned by the identification information assignment means 48, the parking start time identified by the parking start time identification means 50, the parking end time at which the vehicle parked within the boundary line of the parking space 92 identified by the boundary line identification means 25 exits the boundary line, the parking duration, and the like are transmitted from the edge AI terminal 40 to the server 10 together with the identification information of the imaging device 60. When the reception means 22 of the server 10 receives this information, the information processing means 23 stores the information received by the reception means 22 as parking history in the memory unit 34. FIG. 6 is a table showing the parking history stored in the memory unit 34 of the server 10.

[0052] The determination means 52 of the edge AI terminal 40 determines whether the vehicle detected by the vehicle detection means 46 is parked properly within the boundary line of the parking space 92 identified by the boundary line identification means 25 (step ST6 in FIG. 5 ). Specifically, if the vehicle detected by the vehicle detection means 46 is still parked within the boundary line of the parking space 92 even after a preset time has passed since the time identified by the parking start time identification means 50, the determination means 52 determines that the vehicle has been parked for a long time and is not parked properly within the boundary line. Furthermore, even if the vehicle detected by the vehicle detection means 46 in the image captured by the imaging device 60 has moved, the determination means 52 determines that the vehicle has been parked for a long time and is not parked properly within the boundary line based on the assigned identification information if a preset time has passed since the time identified by the parking start time identification means 50. Furthermore, the determination means 52 may detect, based on the image received by the reception means 44, a vehicle parked outside the boundary line of the parking space 92 identified by the boundary line identification means 25 as a vehicle that is not parked within the boundary line in a normal manner. Furthermore, the determination means 52 may determine that a vehicle detected by the vehicle detection means 46 is not parked in a normal manner if the vehicle is parked within the boundary line of the no-parking space 94 identified by the second boundary line identification means 26.

[0053] The determination result by the determination means 52 is transmitted from the edge AI terminal 40 to the server 10 (step ST7 in FIG. 5). At this time, the image captured by the imaging device 60 is also transmitted from the edge AI terminal 40 to the server 10. More specifically, the image processing means 53 processes the image received by the reception means 44 from the imaging device 60 so as to surround with a frame any vehicle determined by the determination means 52 to be parked improperly within the boundary line of the parking space 92. At this time, the image processing means 53 changes the style of the frame surrounding the vehicle determined by the determination means 52 to be parked improperly within the boundary line, depending on the parking time elapsed since the time identified by the parking start time identification means 50. For example, for a vehicle parked for a long period of time, a green frame is used if the parking time is 6 to 8 hours, a yellow frame is used if the parking time is 8 to 12 hours, and an orange frame is used if the parking time is 12 hours or more. Furthermore, if a vehicle has been parked for two hours or more outside the boundary line of the parking space 92 identified by the boundary line identification means 25, the frame surrounding the vehicle will be a light blue frame. For a vehicle that is determined by the determination means 52 to be not parked properly within the boundary line of the parking space 92, an image processed by the image processing means 53 is transmitted from the edge AI terminal 40 to the server 10.

[0054] The edge AI terminal 40 transmits the determination result and information on the image processed by the image processing means 53 to the server 10. When the reception means 22 receives this information, the information processing means 23 processes this information. Specifically, the information processing means 23 stores the information received by the reception means 22 in the storage unit 34 and causes the notification means 28 to notify the administrator terminal 70 of a vehicle determined to be parked improperly within the boundary line of the parking space 92. Specifically, a report regarding the vehicle determined to be parked improperly within the boundary line of the parking space 92 is transmitted from the server 10 to the administrator terminal 70 (step ST8 in FIG. 5). Then, the display unit 74 of the administrator terminal 70 displays the report regarding the vehicle determined to be parked improperly within the boundary line of the parking space 92 (step ST9 in FIG. 5). The report displayed on the display unit 74 is shown in FIG. 7. 7, the report includes information on the parking area 90, information on the parking space 92, parking start time, parking end time, parking duration, vehicle type, coordinates, identification information, parking status (specifically, long-term parking of six hours or more), and an image processed by the image processing means 53, which are displayed on the display unit 74. This allows the administrator operating the administrator terminal 70 to recognize detailed information about a vehicle that has been determined not to be parked properly within the boundary line of the parking space 92.

[0055] According to the system 1 and information processing method of the present embodiment configured as described above, the receiving means 44 receives the image captured by the imaging device 60 and transmitted from the imaging device 60, the boundary line identification means 25 identifies the boundary line of the parking space 92 in the image captured by the imaging device 60, the vehicle detection means 46 detects vehicles based on the image received by the receiving means 44, the determination means 52 determines whether the vehicles detected by the vehicle detection means 46 are parked properly within the boundary line identified by the boundary line identification means 25, and the notification means 28 transmits information about vehicles determined by the determination means 52 to be not parked properly within the boundary line to a preset terminal. Such system 1 and information processing method enable management of improperly parked vehicles with little effort.

[0056] Furthermore, as described above, the system 1 and information processing method of this embodiment are further provided with parking start time identification means 50 that identifies the time when a vehicle detected by the vehicle detection means 46 started parking inside the boundary line, and the determination means 52 determines that a vehicle detected by the vehicle detection means 46 is not parked inside the boundary line in a normal manner if the vehicle is still parked inside the boundary line even after a preset time has elapsed since the time identified by the parking start time identification means 50. In this case, it is possible to detect long-parked vehicles that have been parked for longer than the preset time.

[0057] In addition, identification information assigning means 48 is provided for assigning identification information to vehicles detected by the vehicle detection means 46, and the determination means 52 determines, based on the assigned identification information, that the vehicle is not parked properly within the boundary line when a preset time has passed since the time identified by the parking start time identification means 50, even if the vehicle detected by the vehicle detection means 46 in the image captured by the imaging device 60 has moved. In this case, even if the vehicle is moved after being parked in the parking lot, it can be determined whether the vehicle has been parked for a long period of time based on the total parking time, including the time the vehicle has been parked after being moved.

[0058] Furthermore, the vehicle detection means 46 also detects the coordinates of the vehicle to which identification information has been assigned based on the image captured by the imaging device 60, and the notification means 28 transmits the identification information assigned to the vehicle and information on the vehicle coordinates linked to this identification information to a terminal (for example, the administrator terminal 70) as information regarding the vehicle determined to be not parked properly within the boundary line. In this case, the identification information assigned to the vehicle and information on the vehicle coordinates linked to this identification information can be recognized by the operator of the terminal (for example, the administrator operating the administrator terminal 70).

[0059] Furthermore, the vehicle detection means 46 defines a square or rectangular bounding box 96 that surrounds the vehicle shown in the image received by the reception means 44, and detects the coordinates of the vehicle based on the defined bounding box 96. The determination means 52 determines whether the coordinates of the vehicle detected by the vehicle detection means 46 are within the boundary line identified by the boundary line identification means 25. In this case, it is possible to accurately determine whether the vehicle is parked within the parking space 92.

[0060] Furthermore, a coordinate correction value is set for each parking area 90 or parking space 92 whose boundary line is identified by the boundary line identification means 25, and the vehicle detection means 46 corrects the coordinates of the vehicle based on the coordinate correction value corresponding to the parking area 90 or parking space 92 in which the vehicle is parked, relative to the center of a bounding box 96 surrounding the vehicle, which is defined based on the image received by the reception means 44. This allows the coordinates (X coordinate, Y coordinate) of the bottom of a tall vehicle to be detected with higher accuracy, even if an image is captured looking down on the parking lot from the imaging device 60 installed several meters above the ground. Note that the term "parking space" in the claims is a concept that includes both the parking area 90 and the parking space 92 of this embodiment.

[0061] Furthermore, the determination means 52 detects a vehicle parked outside the boundary line of the parking space 92 as a vehicle that is not parked within the boundary line in a normal manner, based on the image received by the reception means 44. In this case, the determination means 52 can determine that a vehicle that is not parked within the boundary line of the parking space 92 is not parked within the boundary line in a normal manner.

[0062] Also provided is a second boundary line identification means 26 that identifies the boundary line of the no-parking space 94 in the image captured by the imaging device 60, and the determination means 52 determines that a vehicle detected by the vehicle detection means 46 is not parked in a normal manner if the vehicle is parked inside the boundary line of the no-parking space 94 identified by the second boundary line identification means 26. In this case, when the vehicle is parked inside the boundary line of the no-parking space 94, the determination means 52 can determine that the vehicle is not parked in a normal manner.

[0063] Furthermore, image processing means 53 is provided for processing the image received by the receiving means 44 so that vehicles that are determined by the determining means 52 to be parked improperly within the boundary line are surrounded by a frame, and the notification means 28 transmits the image processed by the image processing means 53 to a terminal (e.g., administrator terminal 70) as information relating to the vehicles that are determined to be parked improperly within the boundary line. In this case, since the vehicles that are determined to be parked improperly within the boundary line are surrounded by a frame, the operator of the terminal (e.g., the administrator operating the administrator terminal 70) can more reliably recognize these vehicles.

[0064] Furthermore, the image processing means 53 changes the manner of the frame surrounding a vehicle that is determined by the determination means 52 to be not parked properly within the boundary line, in accordance with the parking time that has elapsed since the time specified by the parking start time specifying means 50. In this case, the terminal operator (for example, the administrator operating the administrator terminal 70) can recognize at a glance how long a vehicle has been parked for a long period of time.

[0065] Also provided is boundary line information receiving means 24 that receives boundary line information from the outside, and boundary line identification means 25 identifies the boundary line based on the boundary line information received by the boundary line information receiving means 24. In this case, the administrator of the system 1 can set the boundary lines of the parking spaces 92 using the administrator terminal 70 or the like. At this time, the administrator may also be able to set the types of vehicles that can be parked in the parking spaces 92 and coordinate correction values ​​(described later) for the X and Y coordinates for each parking space 92 using the administrator terminal 70 or the like.

[0066] Furthermore, the boundary line specifying means 25 may automatically specify the boundary line from the image captured by the imaging device 60. In this case, the boundary line of the parking space 92 can be automatically set without the manager having to set the boundary line of the parking space 92.

[0067] The boundary line identification means 25 is also configured to identify information about the vehicle type of the vehicle that should be parked inside the identified boundary line, and the determination means 52 determines that the vehicle is not parked properly inside the boundary line when a vehicle of a type different from the vehicle that should be parked inside the boundary line identified by the boundary line identification means 25 has been parked inside the boundary line for longer than a predetermined time, based on the image received by the reception means 44. In this case, the determination means 52 can determine that the vehicle is not parked properly when a vehicle of a type different from the vehicle that should be parked inside the boundary line has been parked inside the boundary line for longer than the predetermined time.

[0068] Furthermore, boundary line identification means 25 may identify a boundary line that surrounds parking space 92 where multiple vehicles can be parked (specifically, a boundary line of parking area 90, etc.). In this case, when a vehicle is parked outside of a boundary line that surrounds parking space 92 where multiple vehicles can be parked, determination means 52 can determine that the vehicle is not parked in a normal manner.

[0069] Furthermore, the vehicle detection means 46 is configured to detect a vehicle and its vehicle model based on the image captured by the imaging device 60, using learned data generated by machine learning using training data including the vehicle image and vehicle model information received by the reception means 44. In this case, the vehicle detection means 46 can detect the vehicle model with higher accuracy.

[0070] Furthermore, images included in the training data may include images in which parts of vehicles are missing. When an image is captured from an imaging device 60 installed several meters above the ground in a parking lot looking down on the parking lot, multiple vehicles may appear overlapping in the image captured by the imaging device 60, and some vehicles may be obscured by other vehicles, resulting in an image in which parts of the vehicles are missing. However, by including images in which parts of vehicles are missing in the training data, such vehicles can also be detected with high accuracy.

[0071] The system and information processing method of the present invention are not limited to the above-described aspects, and various modifications can be made.

[0072] For example, in the above description, the vehicle detection means 46, identification information assignment means 48, parking start time identification means 50, determination means 52, and image processing means 53 are provided in the edge AI terminal 40, and the boundary line identification means 25, second boundary line identification means 26, and notification means 28 are provided in the server 10. However, the system of the present invention is not limited to this configuration. Some or all of the vehicle detection means 46, identification information assignment means 48, parking start time identification means 50, determination means 52, and image processing means 53 may be provided in the server 10 rather than the edge AI terminal 40, and some or all of the boundary line identification means 25, second boundary line identification means 26, and notification means 28 may be provided in the edge AI terminal 40 rather than the server 10.

[0073] Furthermore, the system of the present invention may not include the edge AI terminal 40, but may instead have the imaging device 60 and the server 10 communicably connected via a communication network 80 such as the Internet, and images captured by the imaging device 60 transmitted directly to the server 10. In this case, the vehicle detection means 46, identification information assignment means 48, parking start time specification means 50, determination means 52, and image processing means 53 shown in FIG. 1 are provided in the server 10.

[0074] Alternatively, the system of the present invention may not include server 10, but may instead have edge AI terminal 40 and administrator terminal 70 communicably connected via a communication network 80 such as the Internet, with the edge AI terminal 40 creating the above-mentioned report and transmitting the created report directly to administrator terminal 70. In this case, boundary line identification means 25, second boundary line identification means 26, notification means 28, etc., shown in FIG. 1, are provided in edge AI terminal 40.

[0075] Furthermore, the method by which the determination means 52 determines whether a vehicle detected by the vehicle detection means 46 is parked properly within the boundary line identified by the boundary line identification means 25 is not limited to the above-described method. As long as it is possible to determine whether a vehicle is parked properly within the boundary line identified by the boundary line identification means 25, the determination means 52 may make the determination using a method other than the above-described method. [Explanation of symbols]

[0076] 1 System 10 Servers 20 Control Unit 22 Reception methods 23 Information Processing Means 24 Boundary information reception method 25 Boundary identification means 26 Second boundary line identification means 28 Notification methods 34 Storage section 36 Communications Department 40 Edge AI terminal 42 Control Unit 44 Reception methods 46 Vehicle detection means 48 Identification information assignment means 50 Parking start time determination means 52 Judgment means 53 Image processing methods 54 Memory section 56 Communications Department 60 Imaging device 70 Administrator terminal 72 Control Unit 74 Display section 76 Control section 78 Memory section 79 Communications Department 80 Communication Network 90 Parking Area 92 parking spaces 94 No Parking Space 96 Bounding Box 98 Corrected vehicle coordinates 99 Vehicle type and identification information

Claims

1. a receiving unit for receiving an image captured by the imaging device and transmitted from the imaging device; a boundary line specifying means for specifying a boundary line of the parking space in the image captured by the imaging device; a vehicle detection means for detecting a vehicle based on the image received by the reception means; a determination means for determining whether or not a vehicle detected by the vehicle detection means is parked properly within the boundary line identified by the boundary line identification means; a notification means for transmitting information about a vehicle that is determined by the determination means to be not parked properly within the boundary line to a preset terminal; A system equipped with

2. a parking start time specifying means for specifying a time when the vehicle detected by the vehicle detection means starts parking within the boundary line, 2. The system according to claim 1, wherein the determination means determines that the vehicle detected by the vehicle detection means is not parked within the boundary line in a normal manner if the vehicle is still parked within the boundary line even after a predetermined time has elapsed since the time specified by the parking start time specification means.

3. further comprising an identification information assigning means for assigning identification information to the vehicle detected by the vehicle detection means; 3. The system according to claim 2, wherein the determination means determines that the vehicle detected by the vehicle detection means has moved in the image captured by the imaging device, based on the assigned identification information, that the vehicle is not parked within the boundary line in a normal manner when a preset time has elapsed from the time specified by the parking start time specification means.

4. the vehicle detection means detects the coordinates of the vehicle to which the identification information is assigned based on the image captured by the imaging device; The system described in claim 3, wherein the notification means transmits to the terminal information about a vehicle that is determined not to be parked properly within the boundary line, the identification information assigned to the vehicle and information about the vehicle's coordinates linked to this identification information.

5. the vehicle detection means defines a square or rectangular bounding box surrounding the vehicle shown in the image received by the reception means, and detects the coordinates of the vehicle based on the defined bounding box; 2. The system according to claim 1, wherein the determining means determines whether or not the coordinates of the vehicle detected by the vehicle detecting means are within the boundary line identified by the boundary line identifying means.

6. a coordinate correction value is set for each parking space whose boundary line is identified by the boundary line identification means; 6. The system according to claim 5, wherein the vehicle detection means corrects the coordinates of the vehicle relative to the center of a bounding box surrounding the vehicle, which is defined based on the image received by the reception means, based on a coordinate correction value corresponding to the parking space in which the vehicle is parked.

7. The system according to claim 1, wherein the determining means detects a vehicle parked outside the boundary line as a vehicle that is not parked within the boundary line in a normal manner, based on the image received by the receiving means.

8. The vehicle further includes a second boundary line specifying means for specifying a boundary line of the no-parking space in the image captured by the imaging device, 2. The system according to claim 1, wherein the determination means determines that a vehicle detected by the vehicle detection means is not parked in a normal manner if the vehicle is parked within the boundary line of the no-parking space identified by the second boundary line identification means.

9. The image processing device further includes an image processing unit that processes the image received by the receiving unit so as to surround with a frame any vehicle that is determined by the determining unit to be parked improperly within the boundary line, 3. The system according to claim 2, wherein the notification means transmits to the terminal an image processed by the image processing means as information relating to a vehicle determined to be parked improperly within the boundary line.

10. 10. The system according to claim 9, wherein the image processing means changes the manner of the frame surrounding the vehicle that is determined by the determination means to be not parked within the boundary line in a normal manner, in accordance with the parking time that has elapsed since the time specified by the parking start time specification means.

11. further comprising boundary line information receiving means for receiving information on the boundary line from an external source; 2. The system according to claim 1, wherein the boundary line specifying means specifies the boundary line based on the information about the boundary line received by the boundary line information receiving means.

12. The system according to claim 1 , wherein the boundary line specifying means automatically specifies the boundary line from an image captured by the imaging device.

13. the boundary line specifying means also specifies information on the type of vehicle to be parked within the specified boundary line, 2. The system according to claim 1, wherein the determining means determines that a vehicle of a type different from the type of vehicle that should be parked within the boundary line, as identified by the boundary line identifying means, is not parked within the boundary line in a normal manner when the time that the vehicle has been parked within the boundary line is longer than a predetermined time, based on the image received by the receiving means.

14. The system according to claim 1 , wherein the boundary line specifying means specifies, as the boundary line, a boundary line surrounding the parking space capable of parking a plurality of vehicles.

15. The system according to claim 1, wherein the vehicle detection means detects a vehicle and its model based on an image captured by the imaging device using learned data generated by machine learning using training data including images of the vehicle and information on the vehicle model received by the reception means.

16. The system according to claim 15 , wherein the images included in the training data include an image in which a part of a vehicle is missing.

17. An information processing method executed by one or more computers, each having a controller, comprising: the control unit receives an image captured by the imaging device and transmitted from the imaging device; The control unit identifies a boundary line of a parking space in an image captured by the imaging device, The control unit detects a vehicle based on the received image, The control unit determines whether the detected vehicle is parked within the boundary line in a normal manner, The control unit transmits information about a vehicle that is determined not to be parked properly within the boundary line to a preset terminal.

Images