Information processing systems, information processing methods, and programs

Abstract

To improve the security of the parking lot. [Solution] The information processing system according to this disclosure comprises at least one processor. The at least one processor manages parking management information relating to a plurality of parking spaces provided in a parking lot, which includes a parking space that is monitored, where a parked vehicle is photographed by an on-board camera of another vehicle; at least one monitoring parking space, where a parked vehicle photographs a vehicle parked in the monitoring parking space; and a vehicle parked in the parking lot. The processor acquires images obtained from a fixed camera capable of photographing the entire plurality of parking spaces, and, based on the parking management information, selects an on-board camera from the on-board cameras of vehicles parked in the monitoring parking space corresponding to the first parking space to photograph the first vehicle parked in the first parking space, and acquires images obtained from the selected on-board camera.

Description

Technical Field

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Conventionally, a security camera has been installed in a parking lot, and the parking lot has been monitored using the images obtained by the security camera. In such monitoring of a parking lot, for example, by increasing the number of installed cameras, it is possible to reduce the blind spots of the security cameras and improve the security, but the installation and maintenance costs increase. In addition, in the monitoring of parked vehicles, from the viewpoint of facilitating the identification of the perpetrator and the perpetrator's vehicle, it is preferable to take pictures at an angle closer to the human eye level than the downward viewing angle from above. Also, in the in-vehicle cameras of parked vehicles, there were cases where the characteristics of the perpetrator and the perpetrator's vehicle could not be fully captured due to their blind spots.

[0003] For example, Patent Document 1 discloses a technique of communicating with other vehicles existing around the own vehicle, acquiring images of in-vehicle cameras mounted on the other vehicles, selecting an image in which the own vehicle is reflected from the acquired images, and saving the selected image as an image to compensate for the blind spot of the in-vehicle camera of the own vehicle during parking.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when using the in-vehicle cameras of other vehicles for monitoring the own vehicle during parking, it is necessary to dynamically determine which in-vehicle camera of the surrounding other vehicles to use for monitoring the own vehicle, and there is a problem that the communication and processing costs required for this determination are high.

[0006] This disclosure aims, in part, to improve the security of parking lots. [Means for solving the problem]

[0007] The information processing system according to this disclosure comprises at least one processor. The at least one processor manages parking management information relating to a plurality of parking spaces provided in a parking lot, which includes a monitored parking space where a parked vehicle is photographed by an on-board camera of another vehicle parked in another parking space, at least one monitoring parking space where a parked vehicle photographs a vehicle parked in the monitored parking space by an on-board camera, and a vehicle parked in the parking lot. The at least one processor acquires images obtained by a fixed camera installed in the parking lot capable of photographing the entire plurality of parking spaces. Based on the parking management information, the at least one processor selects an on-board camera to photograph a first vehicle parked in a first parking space among the plurality of parking spaces from the on-board cameras of vehicles parked in the at least one monitoring parking space that corresponds to the first parking space as the monitored parking space. The at least one processor acquires images obtained by the selected on-board camera. [Effects of the Invention]

[0008] According to this disclosure, the security of parking lots can be improved. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows an example of the configuration of a monitoring system according to an embodiment. [Figure 2] Figure 2 shows an example of the functional configuration of the monitoring system according to the embodiment. [Figure 3] Figure 3 shows an example of the hardware configuration of an information processing device that realizes each device of the monitoring system according to the embodiment. [Figure 4] Figure 4 shows an example of the configuration of a parking management ledger according to an embodiment. [Figure 5] Figure 5 shows an example of the configuration of mode setting information according to the embodiment. [Figure 6] Figure 6 is a sequence diagram showing an example of the flow of monitoring processing related to mode setting, which is performed by the monitoring system according to the embodiment. [Figure 7] Figure 7 is a flowchart showing an example of the flow of the second mode monitoring process executed by the server according to the embodiment. [Figure 8] Figure 8 is a sequence diagram showing an example of the flow of monitoring processing at the time of receiving goods, which is performed by the monitoring system according to the embodiment. [Figure 9] Figure 9 is a sequence diagram showing an example of the flow of monitoring processing at the time of shipment, which is performed by the monitoring system according to the embodiment. [Figure 10] Figure 10 is a diagram illustrating the updating of the parking management ledger in the second mode monitoring process according to the embodiment. [Figure 11] Figure 11 is a diagram illustrating the generation of recorded data when an anomaly is detected in the monitoring process according to the embodiment. [Figure 12] Figure 12 shows an example of the configuration of the first recorded data based on on-board cameras of surrounding vehicles, which is generated when an anomaly is detected in the monitoring process according to the embodiment. [Figure 13] Figure 13 shows an example of the configuration of the second recorded data based on a fixed camera, which is generated when an anomaly is detected in the monitoring process according to the embodiment. [Figure 14] Figure 14 is a sequence diagram showing an example of the flow of monitoring processing when an anomaly is detected, as performed by the monitoring system according to the embodiment. [Figure 15] Figure 15 is a flowchart showing an example of the ledger update process flow in the first mode of monitoring, as executed by the server according to the embodiment. [Figure 16] Figure 16 shows an example of the configuration of the parking management ledger in the first mode monitoring process according to the embodiment. [Figure 17]FIG. 17 is a diagram for explaining the update of the parking management ledger in the monitoring process of the first mode according to the embodiment. [Figure 18] FIG. 18 is a diagram showing an example of the configuration of the parking management ledger in the monitoring process of the third mode according to the embodiment. [Figure 19] FIG. 19 is a flowchart showing an example of the flow of the ledger update process in the monitoring process of the third mode executed by the server according to the embodiment. [Figure 20] FIG. 20 is a diagram for explaining the update of the parking management ledger in the monitoring process of the third mode according to the embodiment.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of an information processing apparatus, an information processing system, an information processing method, a program, and a recording medium according to the present disclosure will be described with reference to the drawings.

[0011] In the description of the present disclosure, components having the same or substantially the same functions as those described above with respect to the already shown figures may be denoted by the same reference numerals, and the description may be omitted as appropriate. Also, even when representing the same or substantially the same part, the dimensions and ratios may be shown differently in the drawings. Further, for example, from the viewpoint of ensuring the visibility of the drawings, only main components are given reference numerals in the description of each drawing, and components having the same or substantially the same functions as those described above in the already shown figures may not be given reference numerals.

[0012] In the description of the present disclosure, components having the same or substantially the same functions may sometimes be described by adding alphanumeric characters and / or symbols to the end of the reference numerals for distinction. Alternatively, when not distinguishing between a plurality of components having the same or substantially the same functions, they may sometimes be described in an integrated manner by omitting the alphanumeric characters and / or symbols attached to the end of the reference numerals.

[0013] Figure 1 shows an example of the configuration of the monitoring system 1 according to the embodiment. Here, the monitoring system 1 according to the embodiment is an example of an information processing system.

[0014] As shown in Figure 1, the monitoring system 1 includes at least one vehicle 2 parked in the parking lot 101 and an infrastructure unit 3 applied to the parking lot 101. As shown in Figure 1, the infrastructure unit 3 includes a fixed camera 4 and a server 5.

[0015] Parking lot 101 has multiple parking spaces 103, each with a designated space number. Figure 1 shows "Parking Space A" to "Parking Space J" as examples of the multiple parking spaces 103 in parking lot 101. In the example in Figure 1, "Parking Space A" to "Parking Space E" are multiple parking spaces 103 arranged in a row. Also, "Parking Space F" to "Parking Space J" are multiple parking spaces 103 arranged in a row. Furthermore, "Parking Space A" to "Parking Space E" are parking spaces 103 opposite "Parking Space F" to "Parking Space J".

[0016] For example, "Parking Space B" is parking space 103 adjacent to "Parking Space A" and "Parking Space C," and facing "Parking Space G," and vehicle 2-1 is parked in it.

[0017] For example, "Parking Space H" is parking space 103 adjacent to "Parking Space G" and "Parking Space I," and facing "Parking Space C," and vehicle 2-2 is parked there.

[0018] For example, "Parking Space J" is parking space 103 adjacent to "Parking Space I" and facing "Parking Space E," and vehicles 2-3 are parked there.

[0019] Note that the arrangement and number of parking spaces 103 in parking lot 101 are just an example and can be changed as appropriate.

[0020] Note that while Figure 1 illustrates a case where three vehicles 2 are parked in parking lot 101, it is not limited to this. For example, the number of vehicles 2 included in the monitoring system 1 may be the number of vehicles 2 parked in parking lot 101. For example, the number of vehicles 2 included in the monitoring system 1 may be the number of vehicles 2 that are parked in parking lot 101 and registered in the monitoring system 1.

[0021] For example, if there are no vehicles 2 parked in the parking lot 101, the monitoring system 1 will not include vehicles 2 and will only include the infrastructure unit 3. For example, if there are one, two, or four or more vehicles 2 parked in the parking lot 101, the monitoring system 1 will include the number of parked vehicles 2 and the infrastructure unit 3.

[0022] Figure 2 shows an example of the functional configuration of the monitoring system 1 according to the embodiment.

[0023] Vehicle 2 is a vehicle that uses parking lot 101. Vehicle 2 may be any type of mobile vehicle, and may be an EV vehicle such as an electric vehicle driven by a motor, a vehicle such as an automobile driven by an engine (internal combustion engine) as a power source, or a hybrid vehicle driven by both an engine and a motor as power sources. Furthermore, examples of mobile vehicles that can be described as Vehicle 2 include automobiles such as passenger cars, trucks, and motorcycles, but may also include electric bicycles, electric kick scooters, electric wheelchairs, construction machinery, agricultural machinery, ships, trains, airplanes, etc. Furthermore, examples of mobile vehicles that can be described as Vehicle 2 may be configured to move autonomously, or may be configured to move in response to direct and / or remote operation by a user.

[0024] As shown in Figure 2, Vehicle 2 has an on-board unit 21 and a vehicle system 22. The on-board unit 21 is at least one computer mounted in Vehicle 2. The vehicle system 22 includes various parts of Vehicle 2 that are connected to the on-board unit 21 via an on-board network, such as on-board sensors 221 and on-board cameras 222. The on-board sensors 221 include various sensors that detect abnormalities in the vehicle, such as door sensors, tilt sensors, impact sensors, and tire pressure sensors. The on-board cameras 222 include at least one camera that captures the vicinity of the vehicle and blind spots for other vehicles, such as a front camera, rear camera, and drive recorder camera. The vehicle system 22 also includes at least one other computer or memory mounted in Vehicle 2 that is connected to the on-board unit 21 via an on-board network. This other at least one computer or memory acquires and / or holds speed information and shift lever information of Vehicle 2. Here, shift lever information refers to information about the shift position of Vehicle 2, such as "D", "P", and "N". The in-vehicle device 21 may be part of the vehicle system 22 of the vehicle 2.

[0025] The in-vehicle unit 21 has the functions of a communication unit 211, a storage unit 212, a control unit 213, and a display unit 214.

[0026] The communication unit 211 communicates with the server 5. The communication unit 211 also serves as an interface for inputting various types of information from the vehicle system 22. For example, the communication unit 211 inputs sensing information from the on-board sensor 221. For example, the communication unit 211 inputs images from the on-board camera 222. These images may be still images or moving images (also referred to as video or motion pictures). For example, the communication unit 211 inputs vehicle speed information and shift lever information from a computer or memory on the on-board network.

[0027] The memory unit 212 stores various types of information. For example, the memory unit 212 downloads and saves videos taken by the fixed camera 4 and the on-board cameras 222 of nearby vehicles 2 from the server 5.

[0028] The control unit 213 controls the operation of each part of the in-vehicle unit 21. For example, the control unit 213 determines any abnormalities or conditions of the vehicle based on information obtained via the communication unit 211 and notifies the server 5.

[0029] The display unit 214 is a user interface that presents videos downloaded from the server 5 to the user, for example, for confirmation. The user interface exemplified as the display unit 214 may also be capable of receiving and acquiring user input, such as by implementing a touch panel display.

[0030] The fixed camera 4 is responsible for acquiring images used to determine the position of vehicle 2 and identify blind spots within the parking lot 101. The fixed camera 4 is a camera installed in the parking lot 101 so as to be able to capture an overhead view of all parking spaces 103 within the parking lot 101. For example, the fixed camera 4 is mounted on pillars such as poles, walls, roofs, etc., located in the parking lot 101. The fixed camera 4 only needs to be able to capture the area to be monitored within the parking lot 101, and is not limited to a single camera whose shooting range (angle of view) includes the area to be monitored. The fixed camera 4 may also be a combination of multiple cameras, each with a shooting range that includes at least a part of the area to be monitored. Furthermore, the fixed camera 4 may have a variable shooting range, and may be able to capture the area to be monitored by moving its shooting range.

[0031] As shown in Figure 2, the fixed camera 4 has the functions of a communication unit 41 and an imaging unit 42.

[0032] The communication unit 41 communicates with the server 5. The communication unit 41 transmits the video obtained by the imaging unit 42 to the server 5. For example, the communication unit 41 transmits continuously captured video to the server 5 at intervals specified by the server 5 or at predetermined intervals stored in internal memory or elsewhere.

[0033] The camera unit 42 takes pictures of the parking lot 101 according to instructions from the server 5. For example, the camera unit 42 takes pictures continuously and outputs the captured video to the communication unit 41.

[0034] Server 5 is responsible for managing parking conditions, selecting in-vehicle cameras 222 to capture blind spots, and issuing recording instructions to surrounding vehicles 2 in case of abnormalities. Server 5 is a computer installed in the parking lot 101. Server 5 may also be installed outside the parking lot 101, such as in a facility adjacent to the parking lot 101. Furthermore, Server 5 may be a cloud server built on the cloud, or a computer capable of communicating with vehicles 2 and fixed cameras 4 via telecommunication lines such as the internet.

[0035] As shown in Figure 2, the server 5 has the functions of a communication unit 51, a storage unit 52, a control unit 53, an analysis unit 54, and a management unit 55.

[0036] The communication unit 51 communicates with the fixed camera 4. The communication unit 51 also communicates with the vehicle 2.

[0037] The storage unit 52 stores video footage from the vehicle 2 and the fixed camera 4, recorded video data generated based on that footage (see Figures 12 and 13), parking management data, and the like. This parking management data includes a parking management ledger 61 (see Figure 4) and mode setting information 62 (see Figure 5).

[0038] The control unit 53 controls the operation of each part of the server 5. For example, the control unit 53 controls communication, including the exchange of video from the vehicle 2 and the fixed camera 4. For example, the control unit 53 controls recording by the parked vehicle 2.

[0039] The analysis unit 54 analyzes the collected video footage to identify the vehicle's parking position and blind spots.

[0040] The management unit 55 tracks the location of each parked vehicle 2, the time of entry and exit, blind spots, and links this information together.

[0041] The server 5 may be integrated with the fixed camera 4. Alternatively, the server 5 may be implemented by at least one computer mounted on the vehicle 2, such as an in-vehicle unit 21.

[0042] Figure 3 shows an example of the hardware configuration of an information processing device 8 that realizes each device of the monitoring system 1 according to this embodiment. Note that one device of the monitoring system 1 may be realized by one information processing device 8, or by the cooperation of two or more information processing devices 8. Similarly, two or more devices of the monitoring system 1 may be integrated and realized by a single information processing device 8.

[0043] As shown in Figure 3, the information processing device 8 includes a processor 81, a main memory 82, an auxiliary storage device 83, and an interface (I / F) 84. The processor 81, main memory 82, auxiliary storage device 83, and I / F 84 are interconnected by a bus or the like, resulting in a hardware configuration that utilizes a typical computer. Note that each component of the information processing device 8 may be realized by a combination of two or more components. Similarly, two or more components of the information processing device 8 may be integrated and realized by a single component.

[0044] The processor 81 is, for example, at least one CPU (Central Processing Unit). The processor 81 comprehensively controls the operation of the information processing device 8 by executing a program stored in the auxiliary storage device 83, for example, and realizes each of the functions of the information processing device 8.

[0045] For example, the processor 81 of the in-vehicle unit 21 may implement the functions of the in-vehicle unit 21, including the communication unit 211, storage unit 212, control unit 213, and display unit 214 as illustrated in Figure 2. For example, the processor 81 of the fixed camera 4 may implement the functions of the fixed camera 4, including the communication unit 41 and imaging unit 42 as illustrated in Figure 2. For example, the processor 81 of the server 5 may implement the functions of the server 5, including the communication unit 51, storage unit 52, control unit 53, analysis unit 54, and management unit 55 as illustrated in Figure 2.

[0046] In the example shown in Figure 2, only the functions necessary for explaining the main parts of this embodiment are illustrated, but the functions of each device in the monitoring system 1 are not limited to these. Furthermore, some or all of the functions of each device in the monitoring system 1 may be implemented by dedicated hardware circuits.

[0047] Furthermore, each device in the monitoring system 1 may integrate two or more functions to realize a single function. Similarly, each device in the monitoring system 1 may divide a single function to realize two or more functions. Furthermore, in the monitoring system 1, the functions of two or more devices may be integrated to realize at least one function of any of the devices. Similarly, in the monitoring system 1, the functions of one device may be divided to realize two or more functions of two or more devices.

[0048] Here, the processor 81 according to this embodiment is an example of at least one processor in the information processing device 8. Instead of the CPU, or in addition to the CPU, at least one other processor may be used as this at least one processor. Other processors that can be used include various types of processors such as CPUs, GPUs (Graphics Processing Units), and DSPs (Digital Signal Processors), as well as dedicated arithmetic circuits implemented using ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays).

[0049] The main memory 82 is, for example, RAM (Random Access Memory). The main memory 82 temporarily stores data necessary for various processes performed by the processor 81. Here, the main memory 82 in this embodiment is an example of the internal memory of the information processing device 8, and is an example of at least one memory.

[0050] The auxiliary storage device 83 is, for example, ROM (Read Only Memory). The auxiliary storage device 83 stores programs and parameters that realize various processes by the processor 81. Here, the auxiliary storage device 83 in this embodiment is an example of the internal memory of the information processing device 8, and is an example of at least one memory. In addition to or instead of ROM, various storage media and storage devices such as HDD (Hard Disk Drive), SSD (Solid State Drive), and Flash memory can be used as the auxiliary storage device 83 as appropriate.

[0051] For example, the main memory 82 and auxiliary storage 83 of the in-vehicle device 21 may implement a storage unit 212. For example, the main memory 82 and auxiliary storage 83 of the server 5 may implement a storage unit 52.

[0052] I / F84 is an interface to the user and / or external devices. For example, I / F84 may be an output interface for connecting or implementing an output device that outputs audio, images, or video. Suitable output devices include various displays such as liquid crystal displays (LCDs), electro-luminescence (OLED) displays, head-up displays (HUDs), projectors, and speakers. I / F84 may also be an input interface for connecting or implementing an input device that acquires audio, images, video, or user input. Suitable input devices include keyboards, mice, touch panels, cameras, and microphones. Furthermore, I / F84 may be a communication interface for connecting or implementing a communication device that communicates with external devices. Suitable communication devices include wired or wireless communication circuits. Suitable wireless communication circuits include those compatible with various standards such as 3G, 4G, 5G, 6G, Wi-Fi®, Bluetooth®, and infrared communication.

[0053] For example, the I / F84 of the in-vehicle unit 21 may implement a communication unit 211 (communication interface) and a display unit 214 (output interface and / or input interface). For example, the I / F84 of the fixed camera 4 may implement a communication unit 41 (communication interface) and a shooting unit 42 (input interface). For example, the I / F84 of the server 5 may implement a communication unit 51 (communication interface).

[0054] In vehicle 2, the information processing device 8 may be an in-vehicle computer such as an ECU (Electronic Control Unit) or a DCU (Domain Control Unit) such as a CDC (Cockpit Domain Controller) that integrates multiple ECUs, or an OBU (On Board Unit), located inside vehicle 2. Alternatively, the information processing device 8 in vehicle 2 may be an external computer installed near the dashboard of vehicle 2.

[0055] Furthermore, in vehicle 2, the information processing device 8 may send and receive information with other devices installed in vehicle 2 via an in-vehicle network including CAN (Controller Area Network), Ethernet (registered trademark), USB (Universal Serial Bus (registered trademark)), etc., or it may communicate with an information processing device outside of vehicle 2 via a network such as the Internet.

[0056] The following describes an example of the operation of the monitoring system 1 according to the embodiment, with reference to the drawings. Note that the process described below is just one example, and it is possible to change the order of processing, delete some processes, or add other processes.

[0057] First, we will describe the parking management data used in the monitoring process (information processing) according to the embodiment.

[0058] Figure 4 shows an example of the configuration of the parking management ledger 61 according to the embodiment. In the monitoring process according to the embodiment, the server 5 uses the list exemplified as the parking management ledger 61 in Figure 4 to manage information indicating which car is parked where, information indicating which parked cars are watching which cars are parked where, and recorded data. In other words, the parking management ledger 61 is predetermined information relating to a plurality of parking spaces 103 provided in the parking lot 101, and includes information indicating the parking status of the vehicle 2. The parking management ledger 61 also contains information indicating the correspondence between the "watched" parking space 103 in which the vehicle 2 parked is the target of being filmed by the on-board camera 222 of the vehicle 2 parked in another parking space 103, and at least one "watching" parking space 103 in which the vehicle 2 parked in the "watched" parking space 103 is the vehicle 2 that films the vehicle 2 parked in the "watched" parking space 103 with the on-board camera 222. Here, the parking management ledger 61 according to this embodiment is an example of parking management information.

[0059] As shown in Figure 4, the parking management ledger 61 includes an item for the "vehicle being monitored" and an item for the "vehicle monitoring." This is because, in the monitoring process according to the embodiment, a vehicle 2 parked in the parking lot 101 plays two roles: one is monitored by surrounding vehicles 2, and the other monitors surrounding vehicles 2.

[0060] The "Vehicle being monitored" section of the parking management ledger 61 includes the fields "Parking Space," "Vehicle ID," and "Recorded Data." The "Parking Space" field stores an identification code that uniquely identifies parking space 103 in parking lot 101 where the vehicle is parked. The "Vehicle ID" field stores an identification number that uniquely identifies vehicle 2 parked in parking space 103, such as information on the license plate. The "Recorded Data" field stores or links recorded data if recording was made when vehicle 2 malfunctioned.

[0061] The "monitoring vehicle" section of the parking management ledger 61 includes the fields "parking space" and "vehicle ID". The "parking space" field stores the identification number of the parking space 103 of the vehicle 2 that will monitor the parking space 103 listed in the "monitored vehicle" section of that row. For example, if vehicle 2 is parked in "parking space A", this indicates that at least one vehicle 2 parked in at least one parking space 103 from "parking space F" to "parking space H" will be responsible for monitoring vehicle 2 in "parking space A". For example, in the parking management ledger 61 in Figure 4, a priority order for the monitoring vehicles 2 may be defined, such as from left to right. The "vehicle ID" field stores the identification number of the vehicle 2 parked in that parking space 103.

[0062] The designation of each corresponding parking space 103 may be fixed or changed as needed, and changes can be set on this list. Also, the number of monitoring vehicles 2 shown in the parking management ledger 61 in Figure 4 is just an example and can be changed as needed.

[0063] Next, we will describe the monitoring process performed using the parking management ledger 61 mentioned above.

[0064] (Regarding mode switching) The monitoring process according to this embodiment is performed in multiple modes. Specifically, the server 5 switches the mode in which it performs the monitoring process according to the status of the parking lot 101 (hereinafter also referred to as the parking lot status). For example, the server 5 performs the monitoring process in the mode that corresponds to the parking lot status from among the first to third modes which are set in advance. For example, the server 5 changes the items (information) to be managed in the parking management ledger 61 according to the parking lot status.

[0065] Now, let's explain each of the three modes (Modes 1 through 3).

[0066] Furthermore, the monitoring system 1 that executes each of the first to third modes has the same system configuration (hardware configuration). In addition, the initial data held by the server 5 in each mode is the layout information of the parking spaces 103 and the parking management ledger 61.

[0067] (Mode 1) The first mode is a monitoring process in which the blind spots of the on-board cameras 222 of each vehicle 2, captured by the fixed camera 4 of the infrastructure unit 3, are covered by the on-board cameras 222 of other vehicles 2 located around that vehicle 2.

[0068] In Mode 1, the role of fixed camera 4 is to photograph the entire parking lot 101 and identify the blind spots of each vehicle 2's parking space 103 and fixed camera 4. In Mode 1, the role of on-board camera 222 is to photograph a predetermined shooting range when selected and identify the blind spots of fixed camera 4. The number of on-board cameras 222 used in Mode 1 is less than in Modes 2 and 3. In Mode 1, on-board camera 222 selection is performed. Specifically, in Mode 1, on-board camera 222 selection involves checking the parking situation with fixed camera 4, reviewing (reconfiguring) the parking management ledger 61 according to the parking situation, and selecting the on-board camera 222 set in the parking management ledger 61 for each parking space 103. Here, reviewing the parking management ledger 61 includes enabling or disabling (setting use / non-use) the on-board camera 222 that monitors each parking space 103. In Mode 1, the "security performance" is lower than in Modes 2 and 3 due to the smaller number of in-vehicle cameras (222) used. In Mode 1, the "impact of recording data volume and communication traffic" is smaller than in Modes 2 and 3.

[0069] In addition to the effects of the second mode, this first mode, under relatively uncrowded conditions, uses fixed camera 4 to monitor areas that are visible to the camera (areas that can be monitored). In other words, the first mode is efficient because it effectively utilizes fixed camera 4 when parking lot 101 is relatively uncrowded. Furthermore, the first mode can reduce the load on the monitoring system 1 by reducing communication load and saving power.

[0070] (Second mode) The second mode is a monitoring process in which the blind spots of the on-board cameras 222 of each vehicle 2 that cannot be covered by the fixed cameras 4 of the infrastructure unit 3 are covered by the on-board cameras 222 of other vehicles 2 located around that vehicle 2.

[0071] In Mode 2, the role of fixed camera 4 is to photograph the entire parking lot 101 and to identify the parking space 103 for each vehicle 2. In Mode 2, the role of in-vehicle camera 222 is to photograph a predetermined shooting range when selected and to monitor the blind spots of other vehicles' in-vehicle cameras 222. In Mode 2, the number of in-vehicle cameras 222 used is greater than in Mode 1 and less than in Mode 3. In Mode 2, the selection of in-vehicle cameras 222 is performed. Specifically, the selection of in-vehicle cameras 222 in Mode 2 involves selecting an in-vehicle camera 222 that has been pre-configured in the parking management ledger 61 for each parking space 103. In Mode 2, the security performance is higher than in Mode 1 and lower than in Mode 3. In Mode 2, the impact of recording data volume and communication traffic is greater than in Mode 1 and smaller than in Mode 3.

[0072] This second mode is particularly effective when the parking lot 101 is nearly full, based on the settings in the parking management ledger 61. Specifically, because surrounding vehicles 2 can monitor the area around their own vehicle 2 from the vehicle's perspective, it can capture and monitor things like vandalism to the tires, which are difficult to capture with the vehicle's own onboard camera 222.

[0073] (Third mode) The third mode is a monitoring process in which, triggered by a vehicle malfunction, the on-board cameras 222 of all vehicles 2 in the parking lot 101 are turned on.

[0074] In Mode 3, the role of fixed camera 4 is to photograph the entire parking lot 101. In Mode 3, the role of in-vehicle camera 222 is to photograph a predetermined shooting range and monitor the entire parking lot 101, including the blind spots of other vehicles' in-vehicle cameras 222. In Mode 3, the number of in-vehicle cameras 222 used is all parked vehicles 2, which is more than in Modes 2 and 3. In Mode 3, no selection of in-vehicle cameras 222 is performed. Specifically, in Mode 3, all in-vehicle cameras 222 are subject to activation, i.e., recording instructions are given. In Mode 3, the security performance is higher than in Modes 2 and 3 due to the larger number of in-vehicle cameras 222 used. In Mode 3, the impact of recording data volume and communication traffic is greater than in Modes 2 and 3.

[0075] While this third mode offers high security, it places a heavy processing load on server 5. Furthermore, because it includes recordings of the perpetrator's escape route and in-vehicle cameras 222 unrelated to the vehicle itself, it generates a large amount of unnecessary data. However, in addition to the effects of the second mode, the third mode, with the participation of all in-vehicle cameras 222 in the parking lot 101, can potentially allow for the identification of the perpetrator's escape route.

[0076] Figure 5 shows an example of the configuration of mode setting information 62 according to the embodiment. As shown in Figure 5, the first to third modes and the parking situation are predetermined as mode setting information 62. This allows the monitoring processing mode to be switched between the first to third modes depending on the usage status (parking situation) and time of day (parking situation) of the parking lot 101.

[0077] The mode setting information 62 is configured, for example, as a matrix of the congestion status (parking conditions) of parking lot 101 and the time of day. Note that the contents of the matrix are just an example and can be changed as appropriate according to the characteristics of each individual parking lot 101.

[0078] For example, suppose that parking lot 101 is relatively empty during the daytime because many cars are out. For example, suppose that parking lot 101 is full at night because everyone has gone home. For example, suppose that parking lot 101 is occasionally vandalized in the evening or late at night. As shown in Figure 5, the mode setting information 62 for such parking lot 101 is set to mode 1 during the morning hours when parking lot 101 is likely to be empty. Mode setting information 62 is set to mode 2 when it is crowded or full. Mode setting information 62 is also set to mode 2 in the evening when security needs to be strengthened or when it is a time when incidents are likely to occur, even if it is empty. Mode setting information 62 is also set to mode 3 only during the late-night hours when security needs to be strengthened.

[0079] By using the mode setting information 62 configured in this way, it is possible to flexibly respond according to the characteristics of each parking lot 101, such as using the fixed camera 4 as the primary camera when the risk of crime is low, and making extensive use of the vehicle-mounted camera 222 when the risk of crime is high.

[0080] Here, we will explain the information processing related to switching modes for executing monitoring processing based on mode setting information 62. Figure 6 is a sequence diagram showing an example of the flow of monitoring processing related to mode setting executed by the monitoring system 1 according to the embodiment.

[0081] First, in the monitoring process, the fixed camera 4 performs continuous recording to capture images at all times and obtain video footage, and sequentially transmits the video footage obtained from the continuous recording to the server 5 (S101). In the server 5, the control unit 53 acquires the video footage from the fixed camera 4 and stores it in the storage unit 52 as the recording result (S102).

[0082] The management unit 55 then obtains the current time from the control unit 53 (S103), the vehicle ID and parking space information read from the storage unit 52 (S104), and the mode setting information 62 read from the storage unit 52 (S105). The management unit 55 also determines the mode of the monitoring process based on the acquired information and outputs the result of the mode determination to the control unit 53 (S106).

[0083] Thus, in the monitoring process according to this embodiment, the security performance can be adjusted by changing the initial data or the implementation procedure, using the same hardware configuration. Therefore, according to the monitoring process according to this embodiment, it is possible to build a flexible system according to conditions such as communication traffic, parking lot congestion, parking lot layout, parking lot size, and time of day.

[0084] The following explains the monitoring process after the mode has been determined.

[0085] (Regarding monitoring in mode 2) First, the monitoring process in the second mode will be explained. Figure 7 is a flowchart showing an example of the flow of the monitoring process in the second mode, which is executed by the server 5 according to the embodiment. Here, the flow in Figure 7 is assumed to start with the parking management ledger 61 in its initial state. Note that the parking management ledger 61 being in its initial state means that no values ​​have been stored in the "Vehicle ID" and "Recorded Data" fields of the "Vehicle being monitored" and the "Vehicle ID" field of the "Monitoring side".

[0086] First, server 5 receives a new entry or exit signal (S201), recognizes the entry or exit of vehicle 2 based on the received entry or exit signal, and identifies the target parking space 103 (S202).

[0087] (Regarding processing upon receiving goods into storage) Figure 8 is a sequence diagram showing an example of the flow of monitoring processing when a vehicle enters the parking lot, as performed by the monitoring system 1 according to the embodiment. It is assumed that, for example, a predetermined layout map of parking spaces 103 is stored in the storage unit 52 of the server 5. First, in the monitoring process, the fixed camera 4 performs continuous recording to obtain images by constantly taking pictures, and sequentially transmits the images obtained from the continuous recording to the server 5 (S401). In the server 5, the control unit 53 acquires the images from the fixed camera 4 and stores them in the storage unit 52 as recording results (S402). In this way, the parking lot 101 is constantly recorded by the fixed camera 4, and while all parking spaces 103 of the parking lot 101 are captured from an overhead view, a vehicle 2 enters one of the parking spaces 103 of the parking lot 101. The onboard unit 21 of the parked vehicle 2 acquires speed information of "0 (zero)" and shift lever information of "P" from the vehicle system 22 (S403), and accordingly transmits a parking completion notification, vehicle identification information, and parking completion time to the server 5 (S404). Then, the control unit 53 in the server 5 stores the identification information and parking completion time from the parked vehicle 2 in the storage unit 52 (S405). Subsequently, the analysis unit 54 acquires an image of the parked vehicle 2 and a layout map from the storage unit 52 (S406). The image of vehicle 2 acquired here includes an image of the parked vehicle 2 at the time of parking completion and an image from a certain time before the parking completion time, extracted from the video data of the continuously recording fixed camera 4 in the storage unit 52. The analysis unit 54 also identifies the parking space 103 where the vehicle was parked at the time of parking completion based on the difference between the images before and after parking completion and the layout map (S407). Then, the management unit 55 obtains the identified parking space 103, the time of parking completion, and identification information from the analysis unit 54 (S408), and links the identification information of vehicle 2 with the parking space 103 in which it is parked, generating entry information (S409).

[0088] (Regarding processing upon delivery) Figure 9 is a sequence diagram showing an example of the flow of monitoring processing when a vehicle leaves the parking lot, as performed by the monitoring system 1 according to this embodiment. It is assumed that, for example, a predetermined layout map of parking spaces 103 is stored in the storage unit 52 of the server 5. First, in the monitoring process, the fixed camera 4 performs continuous recording to obtain images by constantly taking pictures, and sequentially transmits the images obtained from the continuous recording to the server 5 (S501). In the server 5, the control unit 53 acquires the images from the fixed camera 4 and stores them in the storage unit 52 as recording results (S502). In this way, the parking lot 101 is constantly recorded by the fixed camera 4, and with all parking spaces 103 of the parking lot 101 captured from an overhead view, vehicle 2 leaves the parking lot 101 into one of the parking spaces 103. The onboard unit 21 of the vehicle 2 that has left the parking lot acquires engine "ON" information and shift lever information indicating the shift position "D" from the vehicle system 22 (S503), and accordingly transmits a parking departure notification, vehicle identification information, and parking completion time to the server 5 (S504). Then, the control unit 53 in the server 5 stores the identification information and parking departure time from the vehicle 2 that has left the parking lot in the storage unit 52 (S505). Subsequently, the analysis unit 54 acquires an image of the vehicle 2 that has left the parking lot and a layout map from the storage unit 52 (S506). The image of the vehicle 2 acquired here includes an image of the vehicle 2 at the time of departure and an image taken after a certain period of time has elapsed from the time of departure (for example, 10 seconds after receiving the parking departure notification), extracted from the video data of the fixed camera 4 that is continuously recording in the storage unit 52. Furthermore, the analysis unit 54 identifies the parking space 103 that became vacant at the time of departure based on the difference between images before and after departure and the layout map (S507). The management unit 55 then obtains confirmation that the vehicle has departed from the identified parking space 103 (vacancy confirmation), the departure time, and identification information from the analysis unit 54 (S508), and links the identification information of vehicle 2 to the vacant parking space 103 to generate departure information (S509).

[0089] (Regarding ledger update processing) Returning to Figure 7, let's continue the explanation. After identifying the target parking space 103 (S202), that is, after the entry and exit information is generated as described above (S409 or S509), the management unit 55 on the server 5 executes the ledger update process (S203).

[0090] First, the management unit 55 updates the "Vehicle ID" item for the "monitored vehicle" in the parking management ledger 61 based on the entry or exit information (S301). Then, it updates the "Vehicle ID" item for the "monitoring vehicle" in the parking management ledger 61 (S302). Figure 10 is a diagram illustrating the updating of the parking management ledger 61 in the second mode monitoring process according to the embodiment. Figure 10 illustrates the case when vehicle 2 with "Vehicle ID: xxxx" has finished parking in "Parking Space H". In this case, the management unit 55 updates the "Vehicle ID" item corresponding to "Parking Space H" for the "monitored vehicle" to the value "xxxx" in relation to the "Parking Space" item specified in the parking management ledger 61, and also updates the "Vehicle ID" items for each "Parking Space H" set on the "monitoring vehicle" to the value "xxxx". Furthermore, in the state of the parking management ledger 61 shown in Figure 10, if vehicle 2 with "Vehicle ID: xxxx" exits from "Parking Space H", the management unit 55 similarly clears (updates) the value of "xxxx" stored in the "Vehicle ID" field corresponding to "Parking Space H" on the "monitored side," linked to the "Parking Space" item specified in the parking management ledger 61, and also clears (updates) the value of "xxxx" stored in each "Vehicle ID" field where "Parking Space H" on the "monitoring side" is set. Then, if it was an entry (S303: Yes), the management unit 55 completes the update. On the other hand, if it was an exit (S303: No), the management unit 55 clears (updates) the "Recorded Data" field on the "monitoring side" in the parking management ledger 61 (S304), and then completes the update.

[0091] After the ledger update process (S203), the control unit 53 determines whether it has received an entry or exit signal for another vehicle (S204). If it has received an entry or exit signal for another vehicle (S204: Yes), the flow in Figure 7 returns to the process in S202. For example, if there is a new entry or exit of a vehicle, in the processes of S202 to S203, the server 5 can recognize the entry or exit of the new vehicle 2 in the same manner as described above, and update the parking management ledger 61. For example, even if a vehicle 2 that was parked in one of the parking spaces 103 moves to another parking space 103, a new exit signal and entry signal are received, so the movement (entry or exit) of vehicle 2 can be recognized in the same manner, and the parking management ledger 61 can be updated.

[0092] Thus, the monitoring process in the second mode according to this embodiment can check the status of the parking lot 101 and update the parking management ledger 61 according to the status. This allows the server 5 to respond to dynamic changes such as vehicles entering, leaving, and moving within the parking lot 101.

[0093] (Regarding responses when an anomaly is detected) The control unit 53 determines whether it has received an abnormality detection signal from vehicle 2 (S205). If an abnormality detection signal is received (S205: Yes), the control unit 53 refers to the parking management ledger 61 (S206), instructs the target vehicle 2 on the list to record (S207), and stores the first recording data from the vehicle 2 that was instructed to record, and the second recording data from the fixed camera 4, in the storage unit 52 (S208). On the other hand, if no abnormality detection signal is received (S205: No), the control unit 53 determines that there is no abnormality (S209) and checks the number of parked vehicles in the parking lot (S210). If there is one or more parked vehicles in the parking lot, that is, if vehicle 2 is parked in the parking lot (S210: Yes), the flow in Figure 7 returns to the process in S202. Therefore, the server 5 checks the status of the parking lot 101, such as vehicle 2 entering, leaving, and moving, and continues to update the parking management ledger 61 according to the status. On the other hand, if the number of parked vehicles in the parking area is 0, that is, if there is no vehicle 2 parked in the parking area (S210: No), the process in Figure 7 returns to the process in S201. At this time, the parking management ledger 61 returns to its initial state. Subsequently, if a new entry signal is received, the process described above is executed again.

[0094] Figure 11 is a diagram illustrating the generation of recorded data when an anomaly is detected in the monitoring process according to the embodiment. For example, suppose the parking management ledger 61 is in the state illustrated in Figure 10. Figure 10 illustrates the parking management ledger 61 in a parking lot 101 where four vehicles 2 are parked in "parking space D", "parking space E", "parking space H", and "parking space J". For example, referring to the parking management ledger 61, from the information of the "monitored side", vehicle 2 with "vehicle ID: eeee" is parked in "parking space E", and from the information of the "monitoring side", two vehicles are monitoring this vehicle 2: vehicle 2 with "vehicle ID: xxxx" in "parking space H" and vehicle 2 with "vehicle ID: jjjj" in "parking space J". In this state, if vehicle 2 with "vehicle ID: eeee" emits any abnormal signal, server 5 instructs two vehicles 2 with "vehicle ID: xxxx" and "vehicle ID: jjjj" to start recording, based on the "monitoring side" information in the parking management ledger 61.

[0095] The server 5 then generates first recording data from the recording results obtained by the on-board cameras 222 of the two vehicles 2, "Vehicle ID:xxxx" and "Vehicle ID:jjjj". Figure 12 is a diagram showing an example of the structure of the first recording data (video data) based on the on-board cameras 222 of the surrounding vehicles 2, which is generated when an anomaly is detected in the monitoring process according to the embodiment. As shown in Figure 12, the first recording data includes a "video" item in which the video file itself is stored, "start time" and "end time" items in which the start time and end time of the recording are stored, respectively, and "parking space" and "vehicle ID" items for the "monitored side" and the "monitoring side," respectively. This first recording data is generated for each of the at least one vehicle 2 set as the "monitoring side".

[0096] Furthermore, the server 5 generates data for a second recording file extracted from the recording data of the fixed camera 4, synchronized with the start and end times of the first recording data. Figure 13 is a diagram showing an example of the configuration of the second recording data (video data) based on the fixed camera 4, which is generated when an anomaly is detected in the monitoring process according to the embodiment. As shown in Figure 13, the second recording data includes a "video" item in which the video file itself is stored, "start time" and "end time" items in which the start and end times of the recording are stored, respectively, and "parking space" and "vehicle ID" items for the "surveilled party".

[0097] Here, the flow of monitoring processing when an anomaly is detected in the monitoring system 1 will be explained. Figure 14 is a sequence diagram showing an example of the flow of monitoring processing when an anomaly is detected, as executed by the monitoring system 1 according to the embodiment. When the onboard sensor 221 of the vehicle system 22 detects an anomaly in vehicle 2 (the vehicle itself), it outputs an anomaly detection signal to the onboard unit 21 (S601). The onboard unit 21 notifies the server 5 of the anomaly (anomaly detection signal) in response to the anomaly detection signal from the vehicle system 22 (S602). The onboard unit 21 also instructs the onboard camera 222 of vehicle 2 (the vehicle itself) to start recording (S603). When the control unit 53 in the server 5 receives the anomaly detection signal from vehicle 2 (the vehicle itself), it refers to the parking management ledger 61 (S604) and instructs the vehicle 2 (other vehicle) listed as the "monitoring side" to start recording (recording request) (S605).

[0098] Vehicle 2 (another vehicle) that receives a recording request starts recording with its onboard camera 222, records for a predetermined time or for a time specified by server 5, and uploads the recorded data to server 5 after the recording is finished (S606). At this time, vehicle 2 (another vehicle) that receives the recording request adds the start time, end time, and vehicle ID information to the recorded data. The control unit 53 in server 5 acquires this as first recorded data and stores it in the storage unit 52.

[0099] Furthermore, the analysis unit 54 acquires information such as the ID of the fixed camera 4 in the parking lot 101 that is continuously recording, the vehicle ID of the "vehicle being monitored", and the start and end times of the recording of the first recording data (S607). The analysis unit 54 also extracts video data from the continuously recording data of the fixed camera 4 at a time synchronized with the first recording data, and stores the extracted recording data in the storage unit 52 (S608). The control unit 53 then adds the start and end times of the recording and information of the "vehicle being monitored" vehicle 2 to the extracted recording data, and stores it in the storage unit 52 as the second recording data. The control unit 53 also notifies the vehicle 2 (other vehicle) that responded to the recording request that saving is complete (S609). At this time, vehicle 2 (other vehicle) can delete the recording data that was sent in S606. Subsequently, the parking management ledger 61 stored in the storage unit 52 is updated, and the generated first and second recording data are linked (S610).

[0100] Furthermore, when a recording data request is made from vehicle 2 (the user's vehicle) that has detected an abnormality, along with its vehicle ID (S611), the control unit 53 in the server 5 refers to the parking management ledger 61 stored in the storage unit 52 based on the vehicle ID and retrieves the first and second recording data from the storage unit 52 (S612). The control unit 53 also transmits the first and second recording data from the storage unit 52 to vehicle 2 (the user's vehicle) that requested the recording data (S613). Vehicle 2 stores the received recording data in the storage unit 212. At this time, vehicle 2 can play back the recording data from the server 5. For example, the display unit 214 in the in-vehicle unit 21 plays back the received recording data according to user operation. This allows the user to check the recording data obtained from other cameras in the vehicle's in-vehicle camera 222.

[0101] Thus, in the second mode of monitoring according to the embodiment, when the vehicle is damaged while parked, the video footage obtained from the fixed camera 4 as part of the parking lot infrastructure, the on-board cameras 222 of other vehicles in the vicinity, and the on-board camera 222 of the vehicle itself is acquired as evidence. Therefore, according to the second mode of monitoring according to the embodiment, the security of the vehicle while parked can be improved.

[0102] (Regarding monitoring processing in Mode 1) Next, we will explain the monitoring process in the first mode.

[0103] Furthermore, the storage unit 52 of server 5 is pre-defined and stores information regarding the three-dimensional spatial arrangement of the fixed camera 4 with respect to each parking space 103. This information may be geometric information indicating the arrangement, or it may be information indicating, for each parking space 103, which parking space 103 is determined to have another vehicle 2 parked in front of it from the camera view of the fixed camera 4, given that a vehicle 2 is present. This information may also be integrated with the layout information.

[0104] Figure 15 is a flowchart showing an example of the ledger update process in the first mode monitoring process executed by the server 5 according to this embodiment. Here, we will mainly explain the differences from the ledger update process (S203) in Figure 7, and redundant explanations will be omitted as appropriate.

[0105] If the vehicle is parked in the parking lot (S303: Yes), the management unit 55 determines, using the camera view of the fixed camera 4, whether there are other vehicles parked in front of the parked vehicle 2 in each parking space 103 where the vehicle 2 is parked (S701). Specifically, the management unit 55 determines, using the parking management ledger 61 updated in processing S301 to S302 and the information on the placement relationship of the fixed camera 4 to each parking space 103, whether there are vehicles 2 in the parking space 103 in front of the parking space 103 where the vehicle 2 is parked (on the fixed camera 4 side). For example, in the placement relationship shown in Figure 1, for each vehicle 2 in "parking space B", "parking space H", and "parking space J", the camera view of the fixed camera 4, which is installed with a field of view looking down on each parking space 103 from above, determines that there are no other vehicles 2 parked in front of them. For example, when vehicle 2 enters parking space G, the fixed camera 4 determines, based on its view of vehicle 2 in parking space G, that another vehicle 2 is parked in the nearby parking space H. The management unit 55 then executes process S701 for each parking space 103 in which vehicle 2 is parked.

[0106] If the fixed camera 4 determines that another vehicle 2 is parked in front of the vehicle (S701: Yes), the management unit 55 retains the current parking management ledger 61 (S702). The process then proceeds to S704 as shown in Figure 15.

[0107] On the other hand, if the fixed camera 4 determines that there are no other vehicles 2 parked in front of the vehicle (S701: No), the management unit 55 updates the "parking space" item under "monitoring side" in the parking management ledger 61 (S703). After that, the process shown in Figure 15 proceeds to S704.

[0108] Figure 16 shows an example of the configuration of the parking management ledger 61 in the first mode monitoring process according to the embodiment. In the first mode monitoring process, the parking management ledger 61 is configured to be further updatable for the "parking space" item of the "monitoring side".

[0109] In the first mode of monitoring, during processes S201-S202 and S301-S302, the parts of the parking management ledger 61 shown in Figure 16, indicated by dot hatching, are updated. Specifically, during these processes, the management unit 55 updates the "Vehicle ID" and "Recorded Data" items for the "Monitored Side" and the "Vehicle ID" item for the "Monitoring Side" in the parking management ledger 61.

[0110] Furthermore, in the monitoring process of the first mode, during processing S701 to S705, the portion of the parking management ledger 61 shown in Figure 16 with diagonal hatching is further updated. Specifically, the management unit 55 further updates the "parking space" item under "monitoring side" in the parking management ledger 61. This review (update) of the "parking space" item under "monitoring side" means that the number of on-board cameras 222 under "monitoring side" changes.

[0111] Figure 17 is a diagram illustrating the updating of the parking management ledger 61 in the first mode monitoring process according to the embodiment. Figure 17 illustrates the parking management ledger 61 when vehicle 2 with "vehicle ID: xxxx" is parked in "parking space H" in the first mode monitoring process. In this state, the parking space 103 in front of vehicle 2 in "parking space H" as seen from the camera view of the fixed camera 4 is, for example, "parking space I". On the other hand, in the parking management ledger 61, vehicle 2 is not parked in "parking space I". In this case, the management unit 55 performs a review (update) to remove "parking space D" and "parking space C", which are closer to "parking space I" and shown by diagonal lines in Figure 17, from the list of candidate parking spaces on the "monitoring side". As a result, vehicle 2 with "vehicle ID: xxxx" parked in "parking space H" is monitored by the fixed camera 4 and two vehicles 2 in "parking space A" and "parking space B", which are in the blind spot of the fixed camera 4. Furthermore, the information indicating the update location for the "parking space" item under "monitoring side" in the parking management ledger 61 is pre-specified based on, for example, the shape characteristics of the parking lot 101 and stored in the memory unit 52 or the like. As mentioned above, if there is a parked vehicle in "parking space I", the parking management ledger 61 is not updated.

[0112] Thus, in the first mode of monitoring, the "parking space" item (information) of the "monitoring side" is masked in the parking management ledger 61 for the portion that can be covered by the fixed camera 4. Here, masking the information may mean making the information unselectable, or excluding it if it is selected. This reduces the number of in-vehicle cameras 222 on the "monitoring side". In other words, in the first mode of monitoring, the fixed camera 4 monitors the areas that it can monitor, thereby reducing the number of in-vehicle cameras 222 that record (monitoring side).

[0113] After processing S702 or S703, the management unit 55 determines whether all parked vehicles 2 listed in the parking management ledger 61 have been verified (S704). If not all of the target vehicles 2 have been verified (S704: No), the flow in Figure 15 returns to processing S701. On the other hand, if all of the target vehicles 2 have been verified (S704: Yes), the flow in Figure 15 ends and proceeds to processing S204 in the flow in Figure 7.

[0114] If the vehicle is exiting the parking lot (S303: No), the management department 55, if it had updated the "parking space" item under "monitoring side" in the parking management ledger 61 when the vehicle entered the parking lot, restores it to its original state (S705). After that, the process shown in Figure 15 proceeds to S304.

[0115] Thus, the first mode of monitoring according to the embodiment reduces the number of on-board cameras 222 on the "watching side" by having the fixed camera 4 take on the monitoring duties, thereby reducing the camera load on the vehicle 2 while maintaining security. Specifically, in the first mode of monitoring according to the embodiment, when the parking entry information of vehicle 2 is updated on the parking management ledger 61, the fixed camera 4 checks for the presence of other vehicles 2 in the parking space 103 in front of vehicle 2, and if there are no other vehicles 2, it entrusts the monitoring of that location to the fixed camera 4 and removes some of the "watching side" parking spaces that were set in the parking management ledger 61 from the list.

[0116] (Regarding monitoring processing in mode 3) Next, we will explain the monitoring process in the third mode.

[0117] In the third mode of monitoring, when an abnormality occurs, all surrounding vehicles 2 parked in the parking lot 101 record. This can be expressed as equivalent to all vehicles becoming "monitoring vehicles" with respect to each "parking space" that was previously "monitored" in the parking management ledger 61. In other words, in the third mode of monitoring, it is sufficient for the information of the "monitored" vehicles to be updated in the parking management ledger 61, and management of items for the "monitoring" vehicles becomes unnecessary; that is, a list related to the "monitoring" vehicles is unnecessary. Figure 18 is a diagram showing an example of the configuration of the parking management ledger 61 in the third mode of monitoring according to the embodiment. As shown by the diagonal hatching in Figure 18, in the third mode of monitoring, the list of "monitored" vehicles is the target of updates, while the list of "monitoring" vehicles is not used.

[0118] Figure 19 is a flowchart showing an example of the ledger update process in the third mode monitoring process executed by the server 5 according to the embodiment. Here, we will mainly explain the differences from the ledger update process (S203) in Figure 7, and will omit redundant explanations as appropriate. The management unit 55 updates the "Vehicle ID" item of the "Monitored Side" in the parking management ledger 61 based on the entry or exit information (S301). If it was an entry (S303: Yes), the update is completed. On the other hand, if it was an exit (S303: No), the management unit 55 clears (updates) the "Recorded Data" item of the "Monitoring Side" in the parking management ledger 61 (S304), and then the update is completed. After that, the flow in Figure 19 ends, and the process proceeds to S204 in the flow in Figure 7.

[0119] Figure 20 is a diagram illustrating the update of the parking management ledger 61 in the third mode monitoring process according to the embodiment. Figure 20 shows an example of the parking management ledger 61 when vehicle 2 with "vehicle ID: xxxx" is parked in "parking space H" in the third mode monitoring process. In this case, the server 5 updates the "parking space H" item in the "monitored side" of the parking management ledger 61 to the value of "vehicle ID: xxxx" in relation to the parking space specified. On the other hand, unlike the second mode monitoring process, the "parking space H" field in the "monitoring side" is not used and is not updated. This is because, as described above, when an anomaly is detected in the third mode monitoring process, all parked vehicles 2 are both "monitored side" and "monitoring side".

[0120] (Regarding responses when an anomaly is detected) Here, we will explain an example of operation when an anomaly is detected in the third mode of monitoring. Here, we will mainly explain the differences from the example of operation when an anomaly is detected in the second mode of monitoring, as shown in Figure 14, and redundant explanations will be omitted as appropriate. When the control unit 53 in server 5 receives an abnormality detection signal from vehicle 2 (own vehicle), it refers to the parking management ledger 61 (S604) and issues a recording instruction (recording request) to all other vehicles 2 (other vehicles) listed as "the ones being monitored" (S605). Subsequently, recording is performed on all vehicles 2 (other vehicles) parked in the parking lot 101, and the recording data from all of these vehicles 2 (other vehicles) is stored in the storage unit 52 in server 5 as first recording data from each vehicle 2 (S606).

[0121] Thus, in the third mode of monitoring processing according to this embodiment, the parking management ledger 61 is configured such that it is sufficient if the information of the "monitoring party" is updated, and the information of the "monitoring party" is not used. Furthermore, in the third mode of monitoring processing, when an abnormality occurs, recording instructions (recording requests) are issued to all surrounding vehicles 2 parked in the parking lot 101, as if all of them were vehicles 2 parked in the "parking space" of the "monitoring party". This makes it possible to increase the number of in-vehicle cameras 222 that obtain the first recorded data as evidence video, thereby further enhancing security during parking.

[0122] Generally, when it comes to parking lots, there are users (passengers) who park their vehicles and users (managers, operators) who operate the parking lots. For example, parking lot users have a demand for more secure parking spaces to reduce risks to their vehicles, such as hit-and-runs, car break-ins, and vandalism like door dings. On the other hand, parking lot managers have a demand to keep installation and maintenance costs down and operate the parking lots at a low cost.

[0123] In parking lots, it is common practice to install fixed security cameras at multiple locations within the parking area to enhance security. For example, increasing the number of security cameras in a parking lot can reduce blind spots in on-board cameras that cannot be covered by security cameras alone. However, the larger the parking lot, the more security cameras are needed to cover the area, increasing the burden on the administrator in terms of installation and maintenance costs. Furthermore, spending a large amount of money solely on equipment for parking lot security measures will ultimately pass those costs on to users, making widespread adoption difficult.

[0124] Furthermore, from the perspective of enhancing security, it is desirable to position cameras at eye level to facilitate the identification of information and characteristics of perpetrators, such as vehicle license plates. On the other hand, fixed security cameras installed in parking lots are generally mounted on poles or similar structures to monitor the parking area, including parked vehicles, from above in order to broaden their coverage area. However, surveillance using camera footage taken from above makes it difficult to identify information and characteristics of perpetrators, such as vehicle license plates, and the surveillance area for one's own vehicle is limited due to the fixed direction of monitoring.

[0125] Furthermore, even if surveillance is carried out using cameras in the vehicles in addition to fixed security cameras in the parking lot, there are problems such as the perpetrator being in the blind spot of the fixed camera, or the difficulty in capturing the perpetrator's characteristics with the vehicle's camera in cases of vandalism such as vandalism to tires, or the possibility of blind spots in the vehicle's onboard camera that are not covered by the security cameras.

[0126] Furthermore, in a method where recording is triggered by the detection of damage to the vehicle using onboard sensors, the recording may start after the fact, potentially leading to the loss of information about the perpetrator.

[0127] Furthermore, while using onboard cameras installed in other vehicles in the vicinity of one's own vehicle can reduce the number of security cameras that need to be installed, it also presents the problem of high communication and processing costs associated with dynamically deciding which onboard camera of another vehicle in the vicinity to use for monitoring one's own vehicle.

[0128] In this context, the monitoring system 1 according to this embodiment is configured to utilize not only a fixed camera 4 and the vehicle's own onboard camera 222, but also the onboard cameras 222 of other vehicles 2 parked in the same parking lot 101, which are selected based on the parking management ledger 61, as monitoring cameras.

[0129] This configuration minimizes the number of fixed security cameras installed, providing an overall overview of the parking area and identifying blind spots for vehicles not covered by the fixed cameras. In addition, the vehicle-mounted cameras serve to monitor the area around the vehicle they are mounted on, as well as to monitor blind spots of other vehicles not covered by the fixed security cameras. Therefore, this configuration reduces blind spots in the vehicle's own camera that are not covered by the security cameras, thereby improving the security of the parking lot. Furthermore, since the system is configured to select which vehicle-mounted cameras to use for monitoring the parked vehicle based on the parking management ledger 61, the communication and processing costs required to determine which vehicle-mounted cameras to use for monitoring the parked vehicle can be reduced.

[0130] More specifically, with the above configuration, the extent of damage to one's own vehicle can be determined from the footage of multiple surrounding vehicles. Furthermore, by utilizing in-vehicle cameras, the footage is captured from a human perspective, making it easier to capture license plates and the characteristics of perpetrators in cases of vandalism, thus improving security compared to overhead footage from fixed security cameras. In addition, the area in which one's own vehicle can benefit from surveillance can be expanded from the surveillance area of ​​fixed security cameras to the surveillance area of ​​surrounding parked vehicles. Moreover, because surrounding parked vehicles participate in the surveillance, the number of fixed security cameras that need to be installed can be minimized.

[0131] In other words, with the above configuration, parking lot managers can build a highly secure and low-cost parking system while keeping the number of fixed security cameras to a minimum. Furthermore, with the above configuration, owners of parked vehicles can receive surveillance from multiple angles, not only from fixed security cameras and cameras on their own vehicles, but also from cameras on other vehicles in the vicinity.

[0132] In the monitoring system 1 according to the above embodiment, the management unit 55 may identify parking spaces 103 with high security based on the parking management ledger 61 and recommend the identified parking spaces 103 to vehicles 2 entering or having entered the parking lot 101. For example, the management unit 55 may refer to the parking management ledger 61 and, if there is a parking space 103 where a "monitoring" vehicle 2 is present and no "monitored" vehicle 2 is present, it may be configured to output information recommending the use of that "monitored" parking space 103 to the next vehicle 2 to enter the parking lot. In other words, the management unit 55 may refer to the parking management ledger 61, extract parking spaces 103 where a "monitoring" vehicle 2 is present from among the empty "monitored" parking spaces 103, and be configured to recommend the use of that parking space 103 to the next vehicle 2 to enter the parking lot. With this configuration, the parking side can also enhance security.

[0133] It should be noted that multiple parking spaces 103 may be selected as recommended. In such cases, for example, the selected parking spaces 103 may be recommended in order of the number of "monitoring" vehicles 2.

[0134] Furthermore, the parking space 103 recommended may be one of the "monitoring" parking spaces 103 that contain the vehicle 2 that is being "monitored." In this case, even if vehicle 2 that was parked in the "monitoring" parking space 103 has moved or left the parking lot and is no longer there, the overall security of the parking lot can still be maintained and improved. In this case, the parking spaces 103 selected in descending order of the number of "monitoring" parking spaces 103 containing empty vehicles may also be recommended.

[0135] The estimation of the extracted parking spaces 103 is, for example, realized by the display unit 214 of the in-vehicle unit 21 in vehicle 2, but is not limited to this. The information recommending the use of the extracted parking spaces 103 may be a user-facing terminal such as a smartphone registered in vehicle 2. Furthermore, just as the parking lot 101 is equipped with indicator lights to show whether a space is vacant or not, the parking lot 101 may also be equipped with indicator lights to show the parking spaces 103 recommended based on the parking management ledger 61 from a security standpoint.

[0136] Furthermore, the monitoring system 1 according to the above embodiment can be applied to even larger parking lots by expanding the area corresponding to the shooting range of the fixed camera 4 and the number of on-board cameras capable of monitoring the area around the vehicle, as illustrated in Figure 1, for example, by defining an area as a single monitoring unit. This is because, even if the parking lot is full, there is an upper limit to the number of on-board cameras capable of monitoring the area around the vehicle, so a certain number of parking spaces 103 are treated as a single unit, and mutual monitoring is achieved for each unit. In this case, for example, a fixed camera 4 is provided for each unit. Similarly, a parking management ledger 61 is provided for each unit, for example. On the other hand, the server 5 may be provided for each unit, or for multiple units.

[0137] In the above-described embodiment of the monitoring system 1, the in-vehicle unit 21 may have at least one of the in-vehicle sensor 221 and the in-vehicle camera 222. For example, the in-vehicle unit 21 having the in-vehicle sensor 221 and the in-vehicle camera 222 may be realized by an external computer that can be installed near the dashboard of the OBU or vehicle 2. With this configuration, the in-vehicle unit 21 can be retrofitted or its installation can be simplified, for example, by mounting it on a parked vehicle in a parking lot for commercial vehicles to configure the monitoring system 1.

[0138] In the monitoring system 1 according to the above embodiment, the parking management ledger 61 may be configured as a common file between the first to third modes, or as a separate file. For example, if it is configured as a common file, this can be achieved by switching which items are subject to update and which are not.

[0139] In the above embodiment, the example given was a case where the first to third modes are switched, but this is not the only example. The monitoring system 1 may be configured to be able to execute only one of the first to third modes. In other words, the monitoring system 1 may operate in a single mode, and switching is not a necessary configuration. Furthermore, the monitoring system 1 may be configured to switch between any two of the first to third modes.

[0140] In each of the embodiments described above, "is A" means at least one of "is A" or "is not A". In other words, in each of the embodiments described above, the determination of "is A" may be achieved by determining whether it is A, by determining whether it is not A, or by determining both of these.

[0141] The programs executed by each device of the monitoring system 1 in each of the embodiments described above may be provided as files in an installable or executable format, recorded on a computer-readable recording medium (Computer Program Product) such as a CD-ROM, FD, CD-R, or DVD.

[0142] Furthermore, the programs executed by each device of the monitoring system 1 in each of the above embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the programs executed by each device of the monitoring system 1 in each of the above embodiments may be provided or distributed via a network such as the Internet.

[0143] Furthermore, the programs executed by each device of the monitoring system 1 in each of the above embodiments may be pre-installed and provided in ROM or the like.

[0144] According to at least one embodiment described above, the security of a parking lot can be improved.

[0145] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0146] 1. Monitoring System 101 Parking 103 Parking Spaces 2 vehicles 2-1 Vehicles 2-2 Vehicles 2-3 Vehicles 21 Onboard equipment 211 Communications Department 212 Storage section 213 Control Unit 214 Display section 22 Vehicle Systems 221 In-vehicle sensors 222 In-car cameras 3. Infrastructure Department 4 Fixed cameras 41 Communications Department 42 Photography Department 5 servers 51 Communications Department 52 Storage section 53 Control Unit 54 Analysis Department 55 Management Department 61 Parking Management Register 62 Mode Setting Information 8. Information Processing Device 81 processors 82 Main storage 83 Auxiliary storage device 84 I / F

Claims

1. Equipped with at least one processor, The at least one processor is The system manages parking management information that indicates the correspondence between multiple parking spaces in a parking lot, where a parked vehicle is a monitored parking space that is photographed by the onboard camera of another vehicle parked in another parking space, at least one monitored parking space where a parked vehicle photographs the vehicle parked in the monitored parking space by its onboard camera, and vehicles currently parked in the parking lot. Images obtained by a fixed camera installed in the parking lot capable of capturing the entirety of the aforementioned multiple parking spaces are acquired. Based on the parking management information, an on-board camera that photographs the first vehicle parked in the first parking space among the plurality of parking spaces is selected from the on-board cameras of vehicles parked in at least one monitoring parking space that corresponds to the first parking space as the monitoring parking space. The image obtained from the selected in-vehicle camera is acquired. Information processing system.

2. The at least one processor, upon receiving an abnormality detection signal from the first vehicle, selects the in-vehicle camera. The information processing system according to claim 1.

3. When the at least one processor receives an abnormality detection signal from the first vehicle, it outputs a shooting instruction to the selected in-vehicle camera. The information processing system according to claim 1.

4. The at least one processor generates a second image synchronized with the time of capture of the first image acquired from the selected in-vehicle camera by cutting it out from the image acquired from the fixed camera, and manages the first image and the second image in association with the first parking space as the monitored parking space in the parking management information. The information processing system according to claim 1.

5. The at least one processor causes the information to be managed in the parking management information to differ depending on the conditions of the parking lot. The information processing system according to claim 1.

6. The onboard camera of the vehicle parked in the monitoring parking space is a vehicle-mounted camera capable of capturing blind spots that cannot be captured by the onboard camera of the vehicle parked in the monitoring parking space, and specifically, blind spots that cannot be captured by the fixed camera. The information processing system according to claim 1.

7. The at least one processor masks the information of the at least one monitoring parking space in the parking management information relating to the range that can be captured by the fixed camera, based on the information regarding the arrangement relationship of the fixed camera for each of the plurality of parking spaces. The information processing system according to claim 1.

8. The aforementioned at least one parking space on the monitoring side is all of the parking spaces among the plurality of parking spaces except for the parking space on the monitoring side. The information processing system according to claim 1.

9. An information processing device comprising at least one processor, wherein the at least one processor performs an information processing method, The system manages parking management information that indicates the correspondence between multiple parking spaces in a parking lot, where a parked vehicle is a monitored parking space that is photographed by the onboard camera of another vehicle parked in another parking space, at least one monitored parking space where a parked vehicle photographs the vehicle parked in the monitored parking space by its onboard camera, and vehicles currently parked in the parking lot. Images obtained by a fixed camera installed in the parking lot capable of capturing the entirety of the aforementioned multiple parking spaces are acquired. Based on the parking management information, an on-board camera that photographs the first vehicle parked in the first parking space among the plurality of parking spaces is selected from the on-board cameras of vehicles parked in at least one monitoring parking space that corresponds to the first parking space as the monitoring parking space. The image obtained from the selected in-vehicle camera is acquired. Information processing methods.

10. The system manages parking management information that indicates the correspondence between multiple parking spaces in a parking lot, where a parked vehicle is a monitored parking space that is photographed by the onboard camera of another vehicle parked in another parking space, at least one monitored parking space where a parked vehicle photographs the vehicle parked in the monitored parking space by its onboard camera, and vehicles currently parked in the parking lot. Images obtained by a fixed camera installed in the parking lot capable of capturing the entirety of the aforementioned multiple parking spaces are acquired. Based on the parking management information, an on-board camera that photographs the first vehicle parked in the first parking space among the plurality of parking spaces is selected from the on-board cameras of vehicles parked in at least one monitoring parking space that corresponds to the first parking space as the monitoring parking space. To instruct the computer to acquire images obtained from the selected in-vehicle camera. program.

Images