Information processing device, information processing method, and information processing system

Abstract

To provide a technique for accurately recognizing entering of a vehicle to a parking space or leaving of the vehicle from the parking space.SOLUTION: An information processing device includes a processing section for recognizing entering of a vehicle to a parking space associated with a camera or leaving of the vehicle from the parking space based on the size of a license plate in each one of a plurality of images recognized from the images obtained by the camera.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing system. [Background technology]

[0002] In recent years, technologies for recognizing the state of a vehicle have become known. For example, Patent Document 1 discloses a technology for recognizing the type of a vehicle based on an image of the license plate of the vehicle captured by a camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-135535 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, it is desirable to provide a technology that can recognize with high accuracy the entry of a vehicle into a parking space or the exit of a vehicle from a parking space. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, an information processing device is provided that includes a processing unit that recognizes the entry of a vehicle into a parking space associated with a camera or the exit of the vehicle from the parking space based on the size of the license plate in each of a plurality of images obtained by the camera.

[0006] According to another aspect of the present disclosure, there is provided an information processing method including a processor recognizing a vehicle entering or exiting a parking space associated with a camera based on a size of a license plate in each of a plurality of images acquired by the camera.

[0007] According to another aspect of the present disclosure, there is provided an information processing system having an information processing device including a camera that obtains a plurality of images by imaging, and a processing unit that recognizes the entry of a vehicle into a parking space associated with the camera or the exit of the vehicle from the parking space based on the size of the license plate in each of the plurality of images recognized from the plurality of images. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a configuration example of an information processing system according to an embodiment of the present disclosure. [Diagram 2] 1 is a diagram showing an example of the positional relationship between a camera 10A-1 and a vehicle TA-1 parked in a berth PA-1. [Diagram 3] FIG. 13 is a diagram showing an example of the positional relationship between cameras 10A-11 and 10A-12 and vehicles TA-11 and TA-12 parked in berth PA-1. [Figure 4] 1 is a diagram illustrating an example of a functional configuration of a camera 10 according to an embodiment of the present disclosure. [Diagram 5] 2 is a diagram illustrating an example of a functional configuration of a server 20 according to an embodiment of the present disclosure. [Figure 6] 2 is a diagram showing an example of the configuration of reservation information 231. FIG. [Figure 7] 2 is a diagram showing an example of the configuration of contact information 232. FIG. [Figure 8] FIG. 2 is a diagram showing an example of the configuration of stay time information 233. [Figure 9] FIG. 13 is a diagram for explaining an example of recognition of entry of vehicle TA-1 into berth PA-n. [Figure 10] FIG. 13 is a diagram for explaining an example of recognition of the departure of vehicle TA-1 from berth PA-n. [Figure 11] 13 is a flowchart illustrating an example of a manual reservation process executed by the server 20 according to an embodiment of the present disclosure. [Figure 12]13 is a flowchart showing an example of bath-related processing executed by the server 20 according to an embodiment of the present disclosure. [Figure 13] 13 is a flowchart showing an example of a warehousing-related process executed by the server 20 according to the embodiment of the present disclosure. [Figure 14] 13 is a flowchart showing an example of a shipping-related process executed by a server 20 according to an embodiment of the present disclosure. [Figure 15] 13 is a flowchart showing an example of an automatic reservation process executed by the server 20 according to an embodiment of the present disclosure. [Figure 16] 13 is a flowchart illustrating an example of a stay time related process executed by the server 20 according to an embodiment of the present disclosure. [Figure 17] FIG. 9 is a block diagram showing an example of a hardware configuration of an information processing device 900. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0010] The explanation will be given in the following order. 1. Details of the embodiment 1.1. Example of information processing system configuration 1.2. Example of camera function configuration 1.3. Server Functional Configuration Example 1.4. Server operation example 2. Hardware configuration example 3. Summary

[0011] <1. Details of the embodiment> Next, the embodiments of the present disclosure will be described in detail.

[0012] (1.1. Example of information processing system configuration) Fig. 1 is a diagram showing a configuration example of an information processing system according to an embodiment of the present disclosure. As shown in Fig. 1, the information processing system 1 according to the embodiment of the present disclosure includes cameras 10A-1 to 10A-n (n is an integer equal to or greater than 1) present at a base A, cameras 10B-1 to 10B-n present at a base B, a server 20, a network 30, an administrator terminal 60, and a driver terminal 70.

[0013] (Location A and Location B) Here, the base A has berths PA-1 to PA-n. Each of the berths PA-1 to PA-n is a space (parking space) where a vehicle is parked, and luggage can be loaded and unloaded from the vehicle parked in the space. As an example, the vehicle TA-1 is parked in the berth PA-1. For example, the parking of the vehicle TA-1 in the berth PA-1 means that the vehicle TA-1 stops in the berth PA-1, and the time that the vehicle TA-1 stops in the berth PA-1 is not particularly limited. The camera 10A-1 and the berth PA-1 are associated in advance by the server 20. For example, correspondence information in which the identification information of the camera 10A-1 and the identification information of the berth PA-1 are associated is stored in the server 20 in advance, and the server 20 can recognize the berth PA-1 associated with the camera 10A-1 by acquiring the identification information of the berth PA-1 corresponding to the identification information of the camera 10A-1 transmitted from the camera 10A-1 from the correspondence information. In the same manner, server 20 recognizes berth PA-2 associated with camera 10A-2, . . . , and berth PA-n associated with camera 10A-n.

[0014] The cameras 10A-1 to 10A-n are provided behind or diagonally behind the vehicles parked in the berths PA-1 to PA-n. As an example, the camera 10A-1 is provided behind the vehicle TA-1 parked in the berth PA-1. Here, an example of the positional relationship between the camera 10A-1 and the vehicle TA-1 parked in the berth PA-1 will be described in detail with reference to Figs. 2 and 3.

[0015] Fig. 2 is a diagram showing an example of the positional relationship between the camera 10A-1 and the vehicle TA-1 parked in the berth PA-1. Referring to Fig. 2, the berth PA-2 and the berth PA-1 of the site A are shown in an enlarged manner. The vehicle TA-1 is parked in the berth PA-1.

[0016] Here, an example of the positional relationship between the camera 10A-1 and the vehicle TA-1 parked in the berth PA-1 will be mainly described. However, the positional relationship between the cameras 10A-2 to 10A-n and the vehicles parked in the berths PA-2 to PA-n may be similar to the positional relationship between the camera 10A-1 and the vehicle TA-1 parked in the berth PA-1.

[0017] 2, the camera 10A-1 may be provided at the rear or diagonally rear of the vehicle TA-1. More specifically, the camera 10A-1 may be provided at a position where the license plate attached to the rear face E of the vehicle TA-1 parked in the berth PA-1 fits inside the angle of view θ of the camera 10A-1. Alternatively, the camera 10A-1 may be provided at a position where the license plate attached to the rear face E of the vehicle TA-1 moving (e.g., moving in the forward and backward directions) before entering the berth PA-1 fits inside the angle of view θ of the camera 10A-1.

[0018] Typically, the camera is installed in front of the vehicle TA-1. At this time, the camera obtains a moving image of the license plate attached to the front face F of the vehicle TA-1 parked in the berth PA-1. Then, based on the moving image obtained in this manner, the entry of the vehicle TA-1 into the berth PA-1 or the exit of the vehicle TA-1 from the berth PA-1 is recognized.

[0019] However, since the area in front of the vehicle TA-1 parked in the berth PA-1 is a place where the vehicle TA-1 passes, there are cases where the camera has to be installed far away from the vehicle TA-1, and therefore the recognition accuracy of the vehicle TA-1 entering the berth PA-1 or leaving the berth PA-1 is not improved.

[0020] On the other hand, in the example shown in Fig. 2, the camera 10A-1 is provided behind or diagonally behind the vehicle TA-1. At this time, the camera 10A-1 obtains a moving image of the license plate attached to the back E of the vehicle TA-1 parked in the berth PA-1. Then, based on the moving image obtained in this manner, the entry of the vehicle TA-1 into the berth PA-1 or the exit of the vehicle TA-1 from the berth PA-1 is recognized.

[0021] The area behind or diagonally behind the vehicle TA-1 parked in the berth PA-1 is a place that the vehicle TA-1 does not pass through, so it is easy to install the camera 10A-1 near the vehicle TA-1. This can improve the accuracy of recognition of the entry of the vehicle TA-1 into the berth PA-1 or the exit of the vehicle TA-1 from the berth PA-1.

[0022] The height of the camera 10A-1 is preferably approximately the same as the height of the license plate attached to the rear surface E of the vehicle TA-1. This allows the camera 10A-1 to capture an image of the license plate from close to the front, improving the accuracy of recognition of the entry of the vehicle TA-1 into the berth PA-1 or the exit of the vehicle TA-1 from the berth PA-1.

[0023] A wall WA-1 may be provided between the camera 10A-1 and the vehicle TA-1 in the horizontal direction. The wall WA-1 may be provided at a height that can prevent the vehicle TA-1 from contacting the camera 10A-1, but does not need to be provided at a low position from the ground so that the camera 10A-1 can capture the license plate of the vehicle TA-1.

[0024] Fig. 3 is a diagram showing an example of the positional relationship between the cameras 10A-11 and 10A-12 and the vehicles TA-11 and TA-12 parked in the berth PA-1. Referring to Fig. 3, the vehicles TA-11 and TA-12 are parked in the berth PA-1. In this way, a plurality of vehicles may be parked in each of the berths PA-1 to PA-n.

[0025] Here, an example of the positional relationship between the cameras 10A-11 and 10A-12 and the vehicles TA-1 and TA-12 parked in berth PA-1 will be mainly described. However, the positional relationship between other cameras (such as the cameras 10A-21 to 10A-22 shown in FIG. 3) and vehicles parked in other berths (such as berth PA-2) may be similar to the positional relationship between the cameras 10A-11 and 10A-12 and the vehicles TA-1 and TA-12 parked in berth PA-1.

[0026] As shown in FIG. 3, camera 10A-11 may be provided behind or diagonally behind vehicle TA-11 parked in berth PA-1, and camera 10A-12 may be provided behind or diagonally behind vehicle TA-12 parked in berth PA-1.

[0027] More specifically, the camera 10A-11 may be provided at a position where the license plate attached to the rear surface E11 of the vehicle TA-11 parked in the berth PA-1 fits inside the angle of view θ of the camera 10A-11. Also, the camera 10A-12 may be provided at a position where the license plate attached to the rear surface E12 of the vehicle TA-12 parked in the berth PA-1 fits inside the angle of view θ of the camera 10A-12.

[0028] As in the example shown in FIG. 3, multiple vehicles may be parked in berth PA-1. Similarly, multiple vehicles may be parked in each of berths PA-2 to PA-n. Even in this case, the matters that apply to one vehicle parked in each of berths PA-1 to PA-n may also be applied to each of multiple vehicles parked in each of berths PA-1 to PA-n. However, in the following explanation, for the sake of simplicity, it is mainly assumed that one vehicle can be parked in each of berths PA-1 to PA-n.

[0029] Returning to FIG. 1, the explanation will be continued. Like base A, base B has berths PB-1 to PB-n. As an example, a vehicle TB-n is parked in the berth PB-n. The cameras 10B-1 to 10B-n are provided behind or diagonally behind the vehicles parked in the berths PB-1 to PB-n. As an example, the camera 10B-n is provided behind the vehicle TB-n parked in the berth PB-n.

[0030] In the following description, when the content applies to any of the cameras 10A-1 to 10A-n, the cameras 10A-1 to 10A-n may be referred to as the cameras 10A without any particular distinction. Similarly, the vehicles TA-1 to TA-n may be referred to as the vehicles TA. Furthermore, the berths PA-1 to PA-n may be referred to as the berths PA.

[0031] Similarly, cameras 10B-1 to 10B-n may be referred to as camera 10B without any particular distinction. Vehicles TB-1 to TB-n may be referred to as vehicle TB. Berths PB-1 to PB-n may be referred to as berths PB. Cameras 10A-1 to 10A-n and cameras 10B-1 to 10B-n may be referred to as cameras 10 without any particular distinction.

[0032] In the example shown in Fig. 1, the number of berths PA at base A and the number of berths PB at base B are the same number n, but the number of berths PA at base A and the number of berths PB at base B may be different numbers. Also, Fig. 1 shows a case where there are two bases (i.e., two bases, base A and base B), but the number of bases may be one, or three or more.

[0033] (Server 20) The server 20 is realized by a computer and functions as an example of an information processing device. In the following description, the functions of the camera 10, the server 20, the administrator terminal 60, and the driver terminal 70 will be described in more detail, but the division of functions between the server 20 and the camera 10, the administrator terminal 60, and the driver terminal 70 is not limited. For example, some of the functions of the camera 10 may be held by the server 20 instead of the camera 10.

[0034] (Network 30) The network 30 relays communications between devices connected to the network 30. Here, the type of the network 30 is not limited.

[0035] As a specific example, the network 30 may be configured as a so-called wireless network, such as a network based on the Wi-Fi (registered trademark) standard. As another example, the network 30 may be configured as the Internet, a dedicated line, a LAN (Local Area Network), a WAN (Wide Area Network), or the like. The network 30 may include a plurality of networks, and at least a part of the network may be configured as a wired network.

[0036] (Administrator terminal 60) The administrator terminal 60 is realized by a computer and is a terminal used by the administrator of the base. Here, it is mainly assumed that the administrator terminal 60 is a terminal used by the administrator of the base A. However, the information processing system 1 may also include an administrator terminal used by the administrator of the base B. The administrator terminal 60 may be a PC (Personal Computer) provided in a management room of the base, or may be a terminal (such as a smartphone) that can be carried by the administrator.

[0037] The administrator terminal 60 has an input device (not shown) and may acquire information input by the administrator to the input device from the input device. The input device may be a touch panel, a mouse, a keyboard, a microphone, or another input device.

[0038] The manager terminal 60 may have a presentation device (not shown) and present the information transmitted from the server 20 to the manager by the presentation device. This allows the manager to perceive the information presented by the presentation device. The presentation device may be a display that presents visual information, a speaker that presents audio information, or a force feedback device that presents a force feedback.

[0039] (Driver terminal 70) The driver terminal 70 is realized by a computer and is a terminal used by the driver of the vehicle (i.e., the driver). Here, it is mainly assumed that the driver terminal 70 is a terminal used by the driver of the vehicle TA-1. However, the information processing system 1 may be equipped with a driver terminal used by the driver of another vehicle. The driver terminal 70 may be a terminal provided in the interior of the vehicle TA-1, or may be a terminal (e.g., a smartphone, etc.) that can be carried by the driver.

[0040] The driver terminal 70 has an input device (not shown) and may acquire information input by the driver to the input device from the input device. The input device may be a touch panel, a mouse, a keyboard, a microphone, or another input device.

[0041] The driver terminal 70 may have a presentation device (not shown) and present the information transmitted from the server 20 to the driver by the presentation device. This allows the driver to perceive the information presented by the presentation device. The presentation device may be a display that presents visual information, a speaker that presents audio information, or a force feedback device that presents a force feedback.

[0042] The configuration example of the information processing system 1 according to the embodiment of the present disclosure has been described above.

[0043] (1.2. Example of camera function configuration) Next, an example of the functional configuration of the camera 10 according to the embodiment of the present disclosure will be described with reference to FIG.

[0044] Fig. 4 is a diagram showing an example of a functional configuration of the camera 10 according to the embodiment of the present disclosure. As shown in Fig. 4, the camera 10 according to the embodiment of the present disclosure includes an image sensor 110, a CIS signal processing unit 120, an ISP unit 130, and a recognition model 140. Note that CIS is an abbreviation for a CMOS (Complementary Metal Oxide Semiconductor) image sensor, but the type of the image sensor is not particularly limited. ISP is an abbreviation for Image Signal Processor.

[0045] (Image Sensor 110) The image sensor 110 receives light from a real space and converts the intensity of the received light into an electrical signal. The image sensor 110 sequentially outputs the electrical signals to the CIS signal processor 120. As described above, the image sensor 110 may be a CMOS image sensor, but the type of the image sensor is not particularly limited.

[0046] (CIS signal processing unit 120) The CIS signal processing unit 120 performs various signal processing on the electrical signal input from the image sensor 110. For example, the CIS signal processing unit 120 performs re-mosaic processing, defect correction, exposure time control of the image sensor 110, analog gain adjustment of the image sensor 110, and the like on the signal input from the image sensor 110.

[0047] Parameters are set in the CIS signal processing unit 120. The parameters are adjusted for re-mosaic processing of the signal input from the image sensor 110, defect correction, exposure time control of the image sensor 110, and analog gain adjustment of the image sensor 110.

[0048] (ISP section 130) The ISP unit 130 performs various image signal processing on the image signal after the signal processing by the CIS signal processing unit 120. For example, the ISP unit 130 performs demosaic processing, sharpening processing, noise removal, resolution conversion, digital gain adjustment, tone mapping, color correction, color conversion, normalization processing, quantization, and the like on the image signal after the signal processing by the CIS signal processing unit 120.

[0049] (Cognitive Model 140) The recognition model 140 is a model generated by learning, and performs recognition based on the image processed by the ISP unit 130. For example, the recognition model 140 is configured by a trained DNN (Deep Neural Network). In the following description, the image processed by the ISP unit 130 is also referred to as a "frame."

[0050] More specifically, the recognition model 140 includes a CNN (Convolution Neural Network) and performs frame-based license plate recognition. As an example of a license plate recognition result, the recognition model 140 recognizes the position of the license plate area (i.e., the bounding box corresponding to the license plate) surrounding the license plate from the frame and the vehicle number written on the license plate.

[0051] For example, the license plate area may be a rectangular area. The position of the license plate area may be expressed by the coordinates of the upper left corner and the coordinates of the lower right corner of the license plate area in the frame. Alternatively, the position of the license plate area may be expressed by the coordinates of the upper right corner and the coordinates of the lower left corner of the license plate area in the frame.

[0052] The vehicle number may be a part or the whole of the number written on the license plate. Here, it is mainly assumed that the last four digits of the number written on the license plate (e.g., "12-34") are used as the vehicle number. The recognition model 140 outputs the position of the license plate area and the vehicle number to the server 20.

[0053] The recognition model 140 outputs the vehicle number based on the vehicle number recognized from multiple frames. Here, the recognition model 140 may output the recognized vehicle number itself. Alternatively, when the vehicle numbers recognized from j or more frames (j is an integer greater than or equal to 1 and less than or equal to i) among the vehicle numbers recognized from i frames (i is an integer greater than or equal to 1) are the same, the recognition model 140 may output the vehicle numbers recognized from the j or more frames as true values. This improves the accuracy of the vehicle numbers output from the recognition model 140.

[0054] Note that the recognition model 140 may output an image of the license plate area cut out from the frame (i.e., a cut-out image) and the frame, in addition to the position of the license plate area and the vehicle number, to the server 20. At this time, in the server 20, the cut-out image and the frame may be received by the communication unit 240 (FIG. 5) and stored in the storage unit 230 (FIG. 5).

[0055] 4 shows a case where recognition model 140 exists inside camera 10 and outside image sensor 110, and this recognition model 140 recognizes the position of the license plate area and the vehicle number written on the license plate. However, one or both of the recognition of the position of the license plate area and the recognition of the vehicle number written on the license plate may be performed inside image sensor 110 (for example, an AI (Artificial Intelligence) processing unit (not shown) built into image sensor 110). For example, the AI ​​processing unit may be realized by a DSP (Digital Signal Processor) (not shown) built into image sensor 110.

[0056] For example, both the recognition of the position of the license plate area and the recognition of the vehicle number written on the license plate may be performed inside the image sensor 110. In this case, the output from the image sensor 110 may include the position of the license plate area and the vehicle number recognized inside the image sensor 110. Furthermore, the output from the image sensor 110 may include an image of the license plate area cut out from the frame and the frame, in addition to the position of the license plate area and the vehicle number recognized inside the image sensor 110.

[0057] Alternatively, recognition of the position of the license plate area may be performed inside the image sensor 110, and recognition of the vehicle number on the license plate may be performed by a recognition model 140 located inside the camera 10 and outside the image sensor 110.

[0058] An example of the functional configuration of the camera 10 according to the embodiment of the present disclosure has been described above.

[0059] (1.3. Server Functional Configuration Example) Next, an example of a functional configuration of the server 20 according to the embodiment of the present disclosure will be described mainly with reference to FIG.

[0060] 5 is a diagram illustrating an example of a functional configuration of the server 20 according to an embodiment of the present disclosure. As illustrated in FIG. 5, the server 20 according to an embodiment of the present disclosure includes a control unit 220, a storage unit 230, and a communication unit 240.

[0061] (Control unit 220) The control unit 220 may be configured, for example, by one or more CPUs (Central Processing Units). When the control unit 220 is configured by a processing unit such as a CPU, the processing unit may be configured by an electronic circuit. The control unit 220 may be realized by executing a program by the processing unit.

[0062] The control unit 220 includes an acquisition unit 221, a processing unit 222, and a transmission control unit 223. The functions of the acquisition unit 221, the processing unit 222, and the transmission control unit 223 will be described in detail later.

[0063] (Storage unit 230) The storage unit 230 is a recording medium including a memory, which stores a program executed by the control unit 220 and stores data necessary for executing the program. The storage unit 230 also temporarily stores data for calculations by the control unit 220. The storage unit 230 is configured by a magnetic storage device, a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

[0064] The storage unit 230 stores reservation information 231, contact information 232, and stay time information 233. Here, configuration examples of the reservation information 231, contact information 232, and stay time information 233 will be described with reference to Figs.

[0065] (Reservation information 231) Fig. 6 is a diagram showing an example of the configuration of reservation information 231. As shown in Fig. 6, reservation information 231 is configured by associating berths included in a base, time, vehicle numbers, and statuses. In the example shown in Fig. 6, the width of each time is one hour, but the width of each time is not limited to one hour. For example, the width of each time may be 30 minutes.

[0066] The vehicle number is the vehicle number of a vehicle that has been reserved for a berth and time. For example, referring to FIG. 6, vehicle number "12-34" is registered for berth "PA-1" and time "August 20th, 7:00-8:00". This indicates that a vehicle with vehicle number "12-34" has been reserved to use berth "PA-1" for the time "August 20th, 7:00-8:00".

[0067] The status indicates the reservation status for a berth and time. For example, before a reservation is made, the status is "Not reserved". After a reservation is made and before a vehicle arrives, the status is "Not arrived". After a vehicle arrives and before the vehicle departs, the status is "Arrived". After a vehicle departs, the status is "Departed".

[0068] For example, referring to Fig. 6, vehicle number "12-34" and status "arrival" are registered for berth "PA-1" and time "August 20th, 7:00-8:00". This indicates that a vehicle with vehicle number "12-34" has been reserved to use berth "PA-1" at the time "August 20th, 7:00-8:00", and that the vehicle has already arrived at that time and berth.

[0069] An example of the configuration of the reservation information 231 has been described above.

[0070] (Contact Information 232) Fig. 7 is a diagram showing a configuration example of the contact information 232. As shown in Fig. 7, the contact information 232 is configured by associating a vehicle number with a contact. For example, the contact information 232 is input by a driver of the vehicle and registered in the storage unit 230. For example, the contact may be a contact of the driver of the vehicle. For example, as shown in Fig. 7, the contact may be an email address of the driver terminal 70 of the driver.

[0071] For example, referring to FIG. 7, the vehicle number "12-34" is associated with the contact information "abc@def.jp". This indicates that the contact information of the driver of the vehicle having the vehicle number "12-34" is the email address "abc@def.jp". Note that the contact information is not limited to the email address of the driver terminal 70, and may be the telephone number of the driver terminal 70, etc.

[0072] An example of the configuration of the contact information 232 has been described above.

[0073] (Stay time information 233) Fig. 8 is a diagram showing an example of the configuration of the stay time information 233. As shown in Fig. 8, the stay time information 233 is configured by associating berths included in a base, time, arrival time, departure time, and stay time.

[0074] The arrival time is the time when a vehicle reserved for a berth and time arrives at the berth. For example, referring to FIG. 8, it is shown that a vehicle reserved for berth "PA-1" and time "August 20th 7:00-8:00" arrived at the berth at the arrival time "7:10".

[0075] The departure time is the time when a vehicle reserved for a berth and time departs from the berth. For example, referring to FIG. 8, it is shown that a vehicle reserved for berth "PA-n" and time "August 20th 7:00-8:00" departs from the berth at departure time "7:50".

[0076] The stay time is the time that a vehicle reserved for a berth and time stays at that berth. For example, referring to Fig. 8, it is shown that a vehicle reserved for berth "PA-n" and time "August 20th 7:00-8:00" stayed at that berth for a stay time of "0:45" from arrival time "7:05" to departure time "7:50".

[0077] 8 further shows that the vehicle reserved for berth "PA-1" and time "August 20th, 7:00-8:00" has not yet departed from that berth. Also, assuming that the current time is "8:20", it shows that the vehicle has been at that berth for a stay time of "1:10", from the arrival time of "7:10" to the current time of "8:20".

[0078] Above, we have explained an example of the configuration of the stay time information 233. Returning to FIG.

[0079] (Communication unit 240) The communication unit 240 includes a communication interface, and communicates with the camera 10 , the administrator terminal 60 , and the driver terminal 70 via the network 30 .

[0080] An example of the functional configuration of the server 20 according to the embodiment of the present disclosure has been described above.

[0081] (1.4. Server operation example) Next, an example of the operation of the server 20 according to the embodiment of the present disclosure will be described with reference mainly to Fig. 9 to Fig. 16. Fig. 9 is a diagram for explaining an example of recognition of the entry of the vehicle TA-1 into the berth PA-n. Fig. 10 is a diagram for explaining an example of recognition of the exit of the vehicle TA-1 from the berth PA-n.

[0082] (Manual booking process) Fig. 11 is a flowchart showing an example of manual reservation processing executed by the server 20 according to an embodiment of the present disclosure. More specifically, the flowchart shown in Fig. 11 is a flowchart showing an example of manual reservation processing executed by the server 20 when a vehicle driver manually makes a reservation for a berth and time. Here, it is mainly assumed that the driver of vehicle TA-1 attempts to make a reservation for location A and time "August 20th, 7:00-8:00".

[0083] First, when the driver of vehicle TA-1 inputs location A and time "August 20th, 7:00-8:00" for which he / she wishes to make a reservation into the input device of the driver terminal 70, location A and time "August 20th, 7:00-8:00" are transmitted from the driver terminal 70 to the server 20. In the server 20, the acquisition unit 221 acquires location A and time "August 20th, 7:00-8:00" transmitted from the driver terminal 70 via the communication unit 240 (S111).

[0084] The processing unit 222 judges whether or not there is an unreserved berth at the location A and the time "August 20th, 7:00-8:00" acquired by the acquisition unit 221 based on the reservation information 231 (S112). If there is no unreserved berth at the location A and the time "August 20th, 7:00-8:00" acquired by the acquisition unit 221 ("NO" in S112), the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the reservation has failed is transmitted as a reservation failure notification to each of the manager terminal 60 of the location A and the driver terminal 70 of the driver of the vehicle TA-1 by the communication unit 240 (S117).

[0085] On the other hand, if there is an unreserved berth at location A and time "August 20th, 7:00-8:00" ("YES" in S112), the driver enters the vehicle number "12-34" of vehicle TA-1 into the input device of the driver terminal 70, which is then transmitted from the driver terminal 70 to the server 20, and the acquisition unit 221 acquires the vehicle number transmitted from the driver terminal 70 (S113).

[0086] Here, it is assumed that the vehicle number corresponding to berth PA-1 at base A and the time "August 20th, 7:00-8:00" is not registered and the status is "unreserved" in the reservation information 231. In this case, the processing unit 222 registers the vehicle number "12-34" of the vehicle TA-1 input by the driver to the input device of the driver terminal 70 as the vehicle number corresponding to berth PA-1 at base A and the time "August 20th, 7:00-8:00" (S114).

[0087] The processing unit 222 updates the status corresponding to the berth PA-1 at the base A and the time "August 20, 7:00-8:00" from "unreserved" to "unarrived" (S115). Then, the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the reservation has been completed is transmitted as a reservation completion notification to each of the manager terminal 60 at the base A and the driver terminal 70 of the driver of the vehicle TA-1 by the communication unit 240 (S116).

[0088] An example of the manual reservation process executed by the server 20 according to the embodiment of the present disclosure has been described above.

[0089] (Berth related processing) Fig. 12 is a flowchart showing an example of bath-related processing executed by the server 20 according to an embodiment of the present disclosure. More specifically, the flowchart shown in Fig. 12 is a flowchart showing an example of bath-related processing repeatedly executed by the server 20 during operation of the server 20.

[0090] As shown in Fig. 12, in the berth-related process, a warehousing-related process S20 is executed, a shipping-related process S30 is executed, and a stay time-related process S50 is executed. Then, if the operation of the server 20 is not terminated ("NO" in S60), the operation proceeds to S20. On the other hand, if the operation of the server 20 is terminated ("YES" in S60), the berth-related process is terminated.

[0091] An example of the bath-related process executed by the server 20 according to the embodiment of the present disclosure has been described above.

[0092] (Goods receipt related processing) Fig. 13 is a flowchart showing an example of a warehousing-related process executed by the server 20 according to an embodiment of the present disclosure. More specifically, the flowchart shown in Fig. 13 is a flowchart showing an example of a warehousing-related process executed by the server 20 by being called from the berth-related process shown in Fig. 12. Here, a case is mainly assumed in which a vehicle TA-1 is warehousing at a berth PA-1.

[0093] As described above, the position of the license plate area and the vehicle number are transmitted from each of cameras 10A-1 to 10A-n and 10B-1 to 10B-n to server 20. In server 20, acquisition unit 221 acquires, via communication unit 240, the position of the license plate area and the vehicle number transmitted from each of cameras 10A-1 to 10A-n and 10B-1 to 10B-n.

[0094] Processing unit 222 calculates the size of the license plate in the frame (hereinafter also simply referred to as "license plate size") based on the position of the license plate area transmitted from each of cameras 10A-1 to 10A-n, 10B-1 to 10B-n. Here, it is mainly assumed that the size of the license plate is the area of ​​the license plate. Since the error caused in the area of ​​the license plate due to fluctuation of the license plate is considered to be small, it is expected that the recognition accuracy will be improved. However, the size of the license plate may be the vertical length of the license plate or the horizontal length of the license plate.

[0095] For example, when the position of the license plate area is expressed by the coordinates of the upper left corner and the lower right corner of the license plate area, the processing unit 222 can calculate the horizontal length of the license plate by subtracting the horizontal axis coordinate of the upper left corner from the horizontal axis coordinate of the lower right corner of the license plate area, and can calculate the vertical length of the license plate by subtracting the vertical axis coordinate of the lower right coordinate from the vertical axis coordinate of the upper left corner of the license plate area. The processing unit 222 can also calculate the area of ​​the license plate by multiplying the horizontal length of the license plate by the vertical length of the license plate.

[0096] The processing unit 222 determines whether or not there is a berth in which the vehicle has been stored, based on the size of the license plate calculated from multiple frames (S211). If there is no berth in which the vehicle has been stored ("NO" in S211), the warehousing-related processing ends. On the other hand, if there is a berth in which the vehicle has been stored ("YES" in S211), the processing unit 222 proceeds to operation S212. Here, it is mainly assumed that vehicle TA-1 has been stored in berth PA-1.

[0097] An example of recognition of the entry of vehicle TA-1 into berth PA-1 will be described with reference to Fig. 9. Referring to Fig. 9, frames M0 and M1 are shown. Frame M1 is a frame (second frame) obtained by camera 10A-1 at current time t1. Frame M0 is a frame (first frame) obtained by camera 10A-1 at time t0, which is a predetermined time before the current time t1.

[0098] The vertical length H0 is the vertical length of the license plate N0 recognized from the frame M0. The horizontal length W0 is the horizontal length of the license plate N0 recognized from the frame M0. For example, the processing unit 222 can calculate the size (first size) of the license plate N0 by multiplying the vertical length H0 and the horizontal length W0.

[0099] Similarly, the vertical length H1 is the vertical length of the license plate N1 recognized from the frame M1. Moreover, the horizontal length W1 is the horizontal length of the license plate N1 recognized from the frame M1. For example, the processing unit 222 may calculate the size (second size) of the license plate N1 by multiplying the vertical length H1 by the horizontal length W1. Here, the vehicle TA-1 is about to enter the berth PA-1, so the size of the license plate N1 is larger than the size of the license plate N0.

[0100] The processing unit 222 obtains a comparison result by comparing the size of the license plate N0 recognized from the frame M0 acquired by the camera 10A-1 a predetermined time before the current time with the size of the license plate N1 recognized from the frame M1 acquired by the camera 10A-1 at the current time. The comparison result may include a comparison result in which the size of the license plate N1 is larger than the size of the license plate N0 by a first threshold or more (hereinafter also referred to as a "comparison result indicating an increase in size"), a comparison result in which the size of the license plate N1 is smaller than the size of the license plate N0 by a second threshold or more (hereinafter also referred to as a "comparison result indicating a decrease in size"), or a comparison result that shows neither an increase in size nor a decrease in size (hereinafter also referred to as a "comparison result indicating no change in size").

[0101] The first threshold may be set to any positive value or may be set to zero. When the first threshold is a positive value, the comparison result indicates size invariance when the increase amount from the size of license plate N0 to the size of license plate N1 (i.e., the amount obtained by subtracting the size of license plate N0 from the size of license plate N1) is smaller than the first threshold. Similarly, the second threshold may be set to any positive value or may be set to zero. When the second threshold is a positive value, the comparison result indicates size invariance when the decrease amount from the size of license plate N0 to the size of license plate N1 (i.e., the amount obtained by subtracting the size of license plate N1 from the size of license plate N0) is smaller than the second threshold. The first threshold and the second threshold may be the same value or different values.

[0102] Then, the processing unit 222 may recognize the entry of the vehicle TA-1 into the berth PA-1 when a condition (hereinafter also referred to as a "first entry condition") is satisfied that an event in which the comparison result indicates a size increase has been detected at least once (i.e., once or two or more consecutive times). This allows the processing unit 222 to recognize the entry of the vehicle TA-1 into the berth PA-1 with high accuracy.

[0103] Alternatively, the driver of the vehicle TA-1 may drive the vehicle TA-1 back and forth (so-called turning maneuver) in order to adjust the position and direction of the vehicle TA-1 in the berth PA-1. In such a case, even if the vehicle TA-1 is turning, the processing unit 222 may erroneously recognize that the vehicle TA-1 has entered the berth PA-1.

[0104] Therefore, in consideration of the fact that the recognition of the license plate may be interrupted during the turning maneuver of the vehicle TA-1, the processing unit 222 may recognize the entry of the vehicle TA-1 into the berth PA-1 when the first entry condition is satisfied and when the condition (second entry condition) that the time during which the license plate is continuously recognized is equal to or longer than a predetermined first time is satisfied. This reduces the possibility of erroneously recognizing the entry of the vehicle TA-1 when the vehicle TA-1 moves forward due to turning maneuver and the license plate is not recognized for a time shorter than the first time, even if the size of the license plate of the vehicle TA-1 increases once, and therefore the entry of the vehicle TA-1 into the berth PA-1 can be recognized with even higher accuracy.

[0105] Alternatively, the processing unit 222 may recognize the entry of the vehicle TA-1 into the berth PA-1 when the first entry condition is satisfied and when a condition (third entry condition) is satisfied in which the time during which the size of the license plate or the position of the license plate area does not change is equal to or longer than a predetermined second time. This reduces the possibility of erroneously recognizing the entry of the vehicle TA-1 when, for example, the vehicle TA-1 moves forward due to a turnaround and the license plate moves for a time shorter than the second time, even if the size of the license plate of the vehicle TA-1 increases once, so that the entry of the vehicle TA-1 into the berth PA-1 can be recognized with even higher accuracy.

[0106] Alternatively, it may be assumed that the license plate of the vehicle TA-1 is not within the angle of view of the camera 10A-1 at the time when the vehicle TA-1 stops at the berth PA-1 or immediately before the vehicle TA-1 stops there, and the license plate is not recognized. Therefore, the berth processing unit 222 may recognize the entry of the vehicle TA-1 into the berth PA-1 when the first entry condition is satisfied and when a condition (fourth entry condition) is satisfied in which the license plate is not recognized for a predetermined third time or more is satisfied. This reduces the possibility that the entry of the vehicle TA-1 is erroneously recognized when the vehicle TA-1 moves forward due to a turning maneuver and the license plate is not recognized for a time shorter than the third time, even if the size of the license plate of the vehicle TA-1 is once increased, and therefore the entry of the vehicle TA-1 into the berth PA-1 can be recognized with even higher accuracy.

[0107] Continuing the explanation by returning to Fig. 13, the processing unit 222 determines, based on the reservation information 231, whether or not the status corresponding to the berth PA-1 where the vehicle TA-1 is stored and the time to which the current time belongs indicates "not yet arrived" (S212).

[0108] If the status does not indicate non-arrival ("NO" in S212), the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the status is abnormal is sent as a status abnormality notification to each of the administrator terminal 60 at base A and the driver terminal 70 of the driver of vehicle TA-1 (S213).

[0109] On the other hand, when the status indicates that the vehicle has not yet arrived ("YES" in S212), the acquisition unit 221 acquires the vehicle number of the vehicle TA-1 output from the camera 10A-1. Here, it is assumed that the acquisition unit 221 acquires the vehicle number "12-34" of the vehicle TA-1 output from the camera 10A-1.

[0110] Then, the processing unit 222 acquires the vehicle number corresponding to the berth PA-1 and the time to which the current time belongs, which are registered in the reservation information 231. Here, it is assumed that the acquisition unit 221 acquires the vehicle number "12-34" as the vehicle number corresponding to the berth PA-1 and the time to which the current time belongs, which are registered in the reservation information 231.

[0111] The processing unit 222 compares the vehicle number of the vehicle TA-1 acquired from the camera 10A-1 with the vehicle number acquired from the reservation information 231 (S214). Whether or not the comparison is successful may be determined based on whether or not all of the corresponding four digits of these vehicle numbers match. Alternatively, the comparison may be considered successful when a part of the corresponding four digits of these vehicle numbers match. If the comparison is not successful ("NO" in S214), the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the comparison of the vehicle numbers has failed is transmitted as a vehicle number comparison failure notification to each of the administrator terminal 60 of the base A and the driver terminal 70 of the driver of the vehicle TA-1 (S215).

[0112] On the other hand, if the matching is successful ("YES" in S214), the processing unit 222 updates the status corresponding to the time including berth PA-1 and the current time in the reservation information 231 from "not arrived" to "arrived" (S216). Then, the processing unit 222 registers the current time in the arrival time corresponding to the time including berth PA-1 and the current time in the stay time information 233 (S217). Furthermore, the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the matching of the vehicle number has been successful is transmitted as a vehicle number matching success notification to the manager terminal 60 of the base A and the driver terminal 70 of the driver of the vehicle TA-1.

[0113] 13 shows an example in which the determinations are made in the order of S211, S212, and S214. However, the determinations of S211, S212, and S214 do not necessarily have to be made in the order shown in FIG.

[0114] It is also assumed that the license plate of the vehicle TA-1 is not recognized even though the vehicle TA-1 has entered the berth PA-1. Therefore, the transmission control unit 223 may control the communication unit 240 so that, when the license plate is not recognized even after a predetermined time (e.g., 10 minutes) has passed since the start time "7:00" of the reservation time for the berth PA-1 "August 20th, 7:00-8:00", a notification indicating that the vehicle will not arrive at the reserved berth is transmitted to each of the manager terminal 60 of the base A and the driver terminal 70 of the driver of the vehicle TA-1.

[0115] An example of the warehousing-related process executed by the server 20 according to the embodiment of the present disclosure has been described above.

[0116] (Warehouse issue related processing) Fig. 14 is a flowchart showing an example of a warehousing-related process executed by the server 20 according to an embodiment of the present disclosure. More specifically, the flowchart shown in Fig. 14 is a flowchart showing an example of a warehousing-related process executed by the server 20 by being called from the berth-related process shown in Fig. 12. Here, a case where the vehicle TA-1 is warehousing from the berth PA-1 is mainly assumed.

[0117] The processing unit 222 determines whether or not there is a berth from which the vehicle has been deported, based on the license plate size calculated from multiple frames (S311). If there is no berth from which the vehicle has been deported ("NO" in S311), the deportation-related processing ends. On the other hand, if there is a berth from which the vehicle has been deported ("YES" in S311), the processing unit 222 shifts the operation to S312. Here, it is mainly assumed that the vehicle TA-1 has been deported from berth PA-1.

[0118] An example of recognition of vehicle TA-1 leaving berth PA-1 will be described with reference to Fig. 10. Referring to Fig. 10, frames M2 and M3 are shown. Frame M3 is a frame (fourth frame) obtained by camera 10A-1 at current time t3. Frame M2 is a frame (third frame) obtained by camera 10A-1 at time t2, which is a predetermined time before the current time t3.

[0119] The vertical length H2 is the vertical length of the license plate N2 recognized from the frame M2. The horizontal length W2 is the horizontal length of the license plate N2 recognized from the frame M2. For example, the processing unit 222 can calculate the size (third size) of the license plate N2 by multiplying the vertical length H2 and the horizontal length W2.

[0120] Similarly, the vertical length H3 is the vertical length of license plate N3 recognized from frame M3. Moreover, the horizontal length W3 is the horizontal length of license plate N3 recognized from frame M3. For example, the processing unit 222 may calculate the size (fourth size) of license plate N3 by multiplying the vertical length H3 by the horizontal length W3. Here, because vehicle TA-1 is about to leave berth PA-1, the size of license plate N3 is smaller than the size of license plate N2.

[0121] The processing unit 222 obtains a comparison result by comparing the size of the license plate N2 recognized from the frame M2 obtained by the camera 10A-1 a predetermined time before the current time with the size of the license plate N3 recognized from the frame M3 obtained by the camera 10A-1 at the current time. The processing unit 222 may recognize the departure of the vehicle TA-1 from the berth PA-1 when a condition (exit condition) is satisfied in which an event is detected at least once (i.e., once or twice or more consecutively) in which the size of the license plate N3 recognized from the frame M3 is smaller than the size of the license plate N2 recognized from the frame M2 obtained by the camera 10A-1 a predetermined time before the current time by a second threshold value or more. This allows the processing unit 222 to recognize the departure of the vehicle TA-1 from the berth PA-1 with high accuracy.

[0122] Continuing the explanation by returning to Fig. 14, the processing unit 222 determines, based on the reservation information 231, whether or not the status corresponding to the berth PA-1 from which the vehicle TA-1 was departed and the time to which the current time belongs indicates arrival (S312).

[0123] If the status does not indicate arrival ("NO" in S312), the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the status is abnormal is sent as a status abnormality notification to each of the administrator terminal 60 at base A and the driver terminal 70 of the driver of vehicle TA-1 (S313).

[0124] On the other hand, if the status indicates arrival ("YES" in S312), the acquisition unit 221 acquires the vehicle number of the vehicle TA-1 output from the camera 10A-1. Here, it is assumed that the acquisition unit 221 acquires the vehicle number "12-34" of the vehicle TA-1 output from the camera 10A-1.

[0125] Then, the processing unit 222 acquires the vehicle number corresponding to the berth PA-1 and the time to which the current time belongs, which are registered in the reservation information 231. Here, it is assumed that the acquisition unit 221 acquires the vehicle number "12-34" as the vehicle number corresponding to the berth PA-1 and the time to which the current time belongs, which are registered in the reservation information 231.

[0126] The processing unit 222 compares the vehicle number of the vehicle TA-1 acquired from the camera 10A-1 with the vehicle number acquired from the reservation information 231 (S314). If the comparison is unsuccessful ("NO" in S314), the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the comparison of the vehicle number has failed is sent as a vehicle number comparison failure notification to each of the manager terminal 60 of the base A and the driver terminal 70 of the driver of the vehicle TA-1 (S315).

[0127] On the other hand, if the collation is successful ("YES" in S314), the processing unit 222 updates the status corresponding to the time including berth PA-1 and the current time in the reservation information 231 from "arrival" to "departure" (S316). Then, the processing unit 222 registers the current time in the departure time corresponding to the time including berth PA-1 and the current time in the stay time information 233 (S317).

[0128] An example of the shipping-related process executed by the server 20 according to the embodiment of the present disclosure has been described above.

[0129] (Automatic reservation processing) Fig. 15 is a flowchart showing an example of an automatic reservation process executed by the server 20 according to an embodiment of the present disclosure. More specifically, the flowchart shown in Fig. 15 is a flowchart showing an example of an automatic reservation process executed by the server 20 by being called from the leaving-related process shown in Fig. 14. Here, it is mainly assumed that a reservation is automatically made for the next base B after the base A from which the vehicle TA-1 departs, and for the time that the vehicle TA-1 will stay at the next base B.

[0130] First, the processing unit 222 predicts the destination location B of the vehicle TA-1 that departs from the location A, and the time that the vehicle TA-1 will stay at the destination location B (S411). Here, the destination location B of the vehicle TA-1, and the time that the vehicle TA-1 will stay at the destination location B may be predicted in any manner.

[0131] For example, if a route indicating that the destination base of vehicle TA-1 from base A is base B is registered in advance in the storage unit 230, the destination base B may be predicted from the route registered in the storage unit 230. Furthermore, the time that vehicle TA-1 stays at destination base B may be a time later than the time that includes the departure time of vehicle TA-1 from base A (for example, the time next to the time that includes the departure time).

[0132] Here, we will mainly assume that the time to which vehicle TA-1 departs from location A falls is "August 20th, 8:00-9:00," and that the time next to the departure time, "August 20th, 8:00-9:00," is "August 20th, 9:00-10:00," which is the time to which vehicle TA-1 is predicted to stay at location B, its destination.

[0133] The processing unit 222 judges whether or not there is an unreserved berth at the predicted location B and the predicted time "August 20th, 9:00-10:00" based on the reservation information 231 (S412). If there is no unreserved berth at the location B and the time "August 20th, 9:00-10:00" ("NO" in S412), the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the reservation has failed is transmitted as a reservation failure notification to each of the manager terminal 60 of the location B and the driver terminal 70 of the driver of the vehicle TA-1 by the communication unit 240 (S417).

[0134] On the other hand, if there is an unreserved berth at location B and time "August 20th 9:00-10:00" ("YES" in S412), the vehicle number "12-34" of vehicle TA-1 that has already been input by the driver is obtained.

[0135] Here, it is assumed that the vehicle number corresponding to berth PB-1 at base B and the time "August 20th, 9:00-10:00" is not registered and the status is "unreserved" in the reservation information 231. In this case, the processing unit 222 registers the vehicle number "12-34" of the recognized vehicle TA-1 as the vehicle number corresponding to berth PB-1 at base B and the time "August 20th, 9:00-10:00" (S414).

[0136] The processing unit 222 updates the status corresponding to berth PB-1 at base B and the time "August 20, 9:00-10:00" from "unreserved" to "unarrived" (S415). Then, the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the reservation has been completed is transmitted as a reservation completion notification to each of the manager terminal 60 at base B and the driver terminal 70 of the driver of the vehicle TA-1 by the communication unit 240 (S416).

[0137] An example of the automatic reservation process executed by the server 20 according to the embodiment of the present disclosure has been described above.

[0138] (Time spent processing) Fig. 16 is a flowchart showing an example of stay time related processing executed by the server 20 according to an embodiment of the present disclosure. More specifically, the flowchart shown in Fig. 16 is a flowchart showing an example of stay time related processing executed by the server 20 by being called from the bath related processing shown in Fig. 12. Here, it is mainly assumed that the current time is "8:20".

[0139] First, the processing unit 222 determines whether a predetermined time has elapsed (S511). For example, the predetermined time may elapse at a predetermined cycle (for example, 5 minutes). If the predetermined time has not elapsed ("NO" in S511), the stay time related processing ends. On the other hand, if the predetermined time has elapsed ("YES" in S511), the processing unit 222 calculates the stay time of the vehicle in each of the berths PA-1 to PA-n and PB-1 to PB-n (S512).

[0140] With reference to the stay time information 233 (FIG. 8), for berth PA-n and time "August 20, 7:00-8:00", the arrival time of vehicle TA-n is "7:05", and the departure time of vehicle TA-n is "7:50". When the departure times are registered in this manner, the processing unit 222 may calculate the stay time of vehicle TA-n at berth PA-n of "0:45" by subtracting the arrival time "7:05" from the departure time "7:50".

[0141] Furthermore, referring to the stay time information 233 (FIG. 8), for berth PA-1 and time "August 20, 7:00-8:00", the arrival time of vehicle TA-1 is "7:10", and the departure time of vehicle TA-n is not registered. In this case where the departure time is not registered, the processing unit 222 may calculate the stay time of vehicle TA-1 at berth PA-1 of "1:10" by subtracting the arrival time "7:10" from the current time "8:20".

[0142] The processing unit 222 judges whether the stay time of the vehicle in the berths PA-1 to PA-n and PB-1 to PB-n exceeds a threshold value (third threshold value) (S513). If the stay time of the vehicle in each of the berths PA-1 to PA-n and PB-1 to PB-n is equal to or less than the threshold value (NO in S513), the stay time related processing ends.

[0143] On the other hand, when at least one of the vehicle's stay times in the berths PA-1 to PA-n, PB-1 to PB-n exceeds the threshold value ("YES" in S513), the transmission control unit 223 controls the communication unit 240 so that a notification indicating that the vehicle's stay time in the berth exceeds the threshold value is transmitted as a stay time exceeded notification by the communication unit 240 to the manager terminal 60 and to the driver terminal 70 of the driver of the vehicle whose stay time in the berth exceeds the threshold value (S514).

[0144] For example, if the threshold is one hour, referring to the stay time information 233 (FIG. 8), the stay time "1:10" of the vehicle TA-1 at berth PA-1 and time "August 20, 7:00-8:00" exceeds the threshold of one hour. Therefore, the transmission control unit 223 may control the communication unit 240 so that a notification indicating that the stay time of the vehicle TA-1 in berth PA-1 exceeds the threshold is transmitted as a stay time excess notification to each of the manager terminal 60 of the base A and the driver terminal 70 of the driver of the vehicle TA-1.

[0145] On the other hand, referring to the stay time information 233 (FIG. 8), the stay time "0:45" of the vehicle TA-n at the berth PA-n and the time "August 20, 7:00-8:00" is less than the threshold value of 1 hour. Therefore, a notification indicating that the stay time of the vehicle TA-n at the berth PA-n exceeds the threshold value does not need to be sent.

[0146] Furthermore, the stay time "1:20" of the vehicle TB-1 at berth PB-n and time "August 20, 7:00-8:00" exceeds the threshold value of 1 hour. Therefore, the transmission control unit 223 may control the communication unit 240 so that a notification indicating that the stay time of the vehicle TB-n at berth PB-n exceeds the threshold value is transmitted as a stay time exceeded notification to each of the manager terminal 60 at base B and the driver terminal 70 of the driver of the vehicle TB-n.

[0147] An example of the stay time related process executed by the server 20 according to the embodiment of the present disclosure has been described above.

[0148] <2. Hardware configuration example> Next, a hardware configuration example of an information processing device 900 as an example of the server 20 according to an embodiment of the present disclosure will be described with reference to Fig. 17. Fig. 17 is a block diagram showing a hardware configuration example of the information processing device 900. Note that the server 20 does not necessarily have to have all of the hardware configuration shown in Fig. 17, and some of the hardware configuration shown in Fig. 17 may not be present in the server 20.

[0149] 17, the information processing device 900 includes a CPU (Central Processing unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903. The information processing device 900 may also include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, and a communication device 925. The information processing device 900 may have a processing circuit such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit) instead of or in addition to the CPU 901.

[0150] The CPU 901 functions as an arithmetic processing device and a control device, and controls all or part of the operations in the information processing device 900 according to various programs recorded in the ROM 902, the RAM 903, the storage device 919, or the removable recording medium 927. The ROM 902 stores programs and arithmetic parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901 and parameters that change appropriately during the execution. The CPU 901, the ROM 902, and the RAM 903 are connected to each other by a host bus 907 that is constituted by an internal bus such as a CPU bus. Furthermore, the host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909.

[0151] The input device 915 is a device operated by a user, such as a button. The input device 915 may include a mouse, a keyboard, a touch panel, a switch, a lever, and the like. The input device 915 may also include a microphone that detects the user's voice. The input device 915 may be, for example, a remote control device that uses infrared rays or other radio waves, or an external connection device 929 such as a mobile phone that supports the operation of the information processing device 900. The input device 915 includes an input control circuit that generates an input signal based on information input by the user and outputs it to the CPU 901. The user inputs various data to the information processing device 900 and instructs processing operations by operating this input device 915. In addition, the imaging device 933 described later may also function as an input device by imaging the movement of the user's hand, the user's fingers, and the like. At this time, the pointing position may be determined according to the movement of the hand and the direction of the finger.

[0152] The output device 917 is configured with a device capable of visually or audibly notifying the user of acquired information. The output device 917 may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, or a sound output device such as a speaker or a headphone. The output device 917 may also include a PDP (Plasma Display Panel), a projector, a hologram, a printer device, or the like. The output device 917 outputs the results obtained by the processing of the information processing device 900 as a video such as text or an image, or as a sound such as voice or audio. The output device 917 may also include a light for brightening the surroundings.

[0153] The storage device 919 is a data storage device configured as an example of a storage unit of the information processing device 900. The storage device 919 is configured, for example, with a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. This storage device 919 stores programs and various data executed by the CPU 901, and various data acquired from the outside.

[0154] The drive 921 is a reader / writer for a removable recording medium 927 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is built into or externally attached to the information processing device 900. The drive 921 reads out information recorded on the attached removable recording medium 927 and outputs the information to the RAM 903. The drive 921 also writes information onto the attached removable recording medium 927.

[0155] The connection port 923 is a port for directly connecting a device to the information processing device 900. The connection port 923 may be, for example, a Universal Serial Bus (USB) port, an IEEE 1394 port, a Small Computer System Interface (SCSI) port, or the like. The connection port 923 may also be an RS-232C port, an optical audio terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) port, or the like. By connecting an external connection device 929 to the connection port 923, various types of data may be exchanged between the information processing device 900 and the external connection device 929.

[0156] The communication device 925 is, for example, a communication interface configured with a communication device for connecting to the network 931. The communication device 925 may be, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 925 may also be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various communications. The communication device 925 transmits and receives signals, for example, between the Internet and other communication devices using a predetermined protocol such as TCP / IP. The network 931 connected to the communication device 925 is a network connected by wire or wirelessly, for example, the Internet, a home LAN, infrared communication, radio wave communication, or satellite communication.

[0157] <3. Summary> According to an embodiment of the present disclosure, a server 20 is provided that includes a processing unit 222 that recognizes the entry of a vehicle into a berth or the exit of a vehicle from a berth based on the size of the license plate recognized from each of a plurality of frames. With this configuration, the entry of a vehicle into a berth or the exit of a vehicle from a berth can be recognized with high accuracy.

[0158] Although the preferred embodiment of the present disclosure has been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

[0159] For example, in the warehousing-related process S20 (FIG. 13), an example has been described in which, when a berth where a vehicle has been stored exists, the status indicates that the vehicle has not yet arrived ("YES" in S212), and the vehicle number match is successful ("YES" in S214), the status is updated (S216) and the arrival time is registered (S217). However, there may be cases where the reservation information 231 does not include a status. Or, there may be cases where the reservation information 231 does not include a vehicle number.

[0160] For example, when there is no status in the reservation information 231, the operation of determining whether the status indicates that the vehicle has not yet arrived (S212) may be omitted. Alternatively, when there is no vehicle number in the reservation information 231, the operation of determining whether the vehicle number matching has been successful (S214) may be omitted.

[0161] Similarly, in the out-of-stock related processing S30 (Figure 14), an example has been described in which, when a berth exists from which the vehicle has been out-stocked, if the status indicates arrival ("YES" in S312) and if the vehicle number matching is successful ("YES" in S314), the status is updated (S316) and the departure time is registered (S317).

[0162] However, when there is no status in the reservation information 231, the operation of determining whether the status indicates arrival (S312) may be omitted. Alternatively, when there is no vehicle number in the reservation information 231, the operation of determining whether matching of the vehicle number was successful (S314) may be omitted.

[0163] Furthermore, server 20 may appropriately adjust parameters of camera 10. For example, when processing unit 222 detects that the brightness of a frame output from camera 10 is lower than a threshold value, or when backlighting is detected from a frame output from camera 10, transmission control unit 223 may issue an instruction to CIS signal processing unit 120 to increase the exposure time of image sensor 110. CIS signal processing unit 120 may increase the exposure time of image sensor 110 in accordance with such an instruction.

[0164] Furthermore, the processing unit 222 may retrain the recognition model 140 based on the frames output from the camera 10. For example, the processing unit 222 may assume a case where the detection rate of the license plate based on the frames output from the camera 10 falls outside a predetermined range. In such a case, the processing unit 222 may generate frames for retraining using a simulation technique, and retrain the recognition model 140 using the generated frames for retraining.

[0165] In addition, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to a person skilled in the art from the description of this specification, in addition to or in place of the above effects.

[0166] Note that the following configurations also fall within the technical scope of the present disclosure. (1) A processing unit is provided for recognizing the entry of a vehicle into a parking space associated with the camera or the exit of the vehicle from the parking space based on the size of a license plate in each of a plurality of images obtained by the camera. Information processing device. (2) When the processing unit recognizes the entry of the vehicle into the parking space or the exit of the vehicle from the parking space, the processing unit acquires the vehicle number of the vehicle output based on the plurality of images. The information processing device according to (1). (3) The processing unit includes: A comparison is made between the acquired vehicle number and a vehicle number registered in the reservation information for the parking space. The information processing device according to (2). (4) The information processing device includes: a transmission control unit that controls transmission of a notification indicating the failure of the matching based on the failure of the matching; The information processing device according to (3). (5) When the processing unit recognizes the entry of the vehicle into the parking space, the processing unit registers a current time as an arrival time of the vehicle into the parking space based on the success of the matching. The information processing device according to (3). (6) When the processing unit recognizes the entry of the vehicle into the parking space, the processing unit determines whether or not a status of the parking space indicates "not arrived yet." The information processing device according to (1). (7) The processing unit registers a current time as an arrival time of the vehicle at the parking space based on the determination that the status of the parking space indicates not yet arrived. The information processing device according to (6). (8) The information processing device includes: a transmission control unit that controls transmission of a notification indicating an abnormality in the status based on the determination that the status does not indicate non-arrival; The information processing device according to (6). (9) When the processing unit recognizes the exit of the vehicle from the parking space, the processing unit registers a current time as a departure time of the vehicle from the parking space based on the success of the matching. The information processing device according to (5). (10) When the processing unit recognizes the exit of the vehicle from the parking space, the processing unit determines whether a status of the parking space indicates an arrival. The information processing device according to (7). (11) The processing unit registers a current time as a departure time of the vehicle from the parking space based on the determination that the status of the parking space indicates arrival. The information processing device according to (10). (12) The processing unit calculates a stay time of the vehicle in the parking space by subtracting an arrival time of the vehicle at the parking space from a current time or a departure time of the vehicle from the parking space. The information processing device according to (9) or (11). (13) The processing unit determines whether the stay time exceeds a third threshold. The information processing device according to (12). (14) When the processing unit recognizes the departure of the vehicle from the parking space, the processing unit registers the recognized vehicle number in reservation information for an unreserved parking space existing at a base to which the vehicle is to be moved, for a time period after the departure time of the parking space. The information processing device according to (2). (15) the plurality of images includes a first image and a second image that is later than the first image; The processing unit recognizes the entry of the vehicle into the parking space based on the satisfaction of a condition that an event has been detected at least once in which a second size of the license plate recognized from the second image is larger than a first size of the license plate recognized from the first image by a first threshold or more. The information processing device according to (1). (16) The processing unit recognizes the entry of the vehicle into the parking space when the condition is satisfied and when the time during which the license plate is continuously recognized is equal to or longer than a predetermined first time. The information processing device according to (15). (17) The processing unit recognizes the entry of the vehicle into the parking space when the condition is satisfied and when a time during which the size of the license plate or the position of the license plate area does not change becomes equal to or exceeds a predetermined second time. The information processing device according to (15). (18) The processing unit recognizes the entry of the vehicle into the parking space when the condition is satisfied and when the time during which the license plate is not recognized continues for a predetermined third time period or more. The information processing device according to (15). (19) the plurality of images includes a third image and a fourth image that is later than the third image; The processing unit recognizes the exit of the vehicle from the parking space based on a condition that a fourth size of the license plate recognized from the fourth image is smaller than a third size of the license plate recognized from the third image by a second threshold or more. The information processing device according to (1). (20) The camera is provided behind or diagonally behind the vehicle parked in the parking space. The information processing device according to (1). (twenty one) and a processor recognizes an entry of a vehicle into a parking space associated with the camera or an exit of the vehicle from the parking space based on a size of the license plate in each of a plurality of images acquired by the camera. Information processing methods. (twenty two) A camera for capturing a plurality of images; An information processing device including a processing unit that recognizes the entry of a vehicle into a parking space associated with the camera or the exit of the vehicle from the parking space based on the size of the license plate in each of the plurality of images recognized from the plurality of images; An information processing system having the above configuration. [Explanation of symbols]

[0167] 1. Information Processing Systems 10. Camera 110 Image Sensor 120 CIS signal processing section 130 ISP Department 140 Recognition Model 20 Servers 220 Control section 221 Acquisition Department 222 Processing section 223 Transmission control section 230 Storage section 231 Reservation Information 232 Contact Information 233 Stay Time Information 240 Communications Department 30 Network 60 Administrator terminal 70 Driver Terminal

Claims

1. A processing unit is provided for recognizing the entry of a vehicle into a parking space associated with the camera or the exit of the vehicle from the parking space based on the size of a license plate in each of a plurality of images obtained by the camera. Information processing device.

2. When the processing unit recognizes the entry of the vehicle into the parking space or the exit of the vehicle from the parking space, the processing unit acquires the vehicle number of the vehicle output based on the plurality of images. The information processing device according to claim 1 .

3. The processing unit includes: A comparison is made between the acquired vehicle number and a vehicle number registered in the reservation information for the parking space. The information processing device according to claim 2 .

4. The information processing device includes: a transmission control unit that controls transmission of a notification indicating the failure of the matching based on the failure of the matching; The information processing device according to claim 3 .

5. When the processing unit recognizes the entry of the vehicle into the parking space, the processing unit registers a current time as an arrival time of the vehicle into the parking space based on the success of the matching. The information processing device according to claim 3 .

6. When the processing unit recognizes the entry of the vehicle into the parking space, the processing unit determines whether or not a status of the parking space indicates "not arrived yet." The information processing device according to claim 1 .

7. The processing unit registers a current time as an arrival time of the vehicle at the parking space based on the determination that the status of the parking space indicates not yet arrived. The information processing device according to claim 6.

8. The information processing device includes: a transmission control unit that controls transmission of a notification indicating an abnormality in the status based on the determination that the status does not indicate non-arrival; The information processing device according to claim 6.

9. When the processing unit recognizes the exit of the vehicle from the parking space, the processing unit registers a current time as a departure time of the vehicle from the parking space based on the success of the matching. The information processing device according to claim 5 .

10. When the processing unit recognizes the exit of the vehicle from the parking space, the processing unit determines whether a status of the parking space indicates an arrival. The information processing device according to claim 7.

11. The processing unit registers a current time as a departure time of the vehicle from the parking space based on the determination that the status of the parking space indicates arrival. The information processing device according to claim 10.

12. The processing unit calculates a stay time of the vehicle in the parking space by subtracting an arrival time of the vehicle at the parking space from a current time or a departure time of the vehicle from the parking space. The information processing device according to claim 9.

13. The processing unit determines whether the stay time exceeds a third threshold. The information processing device according to claim 12.

14. When the processing unit recognizes the departure of the vehicle from the parking space, the processing unit registers the recognized vehicle number in reservation information for an unreserved parking space existing at a base to which the vehicle is to be moved, for a time period after the departure time of the parking space. The information processing device according to claim 2 .

15. the plurality of images includes a first image and a second image that is later than the first image; The processing unit recognizes the entry of the vehicle into the parking space based on the satisfaction of a condition that an event has been detected at least once in which a second size of the license plate recognized from the second image is larger than a first size of the license plate recognized from the first image by a first threshold value or more. The information processing device according to claim 1 .

16. The processing unit recognizes the entry of the vehicle into the parking space when the condition is satisfied and when a time during which the license plate is continuously recognized is equal to or longer than a predetermined first time. The information processing device according to claim 15.

17. the plurality of images includes a third image and a fourth image that is later than the third image, The processing unit recognizes the exit of the vehicle from the parking space based on the satisfaction of a condition that an event has been detected at least once in which a fourth size of the license plate recognized from the fourth image is smaller than a third size of the license plate recognized from the third image by a second threshold or more. The information processing device according to claim 1 .

18. The camera is provided behind or diagonally behind the vehicle parked in the parking space. The information processing device according to claim 1 .

19. and a processor recognizes an entry of a vehicle into a parking space associated with the camera or an exit of the vehicle from the parking space based on a size of the license plate in each of a plurality of images acquired by the camera. Information processing methods.

20. A camera for capturing a plurality of images; An information processing device including a processing unit that recognizes the entry of a vehicle into a parking space associated with the camera or the exit of the vehicle from the parking space based on the size of the license plate in each of the plurality of images recognized from the plurality of images; An information processing system having the above configuration.

Images