Fueling system, fueling method, and fueling program

Abstract

This enables proper fuel supply control. [Solution] The refueling system includes: a first image acquisition unit that acquires first image information of at least a portion of a first vehicle parked in a first refueling lane among a plurality of refueling lanes; a second image acquisition unit that acquires second image information of at least a portion of the first vehicle in a different shooting range than the first image information; a first determination unit that makes a determination regarding refueling in the first refueling lane based on the first image information; a second determination unit that makes a determination regarding refueling the first vehicle in the first refueling lane based on the second image information; and an execution information output unit that outputs first refueling execution information regarding the execution of refueling the first vehicle in the first refueling lane based on the result of the determination by the first determination unit and the result of the determination by the second determination unit.

Description

Technical Field

[0001] The present invention relates to a fuel supply system, a fuel supply method, and a fuel supply program.

Background Art

[0002] In self-service gasoline stations or service stations (hereinafter collectively referred to as "SS"), staff with the qualification of a dangerous goods handler are required to constantly monitor to prevent dangerous acts such as smoking by users and illegal acts such as fueling portable cans other than vehicles. For example, it is common to have a mechanism in which staff monitor in real time on a monitor from inside a building or the like an image captured by a surveillance camera near a fuel supply device (meter), and the user can fuel only when fueling is permitted.

[0003] In recent years, for the purpose of reducing the burden on staff who monitor fuel supply devices or eliminating the need to constantly place such staff at SS, the introduction of a self-service fuel supply system that monitors dangerous acts of users and determines whether to permit fueling has been considered. For example, Patent Document 1 discloses a system that determines whether a predetermined condition for permitting fueling by a fuel supply device is satisfied based on the analysis result of the acquired video and permits fueling by the fuel supply device.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In such a decision, for example, it is conceivable that the fuel nozzle would be detected and that the fuel nozzle being correctly inserted into the fuel filler opening would be a condition for permitting refueling. However, depending on the user's position and posture, the fuel nozzle may not be detected, or it may not be possible to detect that the fuel nozzle is correctly inserted into the fuel filler opening, which could result in refueling not starting even if no dangerous behavior is being committed.

[0006] This invention has been made in view of the above-mentioned problems, and aims to enable appropriate fuel supply control. [Means for solving the problem]

[0007] A refueling system according to one aspect of the present invention includes: a first image acquisition unit that acquires first image information of at least a portion of a first vehicle parked in a first refueling lane among a plurality of refueling lanes; a second image acquisition unit that acquires second image information of at least a portion of the first vehicle in a different shooting range than the first image information; a first determination unit that makes a determination regarding refueling in the first refueling lane based on the first image information; a second determination unit that makes a determination regarding refueling the first vehicle in the first refueling lane based on the second image information; and an execution information output unit that outputs first refueling execution information regarding the execution of refueling the first vehicle in the first refueling lane based on the result of the determination by the first determination unit and the result of the determination by the second determination unit. [Effects of the Invention]

[0008] According to the present invention, appropriate fuel supply control becomes possible. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an overview of an example configuration of a fueling system 1, which is one embodiment of the present invention. [Figure 2] This figure shows an example of the equipment layout in SS2. [Figure 3] This is a flowchart illustrating an example of the process in refueling system 1. [Figure 4]This is a flowchart illustrating an example of the process in refueling system 1. [Modes for carrying out the invention]

[0010] Figure 1 is a diagram illustrating an overview of an example configuration of a refueling system 1, which is one embodiment of the present invention. In this embodiment, the refueling system 1 may include an information processing device 10, a refueling device 20, and a camera 30 at the SS2, and may be connected via a network to a terminal 41 installed in a monitoring room 40, if necessary. The SS2 is, for example, a facility where vehicle users refuel their vehicles, and may be a so-called self-service gas station.

[0011] Here, the refueling device 20 is a device that receives information regarding the type of fuel to be supplied, such as premium gasoline or regular gasoline, and the amount to be supplied, based on the user's operation, and then refuels the user's vehicle. The refueling device 20 may also be called a metering machine. The refueling device 20 may, for example, include an input / output device and refueling nozzles for supplying premium gasoline, regular gasoline, or diesel fuel. Examples of input / output devices include image input / output devices such as touch panels that receive user operations and present information to the user, and audio input / output devices such as microphones that collect the user's voice and speakers that convey information to the user by voice.

[0012] For example, a user can refuel by inputting information about the type and amount of fuel to the refueling device 20, removing the refueling nozzle from the nozzle holder, inserting the refueling nozzle into the vehicle's fuel filler, and pulling the lever on the refueling nozzle. Alternatively, the refueling device 20 may, for example, receive refueling start information from the execution information output unit 15 of the information processing device 10 based on the determination results of the first determination unit 13 and the second determination unit 14, and then, when the user pulls the lever on the refueling nozzle that has been removed from the nozzle holder, release high-octane gasoline or the like from the tip of the nozzle. This series of actions performed by the user to refuel is also called the "refueling process."

[0013] The fueling device 20 can stop refueling when it reaches a predetermined amount of fuel specified by the user, or when it detects that the vehicle tank is full using a sensor at the tip of the fueling nozzle or the like. Separately, the execution information output unit 15 can also stop refueling when it receives fueling stop information from the execution information output unit 15, based on the results of the determinations made by the first determination unit 13 and the second determination unit 14. At this time, the input / output device of the fueling device 20 may output information indicating the reason for stopping or the conditions for restarting refueling to inform the user.

[0014] Camera 30 photographs at least a portion of a user's vehicle parked in a refueling lane. In photographing at least a portion of the vehicle, camera 30 also photographs the user's actions and circumstances, the vehicle's condition, the condition of the refueling lane, etc. Multiple cameras 30 are installed in SS2. Each of the multiple cameras 30 may be installed to photograph at least a portion of the vehicles parked in each of the multiple refueling lanes.

[0015] The monitoring room 40 is equipped with a terminal 41 for use by a monitor who monitors refueling at the refueling equipment 20. The monitor monitors SS2 from the monitoring room, for example, via the terminal 41. The monitoring room 40 may be located within the premises of SS2, or it may be located at a different location (remote location) from SS2. The monitor can use the terminal 41 to operate the refueling equipment 20 and perform operations to permit or deny refueling in the refueling lane.

[0016] An example of the equipment layout in SS2 will be explained using Figure 2. Figure 2 is a diagram showing an example of the equipment layout in SS2. SS2 has a plurality of refueling lanes, including at least a first refueling lane 210 and a second refueling lane 220. Each refueling lane is equipped with, for example, a refueling device 20 (e.g., a first refueling device 211, a second refueling device 221) and a camera 30 (e.g., a first camera 212, a second camera 222). Vehicles to be refueled (e.g., a first vehicle 213 and a second vehicle 223) can stop in each refueling lane.

[0017] The first camera 212 captures at least a portion of the first vehicle 213, for example, the first surface 214 of the first vehicle 213, and at least a portion of the second vehicle 223, for example, the third surface 224 of the second vehicle, and generates first image information. The second camera 222 captures at least a portion of the first vehicle 213, for example, the second surface 215 of the first vehicle 213, and at least a portion of the second vehicle 223, for example, the fourth surface 225 of the second vehicle, and generates second image information. Thus, the second image information is image information in which at least a portion of the first vehicle 213 is captured in a different shooting range than the first image information.

[0018] On the other hand, the first camera 212 cannot photograph parts that are in the shadow of the first vehicle 213 and the second vehicle 223, for example, the second face 215 of the first vehicle 213 and the fourth face 225 of the second vehicle 223. Also, the second camera 222 cannot photograph parts that are in the shadow of the first vehicle 213 and the second vehicle 223, for example, the first face 214 of the first vehicle 213 and the third face 224 of the second vehicle 223.

[0019] When granting permission to refuel in refueling system 1, refueling is permitted only if it is confirmed that refueling the vehicle is safe, based on, for example, the condition of the refueling device 20 installed in the refueling lane, the condition of the surrounding environment of the refueling lane, or the behavior of the user using the refueling lane. This confirmation includes automatic confirmation based on the results of analyzing image information, manual confirmation by a supervisor, or a combination of such automatic and manual confirmation. Here, the condition of the refueling device 20 includes, for example, whether the refueling nozzle is securely inserted into the vehicle's fuel filler. The condition of the surrounding environment of the refueling lane includes, for example, whether a fuel can is placed there or whether there are any sources of ignition (cigarettes, etc.). The behavior of the user using the refueling lane includes, for example, whether the user is behaving inappropriately, in particular, whether they are holding a fuel can or whether they are carrying any sources of ignition (cigarettes, etc.).

[0020] In this regard, when a fuel filler is provided on one side of the vehicle, depending on the positional relationship between the surface of the first vehicle 213 where the fuel filler is provided and the first fuel supply device 211, there may be a case where the first camera 212 cannot capture the fuel filler of the first vehicle 213. In such a case, since it is impossible to confirm whether the fuel nozzle is securely inserted into the fuel filler, the first determination unit 13 described later cannot perform determination regarding fuel supply in the first fuel supply lane. Also, the environmental conditions in the area shaded by the first vehicle 213 cannot be captured by the first camera 212. Furthermore, when the user is in the shade of the first vehicle 213, the first camera 212 cannot capture the actions of the user. In these cases, the first determination unit 13 described later cannot perform determination regarding fuel supply to the first vehicle 213 in the first fuel supply lane 210 based on the first image information (for example, the image information captured by the first camera 212). Therefore, the second determination unit 14 described later performs determination regarding fuel supply to the first vehicle 213 in the first fuel supply lane 210 based on the second image information (for example, the image information captured by the second camera 222). Thereby, even when the fuel supply system 1 cannot perform determination regarding fuel supply to the first vehicle 213 in the first fuel supply lane 210 based on the first image information (for example, the image information captured by the first camera 212), it can perform determination regarding fuel supply to the first vehicle 213 in the first fuel supply lane 210 based on the second image information (for example, the image information captured by the second camera 222).

[0021] Note that in FIG. 2, the first fuel supply lane 210 and the second fuel supply lane 220 are shown as fuel supply lanes adjacent to each other, but the first fuel supply lane 210 and the second fuel supply lane 220 may be non - adjacent fuel supply lanes.

[0022] The information processing apparatus 10 may be implemented by, for example, a dedicated or general-purpose information processing terminal such as a PC, and the CPU (Central Processing Unit) (not shown) executes middleware such as an OS (Operating System) and a DBMS (DataBase Management System) developed onto a memory from a recording device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and software operating thereon, so as to realize various functions.

[0023] As shown in FIG. 1, this information processing apparatus 10 may include a first image acquisition unit 11, a second image acquisition unit 12, a first determination unit 13, a second determination unit 14, an execution information output unit 15, and a notification unit 16. Each of these units is realized, for example, by using a storage area such as a memory or by the processor executing a program stored in the storage area.

[0024] The first image acquisition unit 11 acquires first image information in which at least a part of the first vehicle 213 parked in the first fueling lane 210 among a plurality of fueling lanes is photographed.

[0025] The first image information includes, for example, image information in which the first surface 214 of the first vehicle 213 is photographed.

[0026] The image information may be a moving image or a still image. For example, the first image acquisition unit 11 may extract image data within a necessary range from a recorder that constantly records image data in cooperation with the camera 30, triggered by the occurrence of a predetermined event such as the arrival of a user, or may directly acquire the image data sent from the camera 30. Further, the first image acquisition unit 11 may have a function of dynamically changing the orientation, shooting magnification, shooting range, etc. of the camera 30 by the operation of a staff such as a supervisor.

[0027] The second image acquisition unit 12 acquires second image information in which at least a part of the first vehicle 213 is photographed in a shooting range different from the first image information.

[0028] The second image information includes, for example, image information of a second surface 215 of the first vehicle 213 that is different from the first surface 214.

[0029] Furthermore, the second image information may include, for example, image information of a second refueling lane 220 that is different from the first refueling lane 210 (for example, a second refueling lane 220 adjacent to the first refueling lane 210). In this case, the second image information may also include image information of at least a portion of the second vehicle 223 parked in the second refueling lane 220.

[0030] The position in which at least a portion of the first vehicle 213 is photographed in the first image information may be further in the center of the image than the position in which at least a portion of the first vehicle 213 is photographed in the second image information. In other words, the first image information may be, for example, image information taken by a first camera 212 installed to photograph the first vehicle 213 in the first refueling lane. In this case, the second image information may be image information taken by a second camera 222 installed to photograph the second vehicle 223 in the second refueling lane, or image information taken by a second camera 222 installed to photograph areas other than the refueling lanes (for example, the entrance / exit of SS2 or the exterior of the monitoring room).

[0031] Furthermore, the second image information may be image information in which the second refueling lane 220 is positioned more towards the center of the image than the first refueling lane 210. In other words, the second image information may be image information taken by a second camera 222 that is installed to photograph the second vehicle 223 in the second refueling lane.

[0032] Thus, in relation to the first refueling lane 210, for example, the first image information is the main image used when making decisions regarding refueling, and the second image information is the sub-image. Similarly, in relation to the second refueling lane 220, for example, the first image information is the sub-image used when making decisions regarding refueling, and the second image information is the main image.

[0033] The first determination unit 13 makes a determination (first determination) regarding refueling in the first refueling lane 210 based on the first image information and generates first determination result information regarding the result of the determination. The result of the determination may be, for example, refueling permitted, refueling not permitted, or determination impossible.

[0034] The first determination unit 13 can make a determination regarding refueling the first vehicle 213 in the first refueling lane 210 based on, for example, the state of the first refueling device 211 installed in the first refueling lane 210, the state of the surrounding environment of the first refueling lane 210, or the actions of users using the first refueling lane 210, as shown in the first image information. Specifically, the first determination unit 13 may, for example, output a determination result of refueling permission if it can be confirmed that the refueling nozzle is securely inserted, or output a determination result of refueling denial if it can be confirmed that the refueling nozzle is not securely inserted, or output a determination result of inability to determine if it cannot be confirmed that the refueling nozzle is securely inserted. Furthermore, the first determination unit 13 may output a determination result of refueling permission if it is confirmed that there are no portable fuel cans or open flames in the vicinity, or it may output a determination result of refueling denial if it is confirmed that there are no portable fuel cans or open flames in the vicinity, or it may output a determination result of "unable to determine" if it is not confirmed that there are no portable fuel cans or open flames in the vicinity. Furthermore, the first determination unit 13 may output a determination result of refueling permission if it is confirmed that the user is not behaving inappropriately, or it may output a determination result of refueling denial if it is confirmed that the user is behaving inappropriately, or it may output a determination result of "unable to determine" if it is not confirmed whether or not the user is behaving inappropriately.

[0035] When the first determination unit 13 outputs a determination result of "undeterminable," it can output determination-undeterminable information indicating that it is not possible to make a determination regarding refueling the first vehicle 213 in the first refueling lane 210. For example, if the state of the first refueling device 211 is not captured in the first image information, and especially if the refueling nozzle of the first refueling device 211 is not captured, the first determination unit 13 can output determination-undeterminable information indicating that it is not possible to make a determination regarding refueling the first vehicle 213 in the first refueling lane 210.

[0036] The first determination unit 13 analyzes the first image information using an image determination model or algorithm to evaluate at least one of the following: the state of the first refueling device 211 installed in the first refueling lane 210, the state of the surrounding environment of the first refueling lane 210, or the behavior of users using the first refueling lane 210. Based on the results of this evaluation, it makes a determination regarding refueling the first vehicle 213 in the first refueling lane 210. The image determination model or algorithm used by the first determination unit 13 is not particularly limited, and known models can be used as appropriate.

[0037] Furthermore, the first determination unit 13 can make a determination (third determination) regarding refueling the second vehicle 223 in the second refueling lane 220 based on the second image information. The processing of the third determination based on the second image information by the first determination unit 13 is the same as the first determination processing based on the first image information.

[0038] The second determination unit 14 makes a determination (second determination) regarding refueling the first vehicle in the first refueling lane based on the second image information, and generates first determination result information regarding the result of the determination. The result of the determination may be, for example, refueling permitted, refueling not permitted, or determination impossible.

[0039] The second determination unit 14 can make a determination regarding refueling the first vehicle 213 in the first refueling lane 210 based on, for example, the state of the first refueling device 211 installed in the first refueling lane 210, the state of the surrounding environment of the first refueling lane 210, or the actions of users using the first refueling lane 210, as shown in the second image information. Specifically, the second determination unit 14 may, for example, output a determination result of refueling permission if it can be confirmed that the refueling nozzle is securely inserted, or output a determination result of refueling denial if it can be confirmed that the refueling nozzle is not securely inserted, or output a determination result of inability to determine if it cannot be confirmed that the refueling nozzle is securely inserted. Furthermore, the second determination unit 14 may output a determination result of refueling permission if it is confirmed that there are no portable fuel cans or open flames in the vicinity, or it may output a determination result of refueling denial if it is confirmed that there are no portable fuel cans or open flames in the vicinity, or it may output a determination result of "unable to determine" if it is not confirmed that there are no portable fuel cans or open flames in the vicinity. Furthermore, the second determination unit 14 may output a determination result of refueling permission if it is confirmed that the user is not behaving inappropriately, or it may output a determination result of refueling denial if it is confirmed that the user is behaving inappropriately, or it may output a determination result of "unable to determine" if it is not confirmed whether or not the user is behaving inappropriately.

[0040] When the second determination unit 14 outputs a determination result of "unable to determine," it can output determination information indicating that it is not possible to determine whether to refuel the first vehicle 213 in the first refueling lane 210. For example, if the state of the first refueling device 211 is not captured in the second image information, and especially if the refueling nozzle of the first refueling device 211 is not captured, the second determination unit 14 can output determination information indicating that it is not possible to determine whether to refuel the first vehicle 213 in the first refueling lane 210.

[0041] Specifically, the second determination unit 14 analyzes the second image information using an image determination model or algorithm to evaluate at least one of the following: the state of the first refueling device 211 installed in the first refueling lane 210, the state of the surrounding environment of the first refueling lane 210, or the behavior of users using the first refueling lane 210. Based on the results of this evaluation, it makes a determination regarding refueling the first vehicle 213 in the first refueling lane 210. The image determination model or algorithm used by the second determination unit 14 is not particularly limited, and known models can be used as appropriate.

[0042] The second determination unit 14 can make a determination regarding refueling the first vehicle 213 in the first refueling lane 210 based on the second image information, in response to the undeterminable information output by the first determination unit 13. As a result, the second determination unit 14 can perform the second determination process when the first determination unit 13 determines that a determination is not possible, thereby reducing the processing load.

[0043] Furthermore, the second determination unit 14 can make a determination (fourth determination) regarding refueling the first vehicle 213 in the first refueling lane 210 based on the first image information. The processing of the fourth determination based on the first image information by the second determination unit 14 is the same as the second determination processing based on the second image information.

[0044] The execution information output unit 15 outputs first refueling execution information relating to the execution of refueling the first vehicle 213 in the first refueling lane 210, based on at least one of the results of the first determination by the first determination unit 13 (i.e., first determination result information) and the results of the second determination by the second determination unit 14 (i.e., second determination result information).

[0045] Specifically, the execution information output unit 15 can output refueling start information as first refueling execution information, which is used to start refueling the first vehicle 213 in the first refueling lane 210, when the first determination unit 13 determines that refueling is permitted based on the first image information.

[0046] Furthermore, the execution information output unit 15 can output refueling start information as first refueling execution information to initiate refueling of the first vehicle 213 in the first refueling lane 210 when the second determination unit 14 determines that refueling is permitted based on the second image information. At this time, the execution information output unit 15 may also output refueling start information as first refueling execution information to initiate refueling of the first vehicle 213 in the first refueling lane 210 when the first determination unit 13 determines that a determination is not possible based on the first image information, and the second determination unit 14 determines that refueling is permitted based on the second image information. As a result, even when the refueling system 1 cannot make a determination regarding refueling of the first vehicle 213 in the first refueling lane 210 based on the first image information, it can make a determination regarding refueling of the first vehicle 213 in the first refueling lane 210 based on the second image information.

[0047] Furthermore, the execution information output unit 15 can output refueling stop information to stop refueling the first vehicle 213 in the first refueling lane 210 when the first determination unit 13 determines that refueling is not permitted based on the first image information. In this embodiment, stopping refueling includes transitioning from a refueling state to a non-refueling state, and maintaining the non-refueling state.

[0048] Furthermore, the execution information output unit 15 can output refueling stop information to stop refueling the first vehicle 213 in the first refueling lane 210 if the second determination unit 14 determines that refueling is not permitted based on the second image information. In this case, the execution information output unit 15 may also output refueling stop information to stop refueling the first vehicle 213 in the first refueling lane 210 if the first determination unit 13 determines that a determination is not possible based on the first image information, and the second determination unit 14 determines that refueling is not permitted based on the second image information. As a result, even if the refueling system 1 cannot make a determination regarding refueling the first vehicle 213 in the first refueling lane 210 based on the first image information, it can make a determination regarding refueling the first vehicle 213 in the first refueling lane 210 based on the second image information.

[0049] Furthermore, the execution information output unit 15 can output second refueling execution information regarding the execution of refueling the second vehicle 223 in the second refueling lane 220, based on the result of the third determination by the first determination unit 13 and the result of the fourth determination by the second determination unit 14. In this embodiment, the processing related to the second refueling execution information is the same as the processing related to the first refueling execution information.

[0050] The execution information output unit 15 may, for example, output the first refueling execution information to the terminal 41. In this case, the monitor using the terminal 41 will refer to the first refueling execution information through the terminal 41 and authorize the refueling device 20 to refuel the first vehicle 213 in the first refueling lane 210. The refueling device 20 will then begin refueling the first vehicle 213 in the first refueling lane 210.

[0051] Furthermore, the execution information output unit 15 may output, for example, refueling start information to initiate refueling as first refueling execution information to the refueling device 20. In this case, the refueling device 20 will, for example, start refueling the first vehicle in the first refueling lane based on the first refueling execution information.

[0052] Furthermore, the execution information output unit 15 may output refueling stop information to the refueling device 20 to stop refueling, based on the result of the first determination by the first determination unit 13 and the result of the second determination by the second determination unit 14. In this case, the execution information output unit 15 may output refueling stop information if the result of the first determination and the second determination do not indicate permission for refueling.

[0053] Alternatively, instead of the execution information output unit 15 automatically determining the reason for stopping and instructing the refueling device 20 to start or stop refueling, the execution information output unit 15 may transmit the determined reason for stopping to the terminal 41, and the monitor may manually instruct the refueling device 20 to start or stop refueling. Furthermore, if the rate of detection of dangerous behavior is high, the execution information output unit 15 may automatically instruct the refueling device 20 to start or stop refueling, and if the rate of detection of dangerous behavior is moderate and a decision of either permission or denial is possible, the monitor may manually instruct the refueling device 20 to start or stop refueling, thus appropriately combining automatic and manual determination. Here, if the rate of detection of dangerous behavior is high, for example, at least one of the results of the first determination and the second determination may be that refueling is not permitted. Also, if the rate of detection of dangerous behavior is moderate, the result of the first determination and the second determination may be that a determination cannot be made.

[0054] The notification unit 16 notifies the monitor of the first refueling lane 210 if the second determination unit 14 is unable to make a determination regarding refueling in the first refueling lane 210. At this time, the notification unit 16 may, for example, output notification information to the terminal 41 used by the monitor, indicating that the first determination unit 13 and the second determination unit 14 are unable to make a determination regarding refueling in the first refueling lane 210. This allows the monitor to understand that the first determination unit 13 and the second determination unit 14 are unable to make an automatic determination, and to check the status of the refueling lane, for example, through a monitor.

[0055] Furthermore, the notification unit 16 may notify the supervisor if the second determination unit 14 does not determine that refueling is permitted. This allows the supervisor to check the status of the refueling lane, for example, through a monitor.

[0056] Figure 3 is a flowchart illustrating an example of the process in refueling system 1. The flowchart shown in Figure 3 illustrates an example of the process related to refueling the first vehicle in the first refueling lane.

[0057] First, the first image acquisition unit 11 acquires first image information (S301). The first determination unit 13 makes a determination regarding refueling in the first refueling lane 210 based on the first image information (S302). If the first determination unit 13 determines that refueling is permitted (S302: YES), the execution information output unit 15 outputs first refueling execution information regarding the execution of refueling the first vehicle 213 in the first refueling lane 210 (S303).

[0058] If the first determination unit 13 does not determine that refueling is permitted (S302: NO), the second image acquisition unit 12 acquires second image information (S304). The second determination unit 14 makes a determination regarding refueling in the first refueling lane 210 based on the second image information (S305). If the second determination unit 14 determines that refueling is permitted (S305: YES), the execution information output unit 15 outputs first refueling execution information regarding the execution of refueling the first vehicle 213 in the first refueling lane 210 (S303).

[0059] If the second determination unit 14 does not determine that refueling is permitted (S305: NO), the notification unit 16 outputs notification information indicating that a determination regarding refueling in the first refueling lane 210 cannot be made (S306).

[0060] Figure 4 is a flowchart showing an example of the process in the refueling system 1. The flowchart shown in Figure 4 shows an example of the process related to refueling the second vehicle 223 in the second refueling lane 220.

[0061] First, the second image acquisition unit 12 acquires second image information (S401). The first determination unit 13 makes a determination regarding refueling in the second refueling lane 220 based on the second image information (S402). If the first determination unit 13 determines that refueling is permitted (S402: YES), the execution information output unit 15 outputs second refueling execution information regarding the execution of refueling the second vehicle 223 in the second refueling lane 220 (S403).

[0062] If the first determination unit 13 does not determine that refueling is permitted (S402: NO), the first image acquisition unit 11 acquires the first image information (S404). The second determination unit 14 makes a determination regarding refueling in the second refueling lane 220 based on the first image information (S405). If the second determination unit 14 determines that refueling is permitted (S405: YES), the execution information output unit 15 outputs second refueling execution information regarding the execution of refueling the second vehicle 223 in the second refueling lane 220 (S403).

[0063] If the second determination unit 14 does not determine that refueling is permitted (S405: NO), the notification unit 16 outputs notification information indicating that a determination regarding refueling in the second refueling lane 220 cannot be made (S406).

[0064] An embodiment of the present invention has been specifically described above. The refueling system 1 acquires first image information, acquires second image information, makes a first determination regarding refueling the first vehicle 213 in the first refueling lane 210 based on the first image information, makes a second determination regarding refueling the first vehicle 213 in the first refueling lane 210 based on the second image information, and outputs first refueling execution information based on the results of the first determination and the second determination. As a result, the refueling system 1 can, for example, make a determination regarding refueling even if it is not possible to make a determination regarding refueling based on the first image information alone, by further determining the second image information, thereby enabling appropriate refueling control.

[0065] Furthermore, the refueling system 1 can acquire first and second image information such that the position in the first image information where at least a portion of the first vehicle 213 is photographed is more central to the image than the position in the second image information where at least a portion of the first vehicle 213 is photographed. As a result, the refueling system 1 can make decisions regarding refueling based on the second image information, which is, so to speak, a sub-image in relation to the first refueling lane 210, and can perform appropriate refueling control.

[0066] Furthermore, the refueling system 1 can acquire first image information of the first surface 214 of the first vehicle 213 and second image information of the second surface 215 of the first vehicle, which is different from the first surface 214. As a result, the refueling system 1 can make decisions regarding refueling based on the second image information, which is, so to speak, a sub-image in relation to the first refueling lane 210, and can perform appropriate refueling control.

[0067] Furthermore, the refueling system 1 can acquire second image information of a second refueling lane 220, which is different from the first refueling lane 210.

[0068] Furthermore, the refueling system 1 acquires second image information in which at least a portion of the second vehicle 223 parked in the second refueling lane 220 is photographed, makes a third determination regarding refueling the second vehicle 223 in the second refueling lane 220 based on the second image information, makes a fourth determination regarding refueling the second vehicle 223 in the second refueling lane 220 based on the first image information, and outputs second refueling execution information regarding the execution of refueling the second vehicle 223 in the second refueling lane 220 based on the results of the third and fourth determinations. As a result, the refueling system 1 can make determinations regarding the execution of refueling in the second refueling lane 220.

[0069] Furthermore, the refueling system 1 can acquire second image information in which the second refueling lane 220 is captured more centrally than the first refueling lane 210. As a result, the refueling system 1 can make decisions regarding refueling based on the first image information, which is, so to speak, a sub-image in relation to the second refueling lane 220, thereby enabling appropriate refueling control.

[0070] Furthermore, the refueling system 1 can make a first determination regarding refueling the first vehicle 213 in the first refueling lane 210 based on at least one of the following in the first image information: the state of the first refueling device 211 installed in the first refueling lane 210, the state of the surrounding environment of the first refueling lane 210, and the actions of users using the first refueling lane 210. It can also make a second determination regarding refueling the first vehicle 213 in the first refueling lane 210 based on at least one of the following in the second image information: the state of the first refueling device 211 installed in the first refueling lane 210, the state of the surrounding environment of the first refueling lane 210, and the actions of users using the first refueling lane. As a result, the refueling system 1 can analyze the first and second image information to make a determination regarding the execution of refueling.

[0071] Furthermore, if the state of the first refueling device 211 is not captured in the first image information, the refueling system 1 outputs "cannot determine" information indicating that a first determination regarding refueling the first vehicle 213 in the first refueling lane 210 cannot be made. Based on the second image information, the system can then make a second determination regarding refueling the first vehicle 213 in the first refueling lane 210, in accordance with the "cannot determine" information. This allows the refueling system 1 to make a second determination based on the second image information when a first determination based on the first image information cannot be made, thereby reducing the processing load.

[0072] Furthermore, if the refueling system 1 cannot make a second determination regarding refueling in the first refueling lane 210, it can notify the monitor who is monitoring the first refueling lane 210. This allows the refueling system 1 to request confirmation from the monitor at an appropriate time.

[0073] It should be noted that this embodiment is provided to facilitate understanding of the present invention, and the present invention is not limited to the above-described embodiment, and can be modified in various ways without departing from its essence. Furthermore, the above-described embodiment is described in detail to make the present invention easy to understand, and is not necessarily limited to having all the configurations described. In addition, it is possible to add, delete, or replace some of the configurations of the above-described embodiment with other configurations.

[0074] Furthermore, each of the above configurations, functions, processing units, and processing means may be implemented in hardware, in whole or in part, for example, by designing them as integrated circuits. Alternatively, each of the above configurations, functions, and means may be implemented in software by having the processor interpret and execute programs that implement each function. Information such as programs, tables, and files that implement each function can be stored in memory, hard disks, SSDs, or other recording devices, or in recording media such as IC cards, SD cards, or DVDs.

[0075] Furthermore, in the diagrams above, the control lines and information lines shown are those deemed necessary for explanation and do not necessarily represent all control lines and information lines that would be present in the actual implementation. In reality, it can be assumed that almost all components are interconnected. [Industrial applicability]

[0076] This invention can be used in refueling systems at self-service gas stations and service stations. [Explanation of Symbols]

[0077] 1 Fueling system, 10 Information processing device, 11 First image acquisition unit, 12 Second image acquisition unit, 13 First determination unit, 14 Second determination unit, 15 Execution information output unit, 16 Notification unit, 20 Fueling device, 30 Camera, 40 Monitoring room, 41 Terminal, 210 First fueling lane, 211 First fueling device, 212 First camera, 213 First vehicle, 214 First view, 215 Second view, 220 Second fueling lane, 221 Second fueling device, 222 Second camera, 223 Second vehicle, 224 Third view, 225 Fourth view

Claims

1. A first image acquisition unit acquires first image information in which at least a portion of a first vehicle parked in the first refueling lane of multiple refueling lanes is photographed, A second image acquisition unit acquires second image information in which at least a part of the first vehicle is photographed in a different shooting range from the first image information, A first determination unit that makes a determination regarding refueling in the first refueling lane based on the first image information, A second determination unit performs a determination regarding refueling the first vehicle in the first refueling lane based on the second image information, An execution information output unit outputs first refueling execution information relating to the execution of refueling the first vehicle in the first refueling lane, based on the result of the determination by the first determination unit and the result of the determination by the second determination unit. A refueling system equipped with the following features.

2. The refueling system according to claim 1, wherein the position in which at least a portion of the first vehicle is photographed in the first image information is centered in the image, more so than the position in which at least a portion of the first vehicle is photographed in the second image information.

3. The first image information includes image information of the first surface of the first vehicle, The second image information includes image information of a second surface of the first vehicle that is different from the first surface. The refueling system according to claim 1.

4. The refueling system according to any one of claims 1 to 3, wherein the second image information includes image information of a second refueling lane different from the first refueling lane.

5. The second image information includes image information of at least a portion of the second vehicle parked in the second refueling lane, The first determination unit further makes a determination regarding refueling the second vehicle in the second refueling lane based on the second image information, The second determination unit further makes a determination regarding refueling the second vehicle in the second refueling lane based on the first image information, The execution information output unit further outputs second refueling execution information relating to the execution of refueling the second vehicle in the second refueling lane, based on the result of the determination by the first determination unit and the result of the determination by the second determination unit. The refueling system according to claim 4.

6. The refueling system according to claim 4, wherein the second image information includes image information in which the second refueling lane is positioned more centrally in the image than the first refueling lane.

7. The refueling system according to claim 4, wherein the second refueling lane is a refueling lane adjacent to the first refueling lane.

8. The first determination unit makes a determination regarding refueling the first vehicle in the first refueling lane based on at least one of the following in the first image information: the state of the first refueling device installed in the first refueling lane, the state of the surrounding environment of the first refueling lane, and the actions of users using the first refueling lane. The second determination unit makes a determination regarding refueling the first vehicle in the first refueling lane based on at least one of the following in the second image information: the state of the first refueling device installed in the first refueling lane, the state of the surrounding environment of the first refueling lane, and the actions of users using the first refueling lane. A refueling system according to any one of claims 1 to 3.

9. The first determination unit outputs determination impossible information indicating that if the state of the first refueling device is not captured in the first image information, a determination regarding refueling the first vehicle in the first refueling lane cannot be made. The second determination unit, in response to the information that a determination is not possible, makes a determination regarding the refueling of the first vehicle in the first refueling lane based on the second image information. The refueling system according to claim 8.

10. The refueling system according to any one of claims 1 to 3, further comprising a notification unit that notifies a monitor who monitors the first refueling lane when the second determination unit is unable to make a determination regarding refueling in the first refueling lane.

11. Computers At least a portion of the first vehicle parked in the first refueling lane of multiple refueling lanes is photographed and first image information is acquired. Second image information is obtained in which at least a part of the first vehicle is photographed in a different shooting range from the first image information. Based on the first image information, a determination is made regarding refueling in the first refueling lane. Based on the second image information, a determination is made regarding the refueling of the first vehicle in the first refueling lane. Based on the result of the determination based on the first image information and the result of the determination based on the second image information, first refueling execution information relating to the execution of refueling the first vehicle in the first refueling lane is output. Refueling method.

12. On the computer, A first image acquisition unit acquires first image information in which at least a portion of a first vehicle parked in the first refueling lane of multiple refueling lanes is photographed, A second image acquisition unit acquires second image information in which at least a part of the first vehicle is photographed in a different shooting range from the first image information, A first determination unit that makes a determination regarding refueling in the first refueling lane based on the first image information, A second determination unit performs a determination regarding refueling in the first refueling lane based on the second image information, An execution information output unit outputs first refueling execution information relating to the execution of refueling the first vehicle in the first refueling lane, based on the result of the determination by the first determination unit and the result of the determination by the second determination unit. A refueling program designed to achieve this.

Images