Fuel supply system, information processing device, fuel supply control method, and fuel supply control program

Abstract

To provide a fuel supply system capable of executing processing in accordance with a fuel supply scene.SOLUTION: A fuel supply system comprises a fuel dispenser and an information processing device. The information processing device determines the state of the fuel nozzle and the fuel supply state based on notifications from the fuel dispenser. If it is determined that fueling is not permitted based on a determination logic corresponding to whether the fuel supplying action by a fuel supplier is included in any one of a first section before the start of fueling, a second section during fueling, or a third section after the end of fueling, or based on a determination logic corresponding to whether it is included in any two consecutive sections, then fueling by the fuel dispenser is prohibited or stopped.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present disclosure relates to a fueling system, an information processing device, a fueling control method, and a fueling control program.

Background Art

[0002] As a fueling system for service stations, the development of a fueling system that monitors the fueling behavior of customers who are fuelers and determines whether to allow fueling by utilizing AI or the like is underway.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, the fueling scenes at service stations are diverse, and in the development of a fueling system, it is desirable to have a configuration in which the method for determining whether to allow fueling, the method for outputting the determination result, etc. are changed according to the fueling scene.

[0005] In one aspect, an object is to provide a fueling system capable of executing processing according to a fueling scene.

Means for Solving the Problems

[0006] According to one aspect, the fueling system is a fuel dispenser, an information processing device, and is a fueling system having wherein the information processing device determines the state of the fueling nozzle and the state of fueling based on a notification from the fuel dispenser, Based on the determination logic according to which the refueling operation of the refueler is included in any one of the first section before refueling starts, the second section during refueling, and the third section after refueling ends, or based on the determination logic according to which it is included in any two consecutive sections, if it is determined that refueling is not permitted, the refueling by the fuel dispenser is prohibited or stopped.

Advantages of the Invention

[0007] A fueling system capable of executing processing according to a fueling scene can be provided.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14A

Figure 14B

Figure 15A

Figure 15B

Figure 16A

Figure 16B

Figure 17A

Figure 17B

Figure 17C

Figure 17D

Figure 18A

Figure 18B

Figure 19

Figure 20A

Figure 20B

Figure 21A

Figure 21B

Figure 21C

Figure 21D

Figure 22A

Figure 22B

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28A

Figure 28B

Embodiments for Carrying Out the Invention

[0009] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] [First Embodiment] [Learning Process] First, the learning process for constructing the fueling system according to the first embodiment will be described. FIG. 1 is a diagram showing the learning process for constructing the fueling system according to the first embodiment. As shown in FIG. 1, when constructing the fueling system according to the first embodiment, image data collected at a plurality of service stations (for example, service stations 110, 111, ···) is used. Also, at one service station, when different types of fueling are possible, image data collected for each type of fueling is used. Here, the different types of fueling referred to here mean fuel oil for vehicles such as gasoline and light oil (hereinafter referred to as "fuel oil of the first fueling type") and fuel oil for uses other than vehicles such as kerosene (hereinafter referred to as "fuel oil of the second fueling type").

[0011] The example of FIG. 1 shows the case where service station 110 can fuel both the fuel oil of the first fueling type and the fuel oil of the second fueling type. For this reason, in the example of FIG. 1, image data obtained by photographing the fueling scene 110_1 for the first fueling type and image data obtained by photographing the fueling scene 110_2 for the second fueling type are collected.

[0012] The fueling scene 110_1 for the first fueling type includes the fueling act by a fueler (not shown). The camera 130_1 that captures the area including the first fueling machine 120_1 as a video captures the first fueling machine 120_1 and the fueling act by the fueler, and image data with a predetermined frame period is collected by the image data collection device 140.

[0013] As shown in FIG. 1, a fuel nozzle 121_1 is connected to the first fuel dispenser 120_1, and the fuel oil of the first fuel type stored in the tank 126_1 is configured to be supplied to the fuel nozzle 121_1 through the fuel supply pipe by the fuel pump 125_1. Further, as shown in FIG. 1, a fuel valve 124_1 is installed in the fuel supply pipe. Also, in the first fuel dispenser 120_1, as a notification device, a voice output device 122_1 and a fueling data display 123_1 (a device for displaying fueling data and messages from the information processing device 160) are installed.

[0014] Although not shown in FIG. 1, similar image data for the fueling scene 110_2 of the second fuel type is also collected by the image data collection device 140.

[0015] When collecting image data, the image data collection device 140 also collects a signal indicating the removal of the fuel nozzle 121_1 from the first fuel dispenser 120_1 and the return of the fuel nozzle 121_1 to the first fuel dispenser 120_1 (a signal indicating the state of the fuel nozzle).

[0016] The image data collected by the image data collection device 140 and the signal indicating the state of the fuel nozzle are transmitted to the learning device 150 via the network 180, and learning data is generated in the learning device 150. Note that the learning data is generated based on the image data collected at a plurality of service stations (for example, service stations 110, 111,...) and the signal indicating the state of the fuel nozzle. However, the learning data is generated separately for the first fuel type and the second fuel type.

[0017] The learning device 150 is installed at a location different from a plurality of service stations (for example, service stations 110, 111, ···) (for example, a server on the cloud). The learning device 150 uses the learning data for the first fuel type (referred to as the first learning data) to perform learning of the first model and generate the first learned model. Further, the learning device 150 uses the learning data for the second fuel type (referred to as the second learning data) to perform learning of the second model and generate the second learned model.

[0018] The first learned model and the second learned model generated by the learning device 150 are transmitted to a plurality of service stations (for example, service stations 110', 111', ···) via the network 180. Thereby, they are stored in the information processing device of each service station (for example, the information processing device 160 of the service station 110').

[0019] As shown in FIG. 1, the fueling system 110S' according to the first embodiment applied to the service station 110' includes, for the first fuel type, · a camera 130_1, · a first fuel dispenser 120_1, · a thermal sensor 170_1. The first fuel dispenser 120_1 is connected to a fuel nozzle 121_1 to which the fuel oil of the first fuel type stored in the tank 126_1 is supplied via a fuel supply pipe by a fuel supply pump 125_1. A fuel supply valve 124_1 is installed in the fuel supply pipe. Further, a voice output device 122_1 and a fueling data display 123_1 are installed in the first fuel dispenser 120_1 as notification devices.

[0020] Also, the fueling system 110S' according to the first embodiment applied to the service station 110' has a similar configuration for the second fuel type. However, when the measurement area of the thermal sensor 170_1 includes both the fueling scene 110_1 of the first fuel type and the fueling scene 110_2 of the second fuel type, the thermal sensor 170_1 may be used for both the first fuel type and the second fuel type.

[0021] Furthermore, as shown in FIG. 1, the fueling system 110S' according to the first embodiment applied to the service station 110' has an information processing device 160. The information processing device 160 is · Cameras for the first fuel type and the second fuel type (cameras 130_1, 130_2), · Fuel dispensers (first fuel dispenser 120_1, second fuel dispenser 120_2), · Thermal sensors (thermal sensors 170_1, 170_2), · Fuel pumps (fuel pumps 125_1, 125_2), and is communicably connected.

[0022] In the example of FIG. 1, the case where the fueling system 110S' according to the first embodiment is applied to the service station 110' is shown. However, when it is applied to other service stations (for example, service stations 111', ···), the configuration is the same.

[0023] <Description of the fueling scene> Next, a specific example of the fueling scene at the service station 110 will be described. FIGS. 2A and 2B are the first and second figures showing an example of the fueling scene at the service station.

[0024] Among them, FIG. 2A shows an example of the fueling scene for the first fuel type at the service station 110. As shown in FIG. 2A, the service station 110 includes a first area 210 where the vehicle 211 stops, and a second area 220 which is an area where the first fuel dispenser 120_1 for fueling the first fuel type is installed and where the fueling operation by the fueler 221 is performed.

[0025] In the case of the refueling scene shown in Fig. 2A, with the vehicle 211 stopped in the first area 210, the refueler 221 removes the fuel nozzle 121_1 from the nozzle hanger of the first fuel dispenser 120_1 and inserts the fuel nozzle 121_1 into the fuel inlet of the vehicle 211. Also, the refueler 221 starts refueling by operating the lever of the fuel nozzle 121_1 and continues refueling until the desired fuel quantity is reached. Then, when the desired fuel quantity is reached, the refueler 221 stops refueling by returning the lever of the fuel nozzle 121_1, removes the fuel nozzle 121_1 from the fuel inlet, and then returns it to the nozzle hanger of the first fuel dispenser 120_1 to end the refueling operation.

[0026] On the other hand, Fig. 2B shows an example of a refueling scene for the second fuel type at the service station 110. As shown in Fig. 2B, the service station 110 includes an area where the container 231 (poly tank) is placed and an area where the second fuel dispenser 120_2 for refueling the second fuel type is installed, and is an area where the refueling operation by the refueler 241 is performed.

[0027] In the case of the refueling scene shown in Fig. 2B, with the container 231 placed, the refueler 241 removes the fuel nozzle 121_2 from the nozzle hanger of the second fuel dispenser 120_2 and inserts the fuel nozzle 121_2 into the fuel inlet of the container 231. Also, the refueler 241 starts refueling by operating the lever of the fuel nozzle 121_2 and continues refueling until the desired fuel quantity is reached. Then, when the desired fuel quantity is reached, the refueler 241 stops refueling by returning the lever of the fuel nozzle 121_2, removes the fuel nozzle 121_2 from the fuel inlet, and then returns it to the nozzle hanger of the second fuel dispenser 120_2 to end the refueling operation.

[0028] <Process for generating learning data> Next, an overview of the process from when the camera 130_1 captures the area including the first area 210 and the second area 220 as a video until the learning device 150 generates the first learning data based on the video will be described. Fig. 3A is a diagram showing the flow of the process for generating the first learning data.

[0029] As shown in FIG. 3A, the image data collection device 140 collects image data of a predetermined frame period, which is captured as a video by the camera 130_1. Further, as shown in FIG. 3A, the image data collection device 140 collects a signal indicating the state of the fuel nozzle 121_1 from the first fuel dispenser 120_1.

[0030] The learning device 150 extracts, from the image data collected by the image data collection device 140, the image data during the period from when the fuel nozzle 121_1 is removed from the nozzle hanging of the first fuel dispenser 120_1 to when it is returned to the nozzle hanging.

[0031] Further, the learning device 150 performs image recognition processing on the extracted image data, and recognizes the recognition targets (such as vehicles, containers, fuelers, fuel nozzles, etc.) included in the extracted image data. Furthermore, the learning device 150 assigns annotations to the recognized recognition targets (such as vehicles, containers, fuelers, fuel nozzles, etc.).

[0032] Specifically, when a vehicle is recognized in the image data and is recognized within the first region 210 that is part of the image data, an annotation indicating that it is a vehicle is assigned to the bounding box of the vehicle.

[0033] Also, when a container is recognized in the image data, an annotation indicating that it is a container is assigned to the bounding box of the container.

[0034] Also, when a fueler is recognized in the image data, · the fueler is recognized within the second region 220, and · the fuel nozzle is being held in an appropriate posture, in this case, an annotation indicating that it is a fueler holding the fuel nozzle in an appropriate posture is assigned to the bounding box of the fueler.

[0035] Also, when a fueler is recognized in the image data, · the fueler is recognized within the second region 220, and · the fuel nozzle being held is not recognized, in this case, an annotation (fuel nozzle unrecognizable) indicating that the fueler is not holding the fuel nozzle in an appropriate posture is assigned to the circumscribed rectangle (bounding box) of the fueler.

[0036] Also, when the fueler is not recognized within the second region 220 in the image data, · for example, when a fuel nozzle in a state of being inserted into the fuel inlet is recognized, in this case, an annotation (fueler unrecognizable) indicating that the fueler is not holding the fuel nozzle in an appropriate posture is assigned to the circumscribed rectangle (bounding box) of the recognized fuel nozzle.

[0037] The learning device 150 generates first learning data using the image data with any annotation among the extracted image data.

[0038] Subsequently, an overview of the process from when the camera 130_2 that captures the region including the second fuel dispenser 120_2 as a video captures the second fuel dispenser 120_2 and the placed container until the learning device 150 generates second learning data based on the video will be described. FIG. 3B is a diagram showing the flow of the process for generating the second learning data.

[0039] As shown in FIG. 3B, the image data collection device 140 collects image data with a predetermined frame period captured as a video by the camera 130_2. Also, as shown in FIG. 3B, the image data collection device 140 collects a signal indicating the state of the fuel nozzle 121_2 from the second fuel dispenser 120_2.

[0040] The learning device 150 extracts image data of the fuel nozzle 121_2 from the image data collected by the image data collection device 140 during the period from when the fuel nozzle 121_2 is removed from the nozzle mount of the second fuel dispenser 120_2 until it is returned to the nozzle mount.

[0041] Also, the learning device 150 performs image recognition processing on the extracted image data, and recognizes each recognition target (container, fueler, fuel nozzle, etc.) included in the extracted image data. Further, the learning device 150 assigns an annotation to the recognized recognition target (container, fueler, fuel nozzle, etc.).

[0042] Specifically, when a container is recognized in the image data, an annotation indicating that it is a container is assigned to the circumscribed rectangle (bounding box) of the recognized container.

[0043] Also, when a fueler is recognized in the image data, · when the fueler is holding the fuel nozzle in an appropriate posture, an annotation indicating that the fueler is holding the fuel nozzle in an appropriate posture is assigned to the circumscribed rectangle (bounding box) of the fueler.

[0044] Also, when a fueler is recognized in the image data, · when the fuel nozzle held by the fueler cannot be recognized, an annotation (fuel nozzle recognition impossible) indicating that the fueler is not holding the fuel nozzle in an appropriate posture is assigned to the circumscribed rectangle (bounding box) of the fueler.

[0045] Also, when a fueler is not recognized in the image data, · for example, when a fuel nozzle inserted into the fuel inlet is recognized, In such a case, an annotation (indicating that the refueler cannot be recognized) indicating that the refueler is not holding the fuel nozzle in an appropriate posture is given to the circumscribed rectangle (bounding box) of the fuel nozzle.

[0046] The learning device 150 generates second learning data using the image data with any annotation among the extracted image data.

[0047] <Hardware Configuration of the Learning Device> Next, the hardware configuration of the learning device 150 will be described. FIG. 4 is a diagram showing an example of the hardware configuration of the learning device.

[0048] As shown in FIG. 4, the learning device 150 includes a processor 401, a memory 402, an auxiliary storage device 403, an I / F (Interface) device 404, a communication device 405, and a drive device 406. Note that the processor 401, the memory 402, the auxiliary storage device 403, the I / F device 404, the communication device 405, and the drive device 406 of the learning device 150 are interconnected via a bus 407.

[0049] The processor 401 includes various arithmetic devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 401 reads out and executes various programs (for example, a learning program, etc.) on the memory 402.

[0050] The memory 402 includes main storage devices such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processor 401 and the memory 402 form a so-called computer, and the computer realizes various functions by the processor 401 executing various programs read out on the memory 402.

[0051] The auxiliary storage device 403 stores various programs and various information used when the various programs are executed by the processor 401.

[0052] The I / F device 404 is a connection device for connecting the operation device 411, which is an example of an external device, the display device 412, and the learning device 150.

[0053] The communication device 405 is a communication device for communicating with, for example, the image data collection device 140 or the information processing device 160 via a network.

[0054] The drive device 406 is a device for setting the recording medium 413. The recording medium 413 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 413 may also include semiconductor memories that record information electrically, such as ROMs and flash memories.

[0055] Note that various programs installed in the auxiliary storage device 403 are installed, for example, by setting the distributed recording medium 413 in the drive device 406 and reading out the various programs recorded on the recording medium 413. Alternatively, the various programs installed in the auxiliary storage device 403 may be installed by being downloaded from a network via the communication device 405.

[0056] <Functional Configuration of the Learning Device> Next, the functional configuration of the learning device 150 will be described. FIG. 5 is a diagram showing an example of the functional configuration of the learning device. As described above, a learning program is installed in the learning device 150, and when the program is executed, the learning device 150 functions as a learning data generation unit 500A and a learning unit 500B.

[0057] As shown in FIG. 5, the learning data generation unit 500A further includes a collected data acquisition unit 510, an image recognition unit 520, an annotation adding unit 530, and a learning data generation unit 540.

[0058] The data collection unit 510 acquires, from the image data collection device 140, the image data captured by the camera 130_1 and the signal indicating the state of the fuel nozzle 121_1. Further, the data collection unit 510 acquires, from the image data collection device 140, the image data captured by the camera 130_2 and the signal indicating the state of the fuel nozzle 121_2.

[0059] The image recognition unit 520 extracts, from the image data captured by the camera 130_1, the image data during the period from when the fuel nozzle 121_1 is removed from the nozzle hanger of the first fuel dispenser 120_1 until it is returned to the nozzle hanger of the first fuel dispenser 120_1. Further, the image recognition unit 520 performs image recognition processing on the extracted image data. Thereby, the image recognition unit 520 recognizes the recognition targets (such as vehicles, containers, fuelers, fuel nozzles, etc.).

[0060] Also, the image recognition unit 520 extracts, from the image data captured by the camera 130_2, the image data during the period from when the fuel nozzle 121_2 is removed from the nozzle hanger of the second fuel dispenser 120_2 until it is returned to the nozzle hanger of the second fuel dispenser 120_2. Further, the image recognition unit 520 performs image recognition processing on the extracted image data. Thereby, the image recognition unit 520 recognizes the recognition targets (such as containers, fuelers, fuel nozzles, etc.).

[0061] The annotation adding unit 530 adds an annotation to the bounding box of the recognition target recognized by the image recognition unit 520. Since the method of adding the annotation has been described with reference to FIGS. 3A and 3B, the description is omitted here.

[0062] The learning data generation unit 540 generates first learning data and second learning data using the image data annotated by the annotation adding unit 530 as input data and the added annotation as correct data. The learning data generation unit 540 stores the generated first learning data and second learning data in the learning data storage unit 500C.

[0063] Also, as shown in FIG. 5, the learning unit 500B further includes a learning data reading unit 550, a first model 560_1, a comparison / modification unit 570_1, a second model 560_2, and a comparison / modification unit 570_2.

[0064] The learning data reading unit 550 reads the first learning data from the learning data storage unit 500C, inputs the input data included in the first learning data into the first model 560_1, and inputs the correct answer data into the comparison / modification unit 570_1.

[0065] Also, the learning data reading unit 550 reads the second learning data from the learning data storage unit 500C, inputs the input data included in the second learning data into the second model 560_2, and inputs the correct answer data into the comparison / modification unit 570_2.

[0066] The first model 560_1 is a machine learning model that performs image recognition processing. When the input data included in the first learning data is input, it performs image recognition processing and outputs a recognition result.

[0067] The comparison / modification unit 570_1 compares the recognition result output from the first model 560_1 with the correct answer data input by the learning data reading unit 550. Also, the comparison / modification unit 570_1 updates the model parameters of the first model 560_1 so that the recognition result output from the first model 560_1 matches the correct answer data. In this way, the learning process for the first model 560_1 is performed, and the first learned model is generated.

[0068] The second model 560_2 is a machine learning model that performs image recognition processing. When the input data included in the second learning data is input, it performs image recognition processing and outputs a recognition result.

[0069] The comparison / modification unit 570_2 compares the recognition result output from the second model 560_2 with the correct data input by the learning data reading unit 550. Further, the comparison / modification unit 570_2 updates the model parameters of the second model 560_2 so that the recognition result output from the second model 560_2 matches the correct data. In this way, the learning process for the second model 560_2 is performed, and the second learned model is generated.

[0070] Note that the first learned model and the second learned model generated by the learning unit 500B are transmitted to the information processing apparatus 160 or the like.

[0071] <Specific example of learning data> Next, a specific example of the learning data generated by the learning data generation unit 500A of the learning apparatus 150 will be described.

[0072] (1) Specific example of the first learning data FIG. 6 is a diagram showing a specific example of the first learning data. As shown in FIG. 6, the first learning data 600 includes, as information items, "input data" and "correct data". Further, the "input data" further includes an "image data ID".

[0073] The "image data ID" stores an identifier for identifying the image data to which the annotation has been added by the annotation adding unit 530. In the example of FIG. 6, the image data 610 is the image data with the image data ID = ID001, the image data 620 is the image data with the image data ID = ID00L, the image data 630 is the image data with the image data ID = ID00M, and the image data 640 is the image data with the image data ID = ID00N.

[0074] "Correct data" further includes "region" and "annotation data". The "region" stores the first region 210 and the second region 220, and the "annotation data" stores annotation data assigned to the bounding box of the recognition target recognized in the corresponding region.

[0075] In the example of FIG. 6, for the bounding box 611 of the vehicle recognized in the first region 210 which is a part of the image data 610, "Annotation data indicating that it is a vehicle" is shown as being assigned. Also, in the example of FIG. 6, for the bounding box 612 of the fueler recognized in the second region 220 which is a part of the image data 610, "Annotation indicating that it is a fueler holding the fuel nozzle in an appropriate posture" is shown as being assigned.

[0076] Similarly, in the example of FIG. 6, for the bounding box 621 of the vehicle recognized in the first region 210 which is a part of the image data 620, "Annotation data indicating that it is a vehicle" is shown as being assigned. Also, in the example of FIG. 6, for the bounding box 622 of the fueler recognized in the second region 220 which is a part of the image data 620, "Annotation indicating that it is a fueler not holding the fuel nozzle in an appropriate posture (fuel nozzle recognition impossible)" is shown as being assigned.

[0077] Similarly, in the example of FIG. 6, for the bounding box 631 of the vehicle recognized in the first region 210 which is a part of the image data 630, "Annotation data indicating that it is a vehicle" is shown as being assigned. Also, in the example of FIG. 6, for the bounding box 632 of the fuel nozzle recognized in the second region 220 which is a part of the image data 630, "Annotation indicating that the fueler is not holding the fuel nozzle in an appropriate posture (fueler unrecognizable)" indicates that it has been assigned.

[0078] Similarly, in the example of FIG. 6, for the bounding box 641 of the container recognized in the first region 210 which is a part of the image data 640, "Annotation data indicating that it is a container" indicates that it has been assigned. Also, in the example of FIG. 6, for the bounding box 642 of the fueler recognized in the second region 220 which is a part of the image data 640, "Annotation indicating that the fueler is holding the fuel nozzle in an appropriate posture" indicates that it has been assigned.

[0079] (2) Specific examples of the second learning data FIG. 7 is a diagram showing a specific example of the second learning data. As shown in FIG. 7, the second learning data 700 includes, as information items, "input data" and "correct answer data". Further, the "input data" includes "image data ID".

[0080] The "image data ID" stores an identifier for identifying the image data to which an annotation has been given by the annotation adding unit 530. In the example of FIG. 7, the image data 710 is the image data with the image data ID = ID101, the image data 720 is the image data with the image data ID = ID10L, the image data 730 is the image data with the image data ID = ID10M, and the image data 740 is the image data with the image data ID = ID10N.

[0081] The "correct answer data" further includes "annotation data". The "annotation data" stores the annotation data given to the recognized bounding box of the recognition target.

[0082] The example in FIG. 7 shows that for the bounding box 711 of the container recognized in the image data 710, "Annotation data indicating that it is a container" has been assigned. Also, the example in FIG. 7 shows that for the bounding box 712 of the fueler recognized in the image data 710, "Annotation indicating that it is a fueler holding the fuel nozzle in an appropriate posture" has been assigned.

[0083] Similarly, the example in FIG. 7 shows that for the bounding box 721 of the container recognized in the image data 720, "Annotation data indicating that it is a container" has been assigned. Also, the example in FIG. 7 shows that for the bounding box 722 of the fueler recognized in the image data 720, "Annotation indicating that it is a fueler not holding the fuel nozzle in an appropriate posture (fuel nozzle not recognized)" has been assigned.

[0084] Similarly, the example in FIG. 7 shows that for the bounding box 731 of the container recognized in the image data 730, "Annotation data indicating that it is a container" has been assigned. Also, the example in FIG. 7 shows that for the bounding box 732 of the fuel nozzle recognized in the image data 730, "Annotation indicating that it is a fueler not holding the fuel nozzle in an appropriate posture (fueler not recognized)" has been assigned.

[0085] Similarly, the example in FIG. 7 shows that for the bounding box 742 of the fueler recognized in the image data 740, "Annotation indicating that it is a fueler holding the fuel nozzle in an appropriate posture" has been assigned.

[0086] <Flow of Learning Process by Learning Device> Next, the flow of the learning process by the learning device 150 will be described. FIG. 8 is an example of a flowchart showing the flow of the learning process.

[0087] In step S801, the learning device 150 acquires image data and a signal indicating the state of the fuel nozzle from the image data collection device 140.

[0088] In step S802, the learning device 150 extracts the image data during the period from when the fuel nozzle is removed from the nozzle holder until it is returned to the nozzle holder, and performs image recognition processing on the extracted image data.

[0089] In step S803, the learning device 150 specifies the first region and the second region in the image data used for generating the first learning data among the image data on which the image recognition processing has been performed.

[0090] In step S804, the learning device 150 assigns an annotation to the image data used for generating the first learning data. Specifically, the learning device 150 assigns an annotation to the bounding box of the recognition target included in the specified first region and also assigns an annotation to the bounding box of the recognition target included in the specified second region.

[0091] Also, the learning device 150 assigns an annotation to the image data used for generating the second learning data. Specifically, the learning device 150 assigns an annotation to the bounding box of the recognition target.

[0092] In step S805, the learning device 150 generates first learning data for the first fuel type and second learning data for the second fuel type based on the image data with the annotation.

[0093] In step S806, the learning device 150 performs learning processing on the first model using the first learning data to generate a first learned model. Also, the learning device 150 performs learning processing on the second model using the second learning data to generate a second learned model.

[0094] In step S807, the learning device 150 transmits the generated first learned model and second learned model to the information processing device 160 etc. and stores them in the information processing device 160 etc.

[0095] <Hardware Configuration of Information Processing Device> Next, the hardware configuration of the information processing device 160 will be described. FIG. 9 is a diagram showing an example of the hardware configuration of the information processing device.

[0096] As shown in FIG. 9, the information processing device 160 includes a processor 901, a memory 902, an auxiliary storage device 903, a connection device 904, an I / F device 905, a communication device 906, and a drive device 907. Note that the processor 901, memory 902, auxiliary storage device 903, connection device 904, I / F device 905, communication device 906, and drive device 907 of the information processing device 160 are interconnected via a bus 908.

[0097] The processor 901 includes various arithmetic devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 901 reads out and executes various programs (for example, a fuel supply control program etc.) on the memory 902.

[0098] The memory 902 includes main memory devices such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processor 901 and the memory 902 form a so-called computer, and the computer realizes various functions by the processor 901 executing various programs read out on the memory 902.

[0099] The auxiliary storage device 903 stores various programs and various information used when the various programs are executed by the processor 901.

[0100] The connection device 904 is a connection device for connecting the first fuel dispenser 120_1, the second fuel dispenser 120_2, and the information processing device 160. The connection device 904 receives signals indicating the state of the fuel nozzle, signals indicating the fuel quantity, etc. from the first fuel dispenser 120_1 and the second fuel dispenser 120_2. Signals indicating the state of the fuel nozzle, signals indicating the fuel quantity, etc. are measured by the first fuel sensor 921_1 and the second fuel sensor 921_2. Further, the connection device 904 transmits a message for voice output to the first fuel dispenser 120_1 and the second fuel dispenser 120_2 via the voice output devices 122_1 and 122_2. Also, the connection device 904 transmits display information for display to the first fuel dispenser 120_1 and the second fuel dispenser 120_2 via the fuel data displays 123_1 and 123_2. Further, the connection device 904 transmits a signal for instructing the opening and closing operations of the fuel valves 124_1 and 124_2 to the first fuel dispenser 120_1 and the second fuel dispenser 120_2. Furthermore, the connection device 904 transmits a signal for instructing the start / stop of driving to the fuel pumps 125_1 and 125_2.

[0101] Also, the connection device 904 receives thermal sensor data from the thermal sensors 170_1 and 170_2.

[0102] The I / F device 905 is a connection device for connecting an operation device 911 and a display device 912, which are examples of external devices, and the information processing device 160. Note that the operation device 911 is operated by, for example, the staff of the service station 110', and the display device 912 is displayed for, for example, the staff of the service station 110'.

[0103] The communication device 906 communicates with the cameras 130_1 and 130_2 via a network and receives image data. Also, the communication device 906 communicates with the learning device 150 and receives the first learned model and the second learned model.

[0104] The drive device 907 is a device for setting the recording medium 913. The recording medium 913 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, magneto-optical disks, etc. The recording medium 913 may also include semiconductor memories that record information electrically, such as ROMs and flash memories.

[0105] Note that various programs installed in the auxiliary storage device 903 are installed, for example, when the distributed recording medium 913 is set in the drive device 907 and various programs recorded on the recording medium 913 are read out. Alternatively, various programs installed in the auxiliary storage device 903 may be installed by being downloaded from a network via the communication device 906.

[0106] <Functional Configuration of Information Processing Device> Next, the functional configuration of the information processing device 160 will be described. FIG. 10 is a diagram showing an example of the functional configuration of the information processing device. As described above, a fuel supply control program is installed in the information processing device 160, and when the program is executed, the information processing device 160 · First fuel supply type image data acquisition unit 1001, second fuel supply type image data acquisition unit 1002, · First fuel supply sensor data acquisition unit 1003, second fuel supply sensor data acquisition unit 1004, · Image recognition unit 1005, · Section determination units 1006, 1007, · Thermal sensor data acquisition unit 1008, · First fuel supply monitoring units 1009_1, 1009_2, · Output control unit 1010, has.

[0107] The first fuel supply type image data acquisition unit 1001 acquires the image data captured by the camera 130_1 and notifies the image recognition unit 1005.

[0108] The second fuel supply type image data acquisition unit 1002 acquires the image data captured by the camera 130_2 and notifies the image recognition unit 1005.

[0109] The first fuel supply sensor data acquisition unit 1003 acquires the signal from the first fuel supply sensor 921_1 and notifies the section determination unit 1006.

[0110] The second fuel supply sensor data acquisition unit 1004 acquires the signal from the second fuel supply sensor 921_2 and notifies the section determination unit 1007.

[0111] Based on the signal from the first fuel supply sensor 921_1, the section determination unit 1006 classifies the section from when the fuel supply nozzle 121_1 is removed from the nozzle holder of the first fuel dispenser 120_1 until it is returned to the nozzle holder into three sections. Among the three sections, the section before fuel supply start is referred to as the first section, the section during fuel supply is referred to as the second section, and the section after fuel supply end is referred to as the third section.

[0112] Based on the signal from the second fuel supply sensor 921_2, the section determination unit 1007 classifies the section from when the fuel supply nozzle 121_2 is removed from the nozzle holder of the second fuel dispenser 120_2 until it is returned to the nozzle holder into three sections. For the second fuel supply type as well, similar to the first fuel supply type, among the three sections, the section before fuel supply start is referred to as the first section, the section during fuel supply is referred to as the second section, and the section after fuel supply end is referred to as the third section.

[0113] The image recognition unit 1005 extracts the image data included in any one of the first to third sections from the image data notified by the first fuel supply type image data acquisition unit 1001. Also, the image recognition unit 1005 performs image recognition processing on the extracted image data using the first learned model and notifies the recognition result to the first fuel supply monitoring unit 1009_1.

[0114] Further, the image recognition unit 1005 extracts the image data included in any one of the first to third intervals from the image data notified by the second fuel type image data acquisition unit 1002. Further, the image recognition unit 1005 performs image recognition processing on the extracted image data using the second learned model, and notifies the recognition result to the second fuel supply monitoring unit 1009_2.

[0115] The thermal sensor data acquisition unit 1008 acquires the thermal sensor data measured by the thermal sensor 170_1 and notifies it to the first fuel supply monitoring unit 1009_1. Further, the thermal sensor data acquisition unit 1008 acquires the thermal sensor data measured by the thermal sensor 170_2 and notifies it to the second fuel supply monitoring unit 1009_2.

[0116] The first fuel supply monitoring unit 1009_1 · The recognition result notified by the image recognition unit 1005, · The section determination result notified by the section determination unit 1006, · The thermal sensor data of the thermal sensor 170_1 notified by the thermal sensor data acquisition unit 1008, are used to determine whether fuel supply for the first fuel type is permitted. Further, the first fuel supply monitoring unit 1009_1 controls the fuel supply for the first fuel type based on the determination result of whether fuel supply is permitted. Specifically, the first fuel supply monitoring unit 1009_1 controls the fuel supply for the first fuel type by controlling the fuel valve 124_1 or the fuel pump 125_1.

[0117] Further, the first fuel supply monitoring unit 1009_1 notifies the output control unit 1010 of the determination result of whether fuel supply is permitted. In addition, when the first fuel supply monitoring unit 1009_1 determines that fuel supply is not permitted, it also notifies the output control unit 1010 of the information indicating the cause of the non - permission of fuel supply.

[0118] The second fuel supply monitoring unit 1009_2 · The recognition result notified by the image recognition unit 1005, · The section determination result notified by the section determination unit 1007, · Using the thermal sensor data of the thermal sensor 170_2 notified by the thermal sensor data acquisition unit 1008, determine whether refueling is permitted for the second refueling type. Further, the second refueling monitoring unit 1009_2 controls the refueling for the second refueling type based on the determination result of whether refueling is permitted. Specifically, the second refueling monitoring unit 1009_2 controls the refueling for the second refueling type by controlling the refueling valve 124_2 or the refueling pump 125_2.

[0119] In addition, the second refueling monitoring unit 1009_2 notifies the output control unit 1010 of the determination result of whether refueling is permitted. Note that when the second refueling monitoring unit 1009_2 determines that refueling is not permitted, it also notifies the output control unit 1010 of the information indicating the cause of the non-permission of refueling.

[0120] When the output control unit 1010 is notified of the information indicating the cause of the non-permission of refueling from the first refueling monitoring unit 1009_1, · Control so that a message corresponding to the information indicating the cause is output as voice via the voice output device 122_1, · Control so that the display information corresponding to the information indicating the cause is displayed and output on the refueling data display 123_1 of the first fuel dispenser 120_1, · Control so that the information indicating the cause is displayed and output on the display device 912 of the information processing device 160.

[0121] In addition, when the output control unit 1010 is notified of the information indicating the cause of the non-permission of refueling from the second refueling monitoring unit 1009_2, · Control so that a message corresponding to the information indicating the cause is output as voice via the voice output device 122_2, · Control so that the display information corresponding to the information indicating the cause is displayed and output on the refueling data display 123_2 of the second fuel dispenser 120_2, · Control so that the information indicating the cause is displayed and output on the display device 912 of the information processing device 160.

[0122] <Details of each part of the information processing device> Next, details of each part of the information processing apparatus 160 (here, the section determination units 1006 and 1007, the image recognition unit 1005, the first fuel supply monitoring unit 1009_1, the second fuel supply monitoring unit 1009_2, and the output control unit 1010) will be described.

[0123] (1) Details of the section determination units 1006 and 1007 First, as details of the section determination units 1006 and 1007, the functional configurations of the section determination units 1006 and 1007 and specific examples of the processing of the section determination units 1006 and 1007 will be described.

[0124] (1-1) Details of the functional configurations of the section determination units 1006 and 1007 FIG. 11 is a diagram showing an example of the functional configuration of the section determination unit of the information processing apparatus. Since the functional configurations of the section determination unit 1006 and the section determination unit 1007 are the same, they will be described together using FIG. 11.

[0125] As shown in FIG. 11, the section determination units 1006 and 1007 · nozzle state determination unit 1101, · fuel supply amount determination unit 1102, · first section start determination unit 1103, second section transition determination unit 1104, third section transition determination unit 1105, third section end determination unit 1106, · section determination result output unit 1107, have.

[0126] The nozzle state determination unit 1101 acquires a signal indicating the state of the fuel supply nozzle 121_1 (or 121_2) from the first fuel supply sensor data acquisition unit 1003 (or the second fuel supply sensor data acquisition unit 1004).

[0127] When the nozzle state determination unit 1101 receives a signal indicating that the fuel supply nozzle 121_1 (or 121_2) has been removed from the nozzle mount of the first fuel dispenser 120_1 (or the second fuel dispenser 120_2), it notifies the first section start determination unit 1103.

[0128] Further, when the nozzle state determination unit 1101 receives a signal indicating that the fuel supply nozzle 121_1 (or 121_2) has been returned to the nozzle mount of the first fuel dispenser 120_1 (or the second fuel dispenser 120_2), it notifies the third section end determination unit 1106.

[0129] The fuel quantity determination unit 1102 acquires a signal indicating the fuel quantity from the first fuel supply sensor data acquisition unit 1003 (or the second fuel supply sensor data acquisition unit 1004).

[0130] When the signal indicating the fuel quantity acquired by the fuel quantity determination unit 1102 from the first fuel supply sensor data acquisition unit 1003 (or the second fuel supply sensor data acquisition unit 1004) changes from zero to a predetermined threshold value or more, the fuel quantity determination unit 1102 notifies the second section transition determination unit 1104.

[0131] Also, when the signal indicating the fuel quantity acquired by the fuel quantity determination unit 1102 from the first fuel supply sensor data acquisition unit 1003 (or the second fuel supply sensor data acquisition unit 1004) changes from a predetermined threshold value or more to zero, the fuel quantity determination unit 1102 notifies the third section transition determination unit 1105.

[0132] When the first section start determination unit 1103 is notified from the nozzle state determination unit 1101 of a signal indicating that the fuel supply nozzle 121_1 (or 121_2) has been removed, it determines that the first section has started. Also, the first section start determination unit 1103 notifies the second section transition determination unit 1104 and the section determination result output unit 1107 that the first section has started.

[0133] After being notified that the first section has started, when the second section transition determination unit 1104 is notified from the fuel quantity determination unit 1102 that the fuel quantity has changed from zero to a predetermined threshold value or more, it determines that the transition has been made from the first section to the second section. Also, the second section transition determination unit 1104 notifies the third section transition determination unit 1105 and the section determination result output unit 1107 that the transition has been made from the first section to the second section.

[0134] After being notified that the transition to the second section has occurred, the third-section transition determination unit 1105 determines that the transition to the third section has occurred when it is notified from the fuel supply amount determination unit 1102 that the fuel supply amount has changed from equal to or greater than a predetermined threshold value to zero. Further, the third-section transition determination unit 1105 notifies the third-section end determination unit 1106 and the section determination result output unit 1107 that the transition from the second section to the third section has occurred.

[0135] After being notified that the transition to the third section has occurred, the third-section end determination unit 1106 determines that the third section has ended when it is notified from the nozzle state determination unit 1101 that a signal indicating that the fuel supply nozzle 121_1 (or 121_2) has been returned is received. Further, the third-section end determination unit 1106 notifies the section determination result output unit 1107 that the third section has ended.

[0136] When the section determination result output unit 1107 is notified from the first-section start determination unit 1103 that the first section has started, it starts outputting information indicating that it is during the first section. Further, when the section determination result output unit 1107 is notified from the second-section transition determination unit 1104 that the transition to the second section has occurred, it stops outputting information indicating that it is during the first section and starts outputting information indicating that it is during the second section. Also, when the section determination result output unit 1107 is notified from the third-section transition determination unit 1105 that the transition to the third section has occurred, it stops outputting information indicating that it is during the second section and starts outputting information indicating that it is during the third section. Furthermore, when the section determination result output unit 1107 is notified from the third-section end determination unit 1106 that the third section has ended, it stops outputting information indicating that it is during the third section.

[0137] (1-2) Specific examples of the processing of the section determination units 1006 and 1007 FIG. 12 is a diagram showing a specific example of the processing by the section determination unit of the information processing apparatus. As shown in FIG. 12(a), when acquiring, as a signal indicating the state of the fuel supply nozzle 121_1, a signal indicating that the fuel supply nozzle 121_1 has been removed from the nozzle holder, the section determination unit 1006 starts outputting information indicating that it is in the first section. Also, when acquiring a signal indicating that the signal indicating the fuel supply amount of the first fuel dispenser 120_1 has changed from zero to a predetermined threshold value or more, the section determination unit 1006 starts outputting information indicating that it is in the second section. Further, when acquiring a signal indicating that the signal indicating the fuel supply amount of the first fuel dispenser 120_1 has changed from a predetermined threshold value or more to zero, the section determination unit 1006 starts outputting information indicating that it is in the third section. Furthermore, when acquiring, as a signal indicating the state of the fuel supply nozzle 121_1, a signal indicating that the fuel supply nozzle 121_1 has been returned to the nozzle holder, the section determination unit 1006 stops outputting information indicating that it is in the third section.

[0138] Similarly, as shown in FIG. 12(b), when acquiring, as a signal indicating the state of the fuel supply nozzle 121_2, a signal indicating that the fuel supply nozzle 121_2 has been removed from the nozzle holder, the section determination unit 1007 starts outputting information indicating that it is in the first section. Also, when acquiring a signal indicating that the signal indicating the fuel supply amount of the second fuel dispenser 120_2 has changed from zero to a predetermined threshold value or more, the section determination unit 1007 starts outputting information indicating that it is in the second section. Further, when acquiring a signal indicating that the signal indicating the fuel supply amount of the second fuel dispenser 120_2 has changed from a predetermined threshold value or more to zero, the section determination unit 1007 starts outputting information indicating that it is in the third section. Furthermore, when acquiring, as a signal indicating the state of the fuel supply nozzle 121_2, a signal indicating that the fuel supply nozzle 121_2 has been returned to the nozzle holder, the section determination unit 1007 stops outputting information indicating that it is in the third section.

[0139] (2) Details of the image recognition unit 1005 Next, as details of the image recognition unit 1005, the functional configuration of the image recognition unit 1005 and a specific example of the processing of the image recognition unit 1005 will be described.

[0140] (2-1) Details of the functional configuration of the image recognition unit 1005 FIG. 13 is a diagram showing an example of the functional configuration of the image recognition unit of the information processing apparatus. As shown in FIG. 13, the image recognition unit 1005 includes a recognition section acquisition unit 1301, a first learned model 1302, and a region determination unit 1303. Further, the image recognition unit 1005 includes a recognition section acquisition unit 1311 and a second learned model 1312.

[0141] The recognition section acquisition unit 1301 extracts the image data included in any one of the first to third sections notified from the section determination unit 1006 from the image data notified from the first fuel type image data acquisition unit 1001, and inputs the extracted image data to the first learned model 1302.

[0142] The first learned model 1302 is a first learned model generated in the learning apparatus 150. When image data is input, it recognizes the recognition target included in the image data and outputs the recognition result.

[0143] The region determination unit 1303 determines whether the recognition result output by the first learned model 1302 is included in the first region 210 or the second region 220. Further, the region determination unit 1303 outputs the recognition result determined to be included in the first region 210 or the second region 220 to the first fuel monitoring unit 1009_1.

[0144] The recognition section acquisition unit 1311 extracts the image data included in any one of the first to third sections notified from the section determination unit 1007 from the image data notified from the second fuel type image data acquisition unit 1002, and inputs the extracted image data to the second learned model 1312.

[0145] The second learned model 1312 is a second learned model generated in the learning apparatus 150. When image data is input, it recognizes the recognition target included in the image data and outputs the recognition result to the second fuel monitoring unit 1009_2.

[0146] (2-2) Specific Example of the Processing of the Image Recognition Unit 1005 (in the Case of the First Fueling Type) FIG. 14A is a first diagram showing a specific example of the processing by the image recognition unit of the information processing apparatus. As shown in FIG. 14A, the image recognition unit 1005 extracts the image data included in the section (the first to third sections) from when the fueling nozzle 121_1 is removed from the nozzle holder until it is returned, among the image data captured by the camera 130_1.

[0147] Also, as shown in FIG. 14A, the image recognition unit 1005 performs image recognition processing on the extracted image data using the first pre-trained model 1302, and by recognizing the recognition targets (such as vehicles, containers, fuelers, fueling nozzles, etc.), · Vehicles, · Containers, · Fuelers who hold the fueling nozzle in an appropriate posture, · Fuelers who do not hold the fueling nozzle in an appropriate posture (fueling nozzle unrecognizable), · Fuelers who do not hold the fueling nozzle in an appropriate posture (fueler unrecognizable), are output as recognition results.

[0148] Also, the image recognition unit 1005 outputs the recognition results determined to be included in the first area 210 or the second area 220. The recognition results 1401 to 1404 in FIG. 14A show output examples of the recognition results.

[0149] In the example of FIG. 14A, the recognition result 1401 output by the image recognition unit 1005 is · In the first area 210, a bounding box in which a vehicle is recognized, · In the second area 220, a bounding box in which a fueler holding the fueling nozzle in an appropriate posture is recognized, as shown.

[0150] Also, in the example of FIG. 14A, the recognition result 1402 output by the image recognition unit 1005 is · In the first area 210, a bounding box in which a vehicle is recognized, · In the second area 220, a bounding box (fuel nozzle unrecognizable) in which a fueler not holding the fuel nozzle in an appropriate posture is recognized is shown.

[0151] Also, in the example of FIG. 14A, the recognition result 1403 output by the image recognition unit 1005 is · A bounding box in which a vehicle is recognized in the first area 210 · In the second area 220, a bounding box (fueler unrecognizable) in which a fueler not holding the fuel nozzle in an appropriate posture is recognized is shown.

[0152] Furthermore, in the example of FIG. 14A, the recognition result 1404 output by the image recognition unit 1005 is · A bounding box in which a container is recognized in the first area 210 · In the second area 220, a bounding box in which a fueler holding the fuel nozzle in an appropriate posture is recognized is shown.

[0153] (2-3) Specific example of the process of the image recognition unit 1005 (in the case of the second fueling type) FIG. 14B is a second diagram showing a specific example of the process by the image recognition unit of the information processing device. As shown in FIG. 14B, the image recognition unit 1005 extracts the image data included in the section (the first to the third sections) from when the fuel nozzle 121_2 is removed from the nozzle hanger until it is returned, among the image data captured by the camera 130_2.

[0154] Also, as shown in FIG. 14B, the image recognition unit 1005 performs image recognition processing on the extracted image data using the second learned model 1312, and by recognizing the recognition targets (such as the container, the fueler, the fuel nozzle, etc.), · The container · A fueler holding the fuel nozzle in an appropriate posture · A fueler not holding the fuel nozzle in an appropriate posture (fuel nozzle unrecognizable) · A fueler who is not holding the fuel nozzle in an appropriate posture (fueler not recognizable), is output as a recognition result. The recognition results 1411 to 1414 in Fig. 14B show examples of the output of the recognition result.

[0155] In the example of Fig. 14B, the recognition result 1411 output by the image recognition unit 1005 is · The bounding box in which the container is recognized, · The bounding box in which a fueler holding the fuel nozzle in an appropriate posture is recognized. is shown.

[0156] Also, in the example of Fig. 14B, the recognition result 1412 output by the image recognition unit 1005 is · The bounding box in which the container is recognized, · The bounding box in which a fueler not holding the fuel nozzle in an appropriate posture is recognized (fuel nozzle not recognizable). is shown.

[0157] Also, in the example of Fig. 14B, the recognition result 1413 output by the image recognition unit 1005 is · The bounding box in which the container is recognized, · The bounding box in which a fueler not holding the fuel nozzle in an appropriate posture is recognized (fueler not recognizable). is shown.

[0158] Furthermore, in the example of Fig. 14B, the recognition result 1414 output by the image recognition unit 1005 is · The bounding box in which a fueler holding the fuel nozzle in an appropriate posture is recognized. is shown.

[0159] (3) Details of the first fueling monitoring unit 1009_1 and the second fueling monitoring unit 1009_2 Next, as details of the first fuel supply monitoring unit 1009_1 and the second fuel supply monitoring unit 1009_2, the functional configuration and processing flow of the first fuel supply monitoring unit 1009_1 and the functional configuration and processing flow of the second fuel supply monitoring unit 1009_2 will be described.

[0160] (3-1) Details of the functional configuration of the first fuel supply monitoring unit 1009_1 and the second fuel supply monitoring unit 1009_2 FIG. 15A is a diagram showing an example of the functional configuration of the first fuel supply monitoring unit of the information processing apparatus. As shown in FIG. 15A, the first fuel supply monitoring unit 1009_1 includes a recognition result acquisition unit 1501, a determination logic switching unit 1502, a poly-tank determination unit 1506, a first determination logic 1503, a second determination logic 1504, a third determination logic 1505, and a fire determination unit 1507. The first fuel supply monitoring unit 1009_1 also includes a fuel supply control unit 1508 and a cause information acquisition unit 1509. The first determination logic 1503 to the third determination logic 1505, the poly-tank determination unit 1506, and the fire determination unit 1507 are collectively referred to as the first determination unit.

[0161] The recognition result acquisition unit 1501 acquires recognition results for the image data included in the first section to the third section from the image recognition unit 1005. The recognition result acquisition unit 1501 notifies the acquired recognition results to the determination logic switching unit 1502 and the poly-tank determination unit 1506.

[0162] The determination logic switching unit 1502 inputs the recognition result notified by the recognition result acquisition unit 1501 into any one of the first determination logic 1503 to the third determination logic 1505. Which one of the first determination logic 1503 to the third determination logic 1505 to input is determined by the information indicating in which of the first to third sections is notified from the section determination unit 1006. For example, while the information indicating that it is within the first section is notified, the determination logic switching unit 1502 inputs the recognition result into the first determination logic 1503. Also, while the information indicating that it is within the second section is notified, the determination logic switching unit 1502 inputs the recognition result into the second determination logic 1504. Also, while the information indicating that it is within the third section is notified, the determination logic switching unit 1502 inputs the recognition result into the third determination logic 1505.

[0163] Also, the determination logic switching unit 1502 notifies the poly tank determination unit 1506 and the firearm determination unit 1507 of the information indicating that it is within any one of the first to third sections.

[0164] When the poly tank determination unit 1506 is notified of the information indicating that it is within any one of the first to third sections and the recognition result is notified from the recognition result acquisition unit 1501, the poly tank determination unit 1506 determines whether a container is included in the notified recognition result. When the poly tank determination unit 1506 determines that a container is included, it notifies the fuel supply control unit 1508 of fuel supply non - permission.

[0165] The first determination logic 1503 determines whether fuel supply is permitted based on the recognition result in the first section. Also, the first determination logic 1503 notifies the fuel supply control unit 1508 of the determination result of whether fuel supply is permitted in the first section. When the first determination logic 1503 determines fuel supply non - permission, when notifying the determination result to the fuel supply control unit 1508, it also notifies the information indicating the reason for determining fuel supply non - permission.

[0166] The second determination logic 1504 determines whether refueling is permitted based on the recognition result during the second period. Further, the second determination logic 1504 notifies the refueling control unit 1508 of the determination result of whether refueling is permitted during the second period. When the second determination logic 1504 determines that refueling is not permitted, it notifies the refueling control unit 1508 of the determination result and also notifies information indicating the reason for the determination that refueling is not permitted.

[0167] The third determination logic 1505 determines whether refueling is permitted based on the recognition result during the third period. Further, the third determination logic 1505 notifies the refueling control unit 1508 of the determination result of whether refueling is permitted during the third period. When the third determination logic 1505 determines that refueling is not permitted, it notifies the refueling control unit 1508 of the determination result and also notifies information indicating the reason for the determination that refueling is not permitted.

[0168] The fire determination unit 1507 acquires the thermal sensor data measured by the thermal sensor 170_1 from the thermal sensor data acquisition unit 1008. Further, when information indicating that it is in any one of the first to third periods is notified, the fire determination unit 1507 determines whether a temperature region equal to or higher than a predetermined threshold is included. When the fire determination unit 1507 determines that a temperature region equal to or higher than the predetermined threshold is included, it notifies the refueling control unit 1508 that refueling is not permitted.

[0169] The refueling control unit 1508 controls the refueling valve 124_1 or the fuel pump 125_1 based on whether refueling is permitted or not permitted, thereby controlling the refueling for the first refueling type. For example, when refueling permission is notified, the refueling control unit 1508 drives the fuel pump 125_1 to open the refueling valve 124_1. Further, the refueling control unit 1508 · When notified of refueling permission from the poly-tank determination unit 1506, drives the fuel pump 125_1 to open the refueling valve 124_1. Furthermore, · When notified of refueling non-permission from any one of the first determination logic 1503 to the third determination logic 1505, or · When a notification of refueling prohibition is received from the fire determination unit 1507, refueling is stopped by stopping the drive of the fuel pump 125_1, or refueling is prohibited by stopping the drive of the fuel pump 125_1 and closing the fuel valve 124_1. Note that the definitions of refueling stop and refueling prohibition here are only examples, and the refueling stop and refueling prohibition may be defined by other definitions. Alternatively, without distinguishing between refueling stop and refueling prohibition, the entire state where refueling cannot be performed may be referred to as refueling stop or prohibition.

[0170] Also, the fuel supply control unit 1508 notifies the cause information acquisition unit 1509 of fuel supply permission and fuel supply non - permission. When the fuel supply control unit 1508 notifies fuel supply non - permission, it also notifies the cause information acquisition unit 1509 of information indicating the cause of the determination of fuel supply non - permission.

[0171] The cause information acquisition unit 1509 outputs fuel supply permission and fuel supply non - permission to the output control unit 1010. When the cause information acquisition unit 1509 outputs fuel supply non - permission, it also outputs information indicating the cause of fuel supply non - permission (the first to fourth information, warning information, confirmation request information) notified from the fuel supply control unit 1508 to the output control unit 1010.

[0172] FIG. 15B is a diagram showing an example of the functional configuration of the second fuel supply monitoring unit of the information processing device. The difference from the functional configuration of the first fuel supply monitoring unit 1009_1 shown in FIG. 15A is that the functional configuration of the second fuel supply monitoring unit 1009_2 does not include a functional unit corresponding to the poly - tank determination unit 1506. Since the other functional units of the second fuel supply monitoring unit 1009_2 are the same as those of each functional unit of the first fuel supply monitoring unit 1009_1, the description is omitted here.

[0173] Note that what was referred to as the first determination unit in the first fuel supply monitoring unit 1009_1 is collectively referred to as the second determination unit in the second fuel supply monitoring unit 1009_2, including the first determination logic 1513 to the third determination logic 1515 and the fire determination unit 1517.

[0174] (3-2) Flow of Processing Using the First Determination Logic 1503 of the First Fueling Monitoring Unit 1009_1 Next, the flow of processing using the first determination logic 1503 of the first fueling monitoring unit 1009_1 will be described with reference to FIGS. 16A and 16B. FIGS. 16A and 16B are examples of first and second flowcharts showing the flow of processing using the first determination logic of the first fueling monitoring unit.

[0175] In step S1601, the first determination logic 1503 determines whether the currently input recognition result is a recognition result determined to be fueling permission (whether a predetermined permission condition is satisfied). Specifically, it is determined whether a vehicle is recognized in the first area 210 and whether a fueler who holds the fueling nozzle in an appropriate posture is recognized in the second area.

[0176] If it is determined in step S1601 that the recognition result is a recognition result determined to be fueling permission (if YES in step S1601), the process proceeds to step S1602.

[0177] In step S1602, the first determination logic 1503 determines whether the recognition result determined to be fueling permission has continued for a predetermined time T1 (whether a predetermined permission condition has been satisfied for a certain time). If it is determined in step S1602 that it has continued for the predetermined time T1 (if YES in step S1602), the process proceeds to step S1603.

[0178] In step S1603, the first determination logic 1503 determines fueling permission and proceeds to step S1614 in FIG. 16B.

[0179] On the other hand, if it is determined in step S1601 that the recognition result is not a recognition result determined to be fueling permission (if NO in step S1601), the process proceeds to step S1604. Or, if it is determined in step S1602 that it has not continued for the predetermined time T1 (if NO in step S1602), the process proceeds to step S1604.

[0180] In step S1604, the first determination logic 1503 determines whether a state where the recognition result is not determined to be refueling permission has continued for a predetermined time T2 (where T2 > T1).

[0181] In step S1604, if it is determined that it has not continued for the predetermined time T2 (if NO in step S1604), the process returns to step S1601. On the other hand, in step S1604, if it is determined that it has continued for the predetermined time T2 (if YES in step S1604), the process proceeds to step S1605.

[0182] In step S1605, the first determination logic 1503 determines whether the currently input recognition result is a recognition result determined to be refueling not permitted (whether a predetermined non - permission condition is satisfied). Specifically, it is determined whether a vehicle is recognized in the first area 210 and whether a refueler who is not holding the fuel nozzle in an appropriate posture (unrecognizable fuel nozzle) is recognized in the second area.

[0183] In step S1605, if it is determined that the recognition result is a recognition result determined to be refueling not permitted (if YES in step S1605), it is determined that the predetermined non - permission condition has been satisfied for a certain time, and the process proceeds to step S1606.

[0184] In step S1606, the first determination logic 1503 determines refueling not permitted, acquires the first information, and then proceeds to step S1611 in FIG. 16B.

[0185] On the other hand, in step S1605, if it is determined that the recognition result is another recognition result determined to be refueling not permitted (if NO in step S1605), the process proceeds to step S1617 in FIG. 16B.

[0186] In step S1611, the first determination logic 1503 determines whether the currently input recognition result is a recognition result determined as refueling permission (whether a predetermined permission condition is satisfied). Specifically, it determines whether a vehicle is recognized in the first area 210 and whether a refueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0187] If it is determined in step S1611 that the recognition result is not a recognition result determined as refueling permission (if NO in step S1611), the first refueling monitoring unit 1009_1 ends the process using the first determination logic 1503.

[0188] On the other hand, if it is determined in step S1611 that the recognition result is a recognition result determined as refueling permission (if YES in step S1611), the process proceeds to step S1612.

[0189] In step S1612, the first determination logic 1503 determines whether the recognition result determined as refueling permission has continued for a predetermined time T1 (whether the predetermined permission condition has been satisfied for a certain time). If it is determined in step S1612 that it has continued for the predetermined time T1 (if YES in step S1612), the process proceeds to step S1613.

[0190] In step S1613, the first determination logic 1503 determines that refueling is permitted and proceeds to step S1614.

[0191] In step S1614, the determination logic switching unit 1502 determines whether it is within the first section. If it is determined in step S1614 that it is within the first section, it is determined that the process by the first determination logic 1503 is not ended (determined as NO in step S1614), and the process returns to step S1601 in FIG. 16A.

[0192] On the other hand, if it is determined in step S1614 that it is not during the first period, it is determined to end the process by the first determination logic 1503 (determined as YES in step S1614). Then, the first fueling monitoring unit 1009_1 ends the process using the first determination logic 1503.

[0193] On the other hand, if it is determined in step S1612 that the predetermined time T1 has not elapsed continuously (if NO in step S1612), the process proceeds to step S1615.

[0194] In step S1615, the first determination logic 1503 determines whether a state where the recognition result is not a fueling permission has continued for a predetermined time T3 (where T3 > T1).

[0195] In step S1615, if it is determined that the predetermined time T3 has not elapsed continuously (if NO in step S1615), the process returns to step S1611. On the other hand, if it is determined in step S1615 that the predetermined time T3 has elapsed continuously (if YES in step S1615), it is determined that the predetermined non - permission condition has been satisfied for a certain period, and the process proceeds to step S1616.

[0196] In step S1616, the first determination logic 1503 determines fueling non - permission and acquires the first information. Then, the first fueling monitoring unit 1009_1 ends the process using the first determination logic 1503.

[0197] On the other hand, when proceeding from step S1605 in FIG. 16A to step S1617 in FIG. 16B, the first determination logic 1503 determines that the predetermined non - permission condition has been satisfied for a certain period. Specifically, the first determination logic 1503 determines that the vehicle has not been recognized in the first area 210. Alternatively, the first determination logic 1503 determines that a fueler who is not holding the fueling nozzle in an appropriate posture has been recognized (fueler recognition impossible) in the second area 220.

[0198] In step S1618, the first determination logic 1503 determines that refueling is not permitted. After acquiring the second information, the first refueling monitoring unit 1009_1 ends the process using the first determination logic 1503.

[0199] (3-3) Flow of the process using the second determination logic 1504 of the first refueling monitoring unit 1009_1 Next, the flow of the process using the second determination logic 1504 of the first refueling monitoring unit 1009_1 will be described with reference to FIGS. 17A to 17D. FIGS. 17A to 17D are examples of the first to fourth flowcharts showing the flow of the process using the second determination logic of the first refueling monitoring unit.

[0200] In step S1701, the second determination logic 1504 determines whether the currently input recognition result is a recognition result determined to permit refueling (whether a predetermined permission condition is satisfied). Specifically, it is determined whether a vehicle is recognized in the first area 210 and whether a refueler holding the fuel nozzle in an appropriate posture is recognized in the second area.

[0201] If it is determined in step 1701 that the recognition result is a recognition result determined to permit refueling (if YES in step S1701), it is determined that the predetermined permission condition is satisfied, and the process proceeds to step S1702.

[0202] In step S1702, the second determination logic 1504 determines to permit refueling and proceeds to step S1734 in FIG. 17D.

[0203] On the other hand, if it is determined in step S1701 that the recognition result is not a recognition result determined to permit refueling (if NO in step S1701), the process proceeds to step S1703.

[0204] In step S1703, the second determination logic 1504 determines whether the state where the recognition result is not a recognition result determined to permit refueling has continued for a predetermined time T4.

[0205] In step S1703, if it is determined that the predetermined time T4 has not elapsed continuously (if NO in step S1703), the process returns to step S1701. On the other hand, in step S1703, if it is determined that the predetermined time T4 has elapsed continuously (if YES in step S1703), the process proceeds to step S1704.

[0206] In step S1704, the second determination logic 1504 determines whether the currently input recognition result is a recognition result determined as refueling not permitted (whether a predetermined non - permission condition is satisfied). Specifically, it determines whether a vehicle is recognized in the first area 210 and whether a refueler who is not holding the fuel nozzle in an appropriate posture is recognized in the second area (whether the fuel nozzle cannot be recognized).

[0207] In step S1704, if it is determined that the recognition result is another recognition result determined as refueling not permitted (if NO in step S1704), the process proceeds to step S1721 in FIG. 17C.

[0208] On the other hand, in step S1704, if it is determined that the recognition result is a recognition result determined as refueling not permitted (if YES in step S1704), it is determined that the predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S1705.

[0209] In step S1705, the second determination logic 1504 acquires warning information.

[0210] In step S1706, the second determination logic 1504 determines whether the currently input recognition result is a recognition result determined as refueling permitted (whether a predetermined permission condition is satisfied). Specifically, it determines whether a vehicle is recognized in the first area 210 and whether a refueler who is holding the fuel nozzle in an appropriate posture is recognized in the second area.

[0211] If it is determined in step S1706 that the recognition result is a determination of fueling permission (if YES in step S1706), the process proceeds to step S1707.

[0212] In step S1707, the second determination logic 1504 determines whether the recognition result determined to be fueling permission has continued for a predetermined time T1. If it is determined in step S1707 that it has continued for the predetermined time T1 (if YES in step S1707), it is determined that the predetermined permission conditions have been satisfied for a certain period of time, and the process proceeds to step S1708.

[0213] In step S1708, the second determination logic 1504 determines fueling permission and proceeds to step S1734 in FIG. 17D.

[0214] On the other hand, if it is determined in step S1706 that the recognition result is not a determination of fueling permission (if NO in step S1706), the process proceeds to step S1709. Or, if it is determined in step S1707 that it has not continued for the predetermined time T1 (if NO in step S1707), the process proceeds to step S1709.

[0215] In step S1709, the second determination logic 1504 determines whether the state where the recognition result is not a determination of fueling permission has continued for a predetermined time T2 (where T2 > T1).

[0216] If it is determined in step S1709 that it has not continued for the predetermined time T2 (if NO in step S1709), the process returns to step S1706. On the other hand, if it is determined in step S1709 that it has continued for the predetermined time T2 (if YES in step S1709), it is determined that the predetermined non - permission conditions have been satisfied for a certain period of time, and the process proceeds to step S1711 in FIG. 17B.

[0217] In step S1711, the second determination logic 1504 determines fueling non - permission, acquires the first information, and then proceeds to step S1712.

[0218] In step S1712, the second determination logic 1504 determines whether, within a predetermined time T2, the currently input recognition result has become a recognition result determined to be refueling permission (whether a predetermined permission condition is satisfied). Specifically, it is determined whether, within the predetermined time T2, a vehicle is recognized in the first area 210 and a refueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0219] In step S1712, if it is determined that the currently input recognition result has become a recognition result determined to be refueling permission within the predetermined time T2 (if YES in step S1712), it is determined that the predetermined permission condition is satisfied, and the process proceeds to step S1713.

[0220] In step S1713, the second determination logic 1504 determines that refueling is permitted and proceeds to step S1734 in FIG. 17D.

[0221] On the other hand, in step S1712, if it is determined that the recognition result has not become a recognition result determined to be refueling permission within the predetermined time T2 (if NO in step S1712), the first refueling monitoring unit 1009_1 ends the process using the second determination logic 1504.

[0222] On the other hand, when the process proceeds from step S1704 to step S1721 in FIG. 17C, the second determination logic 1504 determines that a predetermined non - permission condition has been satisfied for a certain period of time. Specifically, the second determination logic 1504 determines that a vehicle is not recognized in the first area 210. Alternatively, the second determination logic 1504 determines that a refueler who does not hold the fuel nozzle in an appropriate posture (refueler recognition impossible) is recognized in the second area 220.

[0223] In step S1722, the second determination logic 1504 acquires warning information.

[0224] In step S1723, the second determination logic 1504 determines whether the currently input recognition result is a recognition result determined to be fueling permission (whether it satisfies a predetermined permission condition). Specifically, it determines whether a vehicle is recognized in the first area 210 and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0225] In step S1723, if it is determined that the recognition result is a recognition result determined to be fueling permission (if YES in step S1723), the process proceeds to step S1724.

[0226] In step S1724, the second determination logic 1504 determines whether the recognition result determined to be fueling permission has continued for a predetermined time T1. In step S1724, if it is determined that it has continued for the predetermined time T1 (if YES in step S1724), it is determined that the predetermined permission condition has been satisfied for a certain period of time, and the process proceeds to step S1725.

[0227] In step S1725, the second determination logic 1504 determines that fueling is permitted and proceeds to step S1734 in FIG. 17D.

[0228] On the other hand, in step S1723, if it is determined that the recognition result is not a recognition result determined to be fueling permission (if NO in step S1723), the process proceeds to step S1726. Or, in step S1724, if it is determined that it has not continued for the predetermined time T1 (if NO in step S1724), the process proceeds to step S1726.

[0229] In step S1726, the second determination logic 1504 determines whether the state where the recognition result is not a recognition result determined to be fueling permission has continued for a predetermined time T2 (where T2 > T1).

[0230] In step S1726, if it is determined that the predetermined time T2 has not elapsed continuously (if NO in step S1726), the process returns to step S1723. On the other hand, in step S1726, if it is determined that the predetermined time T2 has elapsed continuously (if YES in step S1726), it is determined that a predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S1731 in FIG. 17D.

[0231] In step S1731, the second determination logic 1504 determines non - fueling permission. After acquiring the second information, the process proceeds to step S1732.

[0232] In step S1732, the second determination logic 1504 determines whether the currently input recognition result has become a recognition result for which fueling permission is determined (whether a predetermined permission condition has been satisfied) within the predetermined time T2. Specifically, it is determined whether, within the predetermined time T2, the vehicle is recognized in the first area 210 and a fueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0233] In step S1732, if it is determined that the currently input recognition result has become a recognition result for which fueling permission is determined within the predetermined time T2 (if YES in step S1732), it is determined that a predetermined permission condition has been satisfied, and the process proceeds to step S1733.

[0234] In step S1733, the second determination logic 1504 determines fueling permission and proceeds to step S1734.

[0235] In step S1734, the determination logic switching unit 1502 determines whether it is during the second section. In step S1734, if it is determined that it is during the second section, it is determined that the process by the second determination logic 1504 is not terminated (if determined NO in step S1734), and the process returns to step S1701 in FIG. 17A.

[0236] On the other hand, in step S1734, when it is determined that it is not during the second period, it is determined to end the process by the second determination logic 1504 (determined as YES in step S1734). Then, the first fuel supply monitoring unit 1009_1 ends the process using the second determination logic 1504.

[0237] Also, in step S1732, when it is not determined that the recognition result is a result of determining fuel supply permission within the predetermined time T2 (when it is NO in step S1732), the first fuel supply monitoring unit 1009_1 ends the process using the second determination logic 1504.

[0238] (3-4) Flow of the process using the third determination logic 1505 of the first fuel supply monitoring unit 1009_1 Next, the flow of the process using the third determination logic 1505 of the first fuel supply monitoring unit 1009_1 will be described with reference to FIGS. 18A and 18B. FIGS. 18A and 18B are examples of the first and second flowcharts showing the flow of the process using the third determination logic of the first fuel supply monitoring unit.

[0239] In step S1801, the third determination logic 1505 determines whether the currently input recognition result is a recognition result of determining fuel supply permission (whether the predetermined permission conditions are satisfied). Specifically, it is determined whether a vehicle is recognized in the first area 210 and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0240] In step 1801, when it is determined that the recognition result is a recognition result of determining fuel supply permission (when it is YES in step S1801), it is determined that the predetermined permission conditions are satisfied, and the process proceeds to step S1802.

[0241] In step S1802, the third determination logic 1505 determines fuel supply permission and proceeds to step S1813 in FIG. 18B.

[0242] On the other hand, if it is determined in step S1801 that the recognition result is not the one determined as refueling permission (if NO in step S1801), the process proceeds to step S1803.

[0243] In step S1803, the third determination logic 1505 determines whether the state where the recognition result is not the one determined as refueling permission has continued for a predetermined time T5.

[0244] In step S1803, if it is determined that it has not continued for the predetermined time T5 (if NO in step S1803), the process returns to step S1801. On the other hand, in step S1803, if it is determined that it has continued for the predetermined time T5 (if YES in step S1803), the process proceeds to step S1804.

[0245] In step S1804, the third determination logic 1505 determines whether the currently input recognition result is the one determined as refueling non - permission (whether a predetermined non - permission condition is satisfied). Specifically, it determines whether a vehicle is recognized in the first area 210 and whether a refueler who is not holding the fuel nozzle in an appropriate posture is recognized (whether the fuel nozzle is unrecognizable) in the second area.

[0246] In step S1804, if it is determined that the recognition result is other than the one determined as refueling non - permission (if NO in step S1804), the process proceeds to step S1806.

[0247] In step S1806, the third determination logic 1505 determines that a predetermined non - permission condition has been satisfied for a certain period. Specifically, the third determination logic 1505 determines that no vehicle is recognized in the first area 210. Alternatively, the third determination logic 1505 determines that a refueler who is not holding the fuel nozzle in an appropriate posture is recognized (the refueler is unrecognizable) in the second area 220.

[0248] In step S1807, the third determination logic 1505 determines that refueling is not permitted. After acquiring the second information, it proceeds to step S1811 in FIG. 18B.

[0249] On the other hand, in step S1804, if it is determined that the recognition result is a recognition result indicating that refueling is not permitted (if YES in step S1804), it is determined that a predetermined non-permission condition has been satisfied for a certain period of time, and the process proceeds to step S1805.

[0250] In step S1805, the third determination logic 1505 determines that refueling is not permitted. After acquiring the first information, it proceeds to step S1811 in FIG. 18B.

[0251] In step S1811, the third determination logic 1505 determines whether the currently input recognition result has become a recognition result indicating that refueling is permitted (whether a predetermined permission condition has been satisfied) within a predetermined time T2. Specifically, it is determined whether, within the predetermined time T2, a vehicle is recognized in the first area 210 and a refueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0252] In step S1811, if it is determined that the currently input recognition result has become a recognition result indicating that refueling is permitted within the predetermined time T2 (if YES in step S1811), it is determined that a predetermined permission condition has been satisfied, and the process proceeds to step S1812.

[0253] In step S1812, the third determination logic 1505 determines that refueling is permitted and proceeds to step S1813.

[0254] In step S1813, the determination logic switching unit 1502 determines whether it is during the third section. In step S1813, if it is determined that it is during the third section, it is determined that the process by the third determination logic 1505 is not terminated (if determined as NO in step S1813), and the process returns to step S1801 in FIG. 18A.

[0255] On the other hand, in step S1813, when it is determined that it is not during the third period, it is determined to end the process by the third determination logic 1505 (determined as YES in step S1813). Then, the first fuel supply monitoring unit 1009_1 ends the process using the third determination logic 1505.

[0256] On the other hand, in step S1811, when it is determined that the currently input recognition result does not become a recognition result determined as fuel supply permission within the predetermined time T2 (when it is NO in step S1811), the process proceeds to step S1814.

[0257] In step S1814, the third determination logic 1505 determines whether a state where the recognition result is not determined as fuel supply permission has continued for the predetermined time T6. In step S1814, when it is determined that a state where the recognition result is not determined as fuel supply permission has not continued for the predetermined time T6 (when it is NO in step S1814), it waits until it continues for the predetermined time T6.

[0258] On the other hand, in step S1814, when it is determined that a state where the recognition result is not determined as fuel supply permission has continued for the predetermined time T6 (when it is YES in step S1814), the process proceeds to step S1815.

[0259] In step S1815, the third determination logic 1505 acquires confirmation request information, and the first fuel supply monitoring unit 1009_1 ends the process using the third determination logic 1505.

[0260] (3-5) Flow of processing using other determination logics of the first fuel supply monitoring unit 1009_1 Next, the flow of processing using other determination logics of the first fuel supply monitoring unit 1009_1 (processing using the poly-tank determination unit 1506 and processing using the fire determination unit 1507) will be described with reference to FIG. 19. FIG. 19 is an example of a flowchart showing the flow of processing using the poly-tank determination unit of the first fuel supply monitoring unit and the flow of processing using the fire determination unit.

[0261] First, the processing flow using the poly-tank determination unit 1506 of the first fuel supply monitoring unit 1009_1 will be described.

[0262] In step S1901 of FIG. 19(a), the poly-tank determination unit 1506 determines whether a container is included (whether a predetermined non-permission condition is satisfied) in the currently input recognition result. If it is determined in step S1901 that a container is included (if YES in step S1901), it is determined that the predetermined non-permission condition is satisfied, and the process proceeds to step S1902.

[0263] In step S1902, the poly-tank determination unit 1506 determines that fuel supply is not permitted, acquires the third information, and then proceeds to step S1903.

[0264] On the other hand, if it is determined in step S1901 that a container is not included (if NO in step S1901), the process directly proceeds to step S1903.

[0265] In step S1903, the poly-tank determination unit 1506 determines whether to end the process. If it corresponds to any of the first section to the third section, in step S1903, the poly-tank determination unit 1506 determines to continue the process (determines NO in step S1903) and returns to step S1901.

[0266] On the other hand, if it does not correspond to any of the first section to the third section, in step S1903, the first fuel supply monitoring unit 1009_1 ends the process using the poly-tank determination unit 1506.

[0267] Subsequently, the processing flow using the poly-tank determination unit 1506 of the second fuel supply monitoring unit 1009_2 will be described.

[0268] In step S1911 of FIG. 19(b), the fire determination unit 1507 determines whether a temperature equal to or higher than a predetermined threshold has been measured in a region other than the first region 210 in the currently input thermal sensor data (whether a predetermined non-permissible condition is satisfied). Note that the region other than the first region 210 is set to avoid misjudging fire detection because the bonnet of the vehicle included in the first region 210 may be at a high temperature.

[0269] If it is determined in step S1911 that a temperature equal to or higher than the predetermined threshold has been measured (if YES in step S1911), it is determined that the predetermined non-permissible condition is satisfied, and the process proceeds to step S1912.

[0270] In step S1912, the fire determination unit 1507 determines that refueling is not permitted, acquires the fourth information, and then proceeds to step S1913.

[0271] On the other hand, if it is determined in step S1911 that a temperature equal to or higher than the predetermined threshold has not been measured (if NO in step S1911), the process directly proceeds to step S1913.

[0272] In step S1913, the fire determination unit 1507 determines whether to end the process. If it corresponds to any of the first to third intervals, in step S1913, the fire determination unit 1507 determines to continue the process (determines NO in step S1913) and returns to step S1911.

[0273] On the other hand, if it does not correspond to any of the first to third intervals, in step S1913, the first fuel supply monitoring unit 1009_1 ends the process using the fire determination unit 1507.

[0274] (3-6) Flow of processing using the first determination logic 1513 of the second fuel supply monitoring unit 1009_2 Next, the flow of the process using the first determination logic 1513 of the second fuel supply monitoring unit 1009_2 will be described with reference to FIGS. 20A and 20B. FIGS. 20A and 20B are examples of the first and second flowcharts showing the flow of the process using the first determination logic of the second fuel supply monitoring unit.

[0275] In step S2001, the first determination logic 1513 determines whether the currently input recognition result is a recognition result determined to be fuel supply permission (whether a predetermined permission condition is satisfied). Specifically, it is determined whether a container is recognized and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized.

[0276] If it is determined in step S2001 that the recognition result is a recognition result determined to be fuel supply permission (if YES in step S2001), the process proceeds to step S2002.

[0277] In step S2002, the first determination logic 1513 determines whether the recognition result determined to be fuel supply permission has continued for a predetermined time T1 (whether a predetermined permission condition has been satisfied for a certain time). If it is determined in step S2002 that it has continued for the predetermined time T1 (if YES in step S2002), the process proceeds to step S2003.

[0278] In step S2003, the first determination logic 1513 determines fuel supply permission and proceeds to step S2014 in FIG. 20B.

[0279] On the other hand, if it is determined in step S2001 that the recognition result is not a recognition result determined to be fuel supply permission (if NO in step S2001), the process proceeds to step S2004. Or, if it is determined in step S2002 that it has not continued for the predetermined time T1 (if NO in step S2002), the process proceeds to step S2004.

[0280] In step S2004, the first determination logic 1513 determines whether a state where the recognition result is not determined as refueling permission has continued for a predetermined time T2 (where T2 > T1).

[0281] In step S2004, if it is determined that the predetermined time T2 has not continued (if NO in step S2004), the process returns to step S2001. On the other hand, in step S2004, if it is determined that the predetermined time T2 has continued (if YES in step S2004), the process proceeds to step S2005.

[0282] In step S2005, the first determination logic 1513 determines whether the currently input recognition result is a recognition result determined as refueling not permitted (whether a predetermined non - permission condition is satisfied). Specifically, it determines whether a container is recognized and whether a refueler who is not holding the refueling nozzle in an appropriate posture is recognized (whether the refueling nozzle cannot be recognized).

[0283] In step S2005, if it is determined that the recognition result is a recognition result determined as refueling not permitted (if YES in step S2005), it is determined that the predetermined non - permission condition has been satisfied for a certain time, and the process proceeds to step S2006.

[0284] In step S2006, the first determination logic 1513 determines that refueling is not permitted, acquires the first information, and then proceeds to step S2011 in FIG. 20B.

[0285] On the other hand, in step S2005, if it is determined that the recognition result is another recognition result determined as refueling not permitted (if NO in step S2005), the process proceeds to step S2017 in FIG. 20B.

[0286] In step S2011, the first determination logic 1513 determines whether the currently input recognition result is a recognition result determined as fueling permission (whether it satisfies a predetermined permission condition). Specifically, it determines whether a container is recognized and whether a fueler who holds the fueling nozzle in an appropriate posture is recognized.

[0287] If it is determined in step S2011 that the recognition result is not a recognition result determined as fueling permission (if NO in step S2011), the process proceeds to step S2017.

[0288] On the other hand, if it is determined in step S2011 that the recognition result is a recognition result determined as fueling permission (if YES in step S2011), the second fueling monitoring unit 1009_2 ends the process using the first determination logic 1513.

[0289] In step S2012, the first determination logic 1513 determines whether the recognition result determined as fueling permission has continued for a predetermined time T1 (whether the predetermined permission condition has been satisfied for a certain time). If it is determined in step S2012 that it has continued for the predetermined time T1 (if YES in step S2012), the process proceeds to step S2013.

[0290] In step S2013, the first determination logic 1513 determines fueling permission and proceeds to step S2014.

[0291] In step S2014, the determination logic switching unit 1512 determines whether it is within the first section. If it is determined in step S2014 that it is within the first section, it is determined that the process by the first determination logic 1513 is not ended (determined as NO in step S2014), and the process returns to step S2001 in FIG. 20A.

[0292] On the other hand, in step S2014, when it is determined that it is not during the first period, it is determined to end the process by the first determination logic 1513 (determined as YES in step S2014). Then, the second fuel supply monitoring unit 1009_2 ends the process using the first determination logic 1513.

[0293] On the other hand, in step S2012, when it is determined that the predetermined time T1 has not continued (when NO in step S2012), the process proceeds to step S2015.

[0294] In step S2015, the first determination logic 1513 determines whether a state where the recognition result is not a fuel supply permission has continued for a predetermined time T3 (however, T3 > T1).

[0295] In step S2015, when it is determined that it has not continued for the predetermined time T3 (when NO in step S2015), the process returns to step S2011. On the other hand, in step S2015, when it is determined that it has continued for the predetermined time T3 (when YES in step S2015), it is determined that a predetermined non - permission condition has been satisfied for a certain period. Then, the second fuel supply monitoring unit 1009_2 ends the process using the first determination logic 1513.

[0296] On the other hand, when the process proceeds from step S2005 in FIG. 20A to step S2017 in FIG. 20B, the first determination logic 1513 determines that a predetermined non - permission condition has been satisfied for a certain period. Specifically, the first determination logic 1513 determines that the container has not been recognized. Alternatively, the first determination logic 1513 determines that a fueler who is not holding the fuel nozzle in an appropriate posture has been recognized (fueler recognition impossible).

[0297] In step S2018, the first determination logic 1513 determines non - fuel - supply permission, acquires second information, and the second fuel supply monitoring unit 1009_2 ends the process using the first determination logic 1513.

[0298] (3-7) Flow of Processing Using the Second Determination Logic 1514 of the Second Fuel Supply Monitoring Unit 1009_2 Next, the flow of processing using the second determination logic 1514 of the second fuel supply monitoring unit 1009_2 will be described with reference to FIGS. 21A to 21D. FIGS. 21A to 21D are examples of the first to fourth flowcharts showing the flow of processing using the second determination logic of the second fuel supply monitoring unit.

[0299] In step S2101, the second determination logic 1514 determines whether the currently input recognition result is a recognition result determined to be fuel supply permission (whether a predetermined permission condition is satisfied). Specifically, it is determined whether a container is recognized and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized.

[0300] If it is determined in step 2101 that the recognition result is a recognition result determined to be fuel supply permission (if YES in step S2101), it is determined that the predetermined permission condition is satisfied, and the process proceeds to step S2102.

[0301] In step S2102, the second determination logic 1514 determines fuel supply permission and proceeds to step S2134 in FIG. 21D.

[0302] On the other hand, if it is determined in step S2101 that the recognition result is not a recognition result determined to be fuel supply permission (if NO in step S2101), the process proceeds to step S2103.

[0303] In step S2103, the second determination logic 1514 determines whether the state where the recognition result is not determined to be fuel supply permission has continued for a predetermined time T4.

[0304] If it is determined in step S2103 that it has not continued for the predetermined time T4 (if NO in step S2103), the process returns to step S2101. On the other hand, if it is determined in step S2103 that it has continued for the predetermined time T4 (if YES in step S2103), the process proceeds to step S2104.

[0305] In step S2104, the second determination logic 1514 determines whether the currently input recognition result is a recognition result determined as refueling not permitted (whether a predetermined non-permission condition is satisfied). Specifically, it determines whether a container is recognized and whether a refueler who is not holding the fuel nozzle in an appropriate posture is recognized (whether the fuel nozzle cannot be recognized).

[0306] In step S2104, if it is determined that the recognition result is another recognition result determined as refueling not permitted (if NO in step S2104), the process proceeds to step S2121 in FIG. 21C.

[0307] On the other hand, in step S2104, if it is determined that the recognition result is a recognition result determined as refueling not permitted (if YES in step S2104), it is determined that a predetermined non-permission condition has been satisfied for a certain period of time, and the process proceeds to step S2105.

[0308] In step S2105, the second determination logic 1514 acquires warning information.

[0309] In step S2106, the second determination logic 1514 determines whether the currently input recognition result is a recognition result determined as refueling permitted (whether a predetermined permission condition is satisfied). Specifically, it determines whether a container is recognized and whether a refueler who is holding the fuel nozzle in an appropriate posture is recognized.

[0310] In step S2106, if it is determined that the recognition result is a recognition result determined as refueling permitted (if YES in step S2106), the process proceeds to step S2107.

[0311] In step S2107, the second determination logic 1514 determines whether the recognition result determined as fueling permission has continued for a predetermined time T1. In step S2107, if it is determined that it has continued for the predetermined time T1 (if YES in step S2107), it is determined that the predetermined permission conditions have been satisfied for a certain period of time, and the process proceeds to step S2108.

[0312] In step S2108, the second determination logic 1514 determines that fueling is permitted and proceeds to step S2134 in FIG. 21D.

[0313] On the other hand, in step S2106, if it is determined that the recognition result is not the one determined as fueling permission (if NO in step S2106), the process proceeds to step S2109. Or, in step S2107, if it is determined that it has not continued for the predetermined time T1 (if NO in step S2107), the process proceeds to step S2109.

[0314] In step S2109, the second determination logic 1514 determines whether the state where the recognition result is not the one determined as fueling permission has continued for a predetermined time T2 (where T2 > T1).

[0315] In step S2109, if it is determined that it has not continued for the predetermined time T2 (if NO in step S2109), the process returns to step S2106. On the other hand, in step S2109, if it is determined that it has continued for the predetermined time T2 (if YES in step S2109), it is determined that the predetermined non - permission conditions have been satisfied for a certain period of time, and the process proceeds to step S2111 in FIG. 21B.

[0316] In step S2111, the second determination logic 1514 determines that fueling is not permitted, acquires the first information, and then proceeds to step S2112.

[0317] In step S2112, the second determination logic 1514 determines whether, within a predetermined time T2, the currently input recognition result has become a recognition result determined as fueling permission (whether a predetermined permission condition is satisfied). Specifically, it is determined whether, within the predetermined time T2, a container has been recognized and a fueler who holds the fuel nozzle in an appropriate posture has been recognized.

[0318] In step S2112, if it is determined that, within the predetermined time T2, the currently input recognition result has become a recognition result determined as fueling permission (if YES in step S2112), it is determined that the predetermined permission condition is satisfied, and the process proceeds to step S2113.

[0319] In step S2113, the second determination logic 1514 determines that fueling is permitted and proceeds to step S2134 in FIG. 21D.

[0320] On the other hand, in step S2112, if it is determined that, within the predetermined time T2, the recognition result has not become a recognition result determined as fueling permission (if NO in step S2112), the second fueling monitoring unit 1009_2 ends the process using the second determination logic 1514.

[0321] On the other hand, when the process proceeds from step S2104 to step S2121 in FIG. 21C, the second determination logic 1514 determines that a predetermined non - permission condition has been satisfied for a certain period of time. Specifically, the second determination logic 1514 determines that the container has not been recognized. Alternatively, the second determination logic 1514 determines that a fueler who does not hold the fuel nozzle in an appropriate posture has been recognized (fueler recognition impossible).

[0322] In step S2122, the second determination logic 1504 acquires warning information.

[0323] In step S2123, the second determination logic 1514 determines whether the currently input recognition result is a recognition result determined to be refueling permitted (whether it satisfies a predetermined permission condition). Specifically, it determines whether a container is recognized and whether a refueler who holds the fuel nozzle in an appropriate posture is recognized.

[0324] In step S2123, if it is determined that the recognition result is a recognition result determined to be refueling permitted (if YES in step S2123), the process proceeds to step S2124.

[0325] In step S2124, the second determination logic 1514 determines whether the recognition result determined to be refueling permitted has continued for a predetermined time T1. In step S2124, if it is determined that it has continued for the predetermined time T1 (if YES in step S2124), it is determined that the predetermined permission condition has been satisfied for a certain period of time, and the process proceeds to step S2125.

[0326] In step S2125, the second determination logic 1514 determines that refueling is permitted and proceeds to step S2134 in FIG. 21D.

[0327] On the other hand, if it is determined in step S2123 that the recognition result is not a recognition result determined to be refueling permitted (if NO in step S2123), the process proceeds to step S2126. Or, if it is determined in step S2124 that it has not continued for the predetermined time T1 (if NO in step S2124), the process proceeds to step S2126.

[0328] In step S2126, the second determination logic 1514 determines whether the state where the recognition result is not a recognition result determined to be refueling permitted has continued for a predetermined time T2 (where T2 > T1).

[0329] In step S2126, if it is determined that the predetermined time T2 has not elapsed continuously (if NO in step S2126), the process returns to step S2123. On the other hand, in step S2126, if it is determined that the predetermined time T2 has elapsed continuously (if YES in step S2126), it is determined that a predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S2131 in FIG. 21D.

[0330] In step S2131, the second determination logic 1514 determines that refueling is not permitted. After acquiring the second information, the process proceeds to step S2132.

[0331] In step S2132, the second determination logic 1514 determines whether the currently input recognition result has become a recognition result for which refueling is permitted (whether a predetermined permission condition has been satisfied) within the predetermined time T2. Specifically, it determines whether, within the predetermined time T2, the container has been recognized and a refueler who holds the fuel nozzle in an appropriate posture has been recognized.

[0332] In step S2132, if it is determined that the currently input recognition result has become a recognition result for which refueling is permitted within the predetermined time T2 (if YES in step S2132), it is determined that a predetermined permission condition has been satisfied, and the process proceeds to step S2133.

[0333] In step S2133, the second determination logic 1514 determines that refueling is permitted, and the process proceeds to step S2134.

[0334] In step S2134, the determination logic switching unit 1512 determines whether it is within the second section. In step S2134, if it is determined that it is within the second section, it is determined that the process by the second determination logic 1514 is not terminated (if determined NO in step S2134), and the process returns to step S2101 in FIG. 21A.

[0335] On the other hand, in step S2134, when it is determined that it is not during the second period, it is determined to end the process by the second determination logic 1514 (determined as YES in step S2134). Then, the second fuel supply monitoring unit 1009_2 ends the process using the second determination logic 1514.

[0336] Also, in step S2132, when it is not determined that the recognition result is a fuel supply permission determination within the predetermined time T2 (NO in step S2132), the second fuel supply monitoring unit 1009_2 ends the process using the second determination logic 1514.

[0337] (3-8) Flow of the process using the third determination logic 1515 of the second fuel supply monitoring unit 1009_2 Next, the flow of the process using the third determination logic 1515 of the second fuel supply monitoring unit 1009_2 will be described with reference to FIGS. 22A and 22B. FIGS. 22A and 22B are examples of the first and second flowcharts showing the flow of the process using the third determination logic of the second fuel supply monitoring unit.

[0338] In step S2201, the third determination logic 1515 determines whether the currently input recognition result is a recognition result for which fuel supply is permitted (whether a predetermined permission condition is satisfied). Specifically, it is determined whether a container is recognized and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized.

[0339] In step 2201, when it is determined that the recognition result is a recognition result for which fuel supply is permitted (YES in step S2201), it is determined that the predetermined permission condition is satisfied, and the process proceeds to step S2202.

[0340] In step S2202, the third determination logic 1515 determines that fuel supply is permitted and proceeds to step S2213 in FIG. 22B.

[0341] On the other hand, if it is determined in step S2201 that the recognition result is not the one for which refueling is permitted (if NO in step S2201), the process proceeds to step S2203.

[0342] In step S2203, the third determination logic 1515 determines whether the state where the recognition result is not the one for which refueling is permitted has continued for a predetermined time T5.

[0343] In step S2203, if it is determined that it has not continued for the predetermined time T5 (if NO in step S2203), the process returns to step S2201. On the other hand, in step S2203, if it is determined that it has continued for the predetermined time T5 (if YES in step S2203), the process proceeds to step S2204.

[0344] In step S2204, the third determination logic 1515 determines whether the currently input recognition result is the one for which refueling is not permitted (whether a predetermined non - permission condition is satisfied). Specifically, it determines whether a container is recognized and whether a refueler who is not holding the fuel nozzle in an appropriate posture is recognized (whether the fuel nozzle cannot be recognized).

[0345] In step S2204, if it is determined that the recognition result is the other one for which refueling is not permitted (if NO in step S2204), the process proceeds to step S2206.

[0346] In step S2206, the third determination logic 1515 determines that the predetermined non - permission condition has been satisfied for a certain period of time. Specifically, the third determination logic 1515 determines that the container has not been recognized. Alternatively, the third determination logic 1515 determines that a refueler who is not holding the fuel nozzle in an appropriate posture has been recognized (the refueler cannot be recognized).

[0347] In step S2207, the third determination logic 1515 determines that refueling is not permitted, acquires the second information, and then proceeds to step S2211 in FIG. 22B.

[0348] On the other hand, in step S2204, if it is determined that the recognition result is a refueling non - permission determination result (if YES in step S2204), it is determined that a predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S2205.

[0349] In step S2205, the third determination logic 1515 determines refueling non - permission, acquires the first information, and then proceeds to step S2211 in FIG. 22B.

[0350] In step S2211, the third determination logic 1515 determines whether the currently input recognition result has become a recognition result for which refueling is permitted (whether a predetermined permission condition has been satisfied) within a predetermined time T2. Specifically, it is determined whether, within the predetermined time T2, a container has been recognized and a refueler who holds the fuel nozzle in an appropriate posture has been recognized.

[0351] In step S2211, if it is determined that the currently input recognition result has become a recognition result for which refueling is permitted within the predetermined time T2 (if YES in step S2211), it is determined that a predetermined permission condition has been satisfied, and the process proceeds to step S2212.

[0352] In step S2212, the third determination logic 1515 determines refueling permission and proceeds to step S2213.

[0353] In step S2213, the determination logic switching unit 1512 determines whether it is within the third section. In step S2213, if it is determined that it is within the third section, it is determined that the process by the third determination logic 1515 is not terminated (if determined NO in step S2213), and the process returns to step S2201 in FIG. 22A.

[0354] On the other hand, in step S2213, if it is determined that it is not within the third interval, it is determined to end the process by the third determination logic 1515 (determined as YES in step S2213). Then, the second fuel supply monitoring unit 1009_2 ends the process using the third determination logic 1515.

[0355] On the other hand, in step S2211, if it is determined that the currently input recognition result has not become a recognition result determined as fuel supply permission within the predetermined time T2 (if NO in step S2211), the process proceeds to step S2214.

[0356] In step S2214, the third determination logic 1515 determines whether the state of not being a recognition result determined as fuel supply permission has continued for the predetermined time T6. In step S2214, if it is determined that the state of not being a recognition result determined as fuel supply permission has not continued for the predetermined time T6 (if NO in step S2214), it waits until it continues for the predetermined time T6.

[0357] On the other hand, in step S2214, if it is determined that the state of not being a recognition result determined as fuel supply permission has continued for the predetermined time T6 (if YES in step S2214), the process proceeds to step S2215.

[0358] In step S2215, the third determination logic 1515 acquires confirmation request information, and the second fuel supply monitoring unit 1009_2 ends the process using the third determination logic 1515.

[0359] (3-9) Flow of processing using other determination logic of the second fuel supply monitoring unit 1009_2 Next, the flow of processing using other determination logic of the second fuel supply monitoring unit 1009_2 (processing using the fire determination unit 1517) will be described with reference to FIG. 23. FIG. 23 is an example of a flowchart showing the flow of processing using the fire determination unit of the second fuel supply monitoring unit.

[0360] In step S2301, the fire determination unit 1517 determines whether a temperature equal to or higher than a predetermined threshold has been measured for the currently input thermal sensor data (whether a predetermined non - permission condition is satisfied).

[0361] In step S2301, if it is determined that a temperature equal to or higher than the predetermined threshold has been measured (if YES in step S2301), it is determined that the predetermined non - permission condition is satisfied, and the process proceeds to step S2302.

[0362] In step S2302, the fire determination unit 1517 determines non - fueling permission, acquires the fourth information, and then proceeds to step S2303.

[0363] On the other hand, in step S2301, if it is determined that a temperature equal to or higher than the predetermined threshold has not been measured (if NO in step S2301), the process directly proceeds to step S2303.

[0364] In step S2303, the fire determination unit 1517 determines whether to end the process. If it corresponds to any of the first to third intervals, in step S2303, the fire determination unit 1517 determines to continue the process (determines NO in step S2303) and returns to step S2301.

[0365] On the other hand, if it does not correspond to any of the first to third intervals, in step S2303, the second fuel supply monitoring unit 1009_2 ends the process using the fire determination unit 1517.

[0366] (4) Details of the output control unit 1010 Next, as details of the output control unit 1010, the functional configuration and specific examples of the processes of the output control unit 1010 will be described.

[0367] (4 - 1) Details of the functional configuration of the output control unit 1010 FIG. 24 is a diagram showing an example of the functional configuration of the output control unit of the information processing apparatus. As shown in FIG. 24, the output control unit 1010 includes an output information acquisition unit 2401 and a transmission unit 2402.

[0368] The output information acquisition unit 2401 acquires from the first fueling monitoring unit 1009_1, · information indicating fueling permission, · information indicating non - permission for fueling and the first to fourth information, · warning information, · confirmation request information, and acquires them.

[0369] Also, the output information acquisition unit 2401 acquires from the second fueling monitoring unit 1009_2, · information indicating fueling permission, · information indicating non - permission for fueling and the first, second, and fourth information, · warning information, · confirmation request information, and acquires them.

[0370] Also, the output information acquisition unit 2401 notifies the transmission unit 2402 of the information acquired from the first fueling monitoring unit 1009_1. At this time, the output information acquisition unit 2401 refers to the output information storage unit 2403, · cause information (information indicating the cause of non - permission for fueling) corresponding to the first to fourth information, · messages corresponding to the first to fourth information, · display information corresponding to the first to fourth information, · messages corresponding to the warning information, · display information corresponding to the warning information, · cause information corresponding to the confirmation request information, · messages corresponding to the confirmation request information, · display information corresponding to the confirmation request information, and acquires them. Then, the output information acquisition unit 2401 converts the first to fourth information, warning information, and confirmation request information into the corresponding cause information, messages, and display information, and then notifies the transmission unit 2402.

[0371] Similarly, the output information acquisition unit 2401 notifies the information obtained from the second fuel supply monitoring unit 1009_2 to the transmission unit 2402. At this time, the output information acquisition unit 2401 refers to the output information storage unit 2403 and · Cause information corresponding to the first information, the second information, and the fourth information (information indicating the cause of fuel supply non - permission), and · Messages corresponding to the first information, the second information, and the fourth information, and · Display information corresponding to the first information, the second information, and the fourth information, and · Messages corresponding to the warning information, and · Display information corresponding to the warning information, and · Cause information corresponding to the confirmation request information, and · Messages corresponding to the confirmation request information, and · Display information corresponding to the confirmation request information, and acquires them. Then, the output information acquisition unit 2401 converts the first information, the second information, the fourth information, the warning information, and the confirmation request information into the corresponding cause information, messages, and display information, and then notifies the transmission unit 2402.

[0372] The transmission unit 2402 transmits the information notified from the output information acquisition unit 2401 to · The display device 912, which is a display device for the staff of the service station 110', · The voice output device 122_1 of the first fuel dispenser 120_1 or the voice output device 122_2 of the second fuel dispenser 120_2, · The fuel supply data display 123_1 of the first fuel dispenser 120_1 or the fuel supply data display 123_2 of the second fuel dispenser 120_2. Specifically, the transmission unit 2402 controls the information obtained from the first fuel supply monitoring unit 1009_1,

[0373] · Information indicating fuel supply permission, information indicating fuel supply non - permission, cause information, and confirmation request information to be displayed on the display device 912, · Messages to be voice - output by the voice output device 122_1 of the first fuel dispenser 120_1, · Messages to be voice - output by the voice output device 122_1 of the first fuel dispenser 120_1, ·Control so that the indication information is displayed on the fueling data display 123_1 of the first fuel dispenser 120_1.

[0374] Similarly, the transmission unit 2402 controls so that the information acquired from the second fuel monitoring unit 1009_2, ·Information indicating fueling permission, information indicating non - permission to fuel, cause information, and confirmation request information are controlled to be displayed on the display device 912, ·A message is controlled to be output as voice by the voice output device 122_2 of the second fuel dispenser 120_2, ·Control so that the display information is displayed on the fueling data display 123_2 of the second fuel dispenser 120_2.

[0375] (4 - 2) Specific example of the processing of the output control unit 1010 Next, as a specific example of the processing of the output control unit 1010, specific examples of the cause information, message, and display information converted by the output control unit 1010 will be described. FIG. 25 is a diagram showing an example of the output information.

[0376] In FIG. 25, the output information 2500 includes cause information, a message, and display information corresponding to the first to fourth information, warning information, and confirmation request information acquired from the first fuel monitoring unit 1009_1. Also, the output information 2510 includes cause information, a message, and display information corresponding to the first information, second information, fourth information, warning information, and confirmation request information acquired from the second fuel monitoring unit 1009_2.

[0377] As shown in FIG. 25, the output information 2500 and 2510 include, as information items, "information type", "cause information", "message output to the customer", and "display information on the fueling data display of the fuel dispenser".

[0378] In the case of the output information 2500, any one of the first to fourth information is stored in the "information type", and in the case of the output information 2510, any one of the first information, second information, and fourth information is stored.

[0379] In the case of output information 2500, any one of "nozzle indeterminate", "judgment impossible", "refueling from outside the vehicle", and "fire detection" is stored in the "cause information", and in the case of output information 2510, any one of "nozzle indeterminate", "judgment impossible", and "fire detection" is stored.

[0380] In the case of output information 2500, messages corresponding to the first to fourth information, which are output as voice from the voice output device 122_1, are stored in the "message output to the customer". Also, in the case of output information 2510, messages corresponding to the first, second, and fourth information, which are output as voice from the voice output device 122_2, are stored.

[0381] In the case of output information 2500, display information corresponding to the first to fourth information, which is displayed on the fueling data display 123_1 of the first fuel dispenser 120_1, is stored in the "display information of the fueling data display of the fuel dispenser". Also, in the case of output information 2510, display information corresponding to the first, second, and fourth information, which is displayed on the fueling data display 123_2 of the second fuel dispenser 120_2, is stored.

[0382] <Flow of fueling control process by information processing device> Next, the flow of the fueling control process by the fueling system 110S' according to the first embodiment will be described. FIG. 26 is an example of a flowchart showing the flow of the fueling control process by the fueling system according to the first embodiment.

[0383] In step S2601, the information processing device 160 acquires the image data captured by the cameras 130_1 and 130_2.

[0384] In step S2602, the information processing device 160 acquires the signals from the first fueling sensor 921_1 and the second fueling sensor 921_2.

[0385] In step S2603, the information processing device 160 acquires the thermal sensor data measured by the thermal sensors 170_1 and 170_2.

[0386] In step S2604, the information processing apparatus 160 determines which of the first section to the third section it corresponds to.

[0387] In step S2605, the information processing apparatus 160 determines whether to determine the fueling permission or not. In step S2605, if it is determined that it does not correspond to any of the first section to the third section, it is determined not to determine the fueling permission (determined as NO in step S2605), and the process proceeds to step S2610.

[0388] On the other hand, in step 2605, if it is determined that it corresponds to any of the first section to the third section, it is determined to determine the fueling permission (determined as YES in step S2605), and the process proceeds to step S2606.

[0389] In step S2606, the information processing apparatus 160 performs image recognition processing on the image data using the first learned model or the second learned model.

[0390] In step S2607, the information processing apparatus 160 determines the fueling permission or not from the recognition result using the determination logic corresponding to the determined section.

[0391] In step S2608, the information processing apparatus 160 controls the fueling of the first fuel dispenser 120_1 and the second fuel dispenser 120_2 based on the determination result of the fueling permission or not. Note that when the determination result is non - fueling permission, the information processing apparatus 160 controls the prohibition or stop of fueling by the first fuel dispenser 120_1 or the second fuel dispenser 120_2 according to the cause information of the non - fueling permission.

[0392] In step S2609, the information processing apparatus 160 outputs the determination result of the fueling permission or not. Also, when the determination result is non - fueling permission, the information processing apparatus 160 outputs the cause information of the non - fueling permission and the like.

[0393] In step S2610, the information processing apparatus 160 determines whether to end the fuel supply control process. If it is determined in step S2610 to continue the fuel supply control process (if NO in step S2610), the process returns to step S2601. On the other hand, if it is determined in step S2610 to end the fuel supply control process (if YES in step S2610), the fuel supply control process ends.

[0394] <Summary> As is clear from the above description, the fuel supply system 110S' according to the first embodiment divides the period from when the fuel supply nozzle is removed from the nozzle holder until it is returned to the nozzle holder into three sections, and determines whether fuel supply is permitted using determination logic corresponding to each section. Thus, according to the first embodiment, it is possible to provide a fuel supply system capable of executing processing according to the fuel supply scene.

[0395] Further, when determining not to permit fuel supply based on the recognition result regarding the recognition target, the fuel supply system 110S' according to the first embodiment is configured to acquire cause information for not permitting fuel supply and output a message corresponding to the cause information as voice. Thus, according to the first embodiment, it is possible to provide a fuel supply system capable of executing processing according to the fuel supply scene.

[0396] Further, when performing image recognition processing regarding the recognition target, the fuel supply system 110S' according to the first embodiment is configured to use a learned model corresponding to the type of fuel supply. Thus, according to the first embodiment, it is possible to provide a fuel supply system capable of executing processing according to the fuel supply scene.

[0397] [Second Embodiment] In the above-described first embodiment, the information processing apparatus 160 has been described as performing fueling control for the first fuel dispenser 120_1, the second fuel dispenser 120_2, and the like. However, the fueling control for the first fuel dispenser 120_1, the second fuel dispenser 120_2, and the like may be configured to be performed by a device separate from the information processing apparatus 160 (for example, a fueling controller such as a self-service console provided in a service station). Hereinafter, the second embodiment will be described centering on the differences from the first embodiment.

[0398] <Functional Configuration of Information Processing Apparatus> FIG. 27 is a diagram showing another example of the functional configuration of the information processing apparatus. The difference from the functional configuration of the information processing apparatus 160 shown in FIG. 10 is that, in the case of the information processing apparatus 160' shown in FIG. 27, the functions of the first fueling monitoring unit 1009_1' and the second fueling monitoring unit 1009_2' are different from the functions of the first fueling monitoring unit 1009_1 and the second fueling monitoring unit 1009_2. Another difference from the functional configuration of the information processing apparatus 160 shown in FIG. 10 is that, in the case of the information processing apparatus 160' shown in FIG. 27, the first fueling monitoring unit 1009_1' and the second fueling monitoring unit 1009_2' are communicably connected to the fueling controller 2710.

[0399] The fueling controller 2710 is a self-service console (SSC: Self Service Console), and controls the fueling of the first fuel dispenser 120_1 based on the determination result of whether fueling is permitted transmitted from the first fueling monitoring unit 1009_1'. Further, the fueling controller 2710 controls the fueling of the second fuel dispenser 120_2 based on the determination result of whether fueling is permitted transmitted from the second fueling monitoring unit 1009_2'.

[0400] <Details of First Fueling Monitoring Unit and Second Fueling Monitoring Unit> Next, the details of the first fuel supply monitoring unit 1009_1' and the second fuel supply monitoring unit 1009_2' will be described. FIG. 28A is a diagram showing another example of the functional configuration of the first fuel supply monitoring unit of the information processing apparatus. In the above first embodiment, the difference from the functional configuration of the first fuel supply monitoring unit 1009_1 of the information processing apparatus 160 described with reference to FIG. 15A is that the first fuel supply monitoring unit 1009_1' of the information processing apparatus 160' in FIG. 28A has a transmission unit 2801. Further, in the case of the first fuel supply monitoring unit 1009_1' of the information processing apparatus 160' in FIG. 28A, the transmission unit 2801 is communicably connected to the fuel supply controller 2710.

[0401] The transmission unit 2801 transmits the determination result of fuel supply permission or non - permission to the fuel supply controller 2710. Thereby, the fuel supply controller 2710 controls the fuel supply for the first fuel supply type by controlling the fuel supply valve 124_1 or the fuel supply pump 125_1.

[0402] For example, when the fuel supply permission is transmitted from the transmission unit 2801, the fuel supply controller 2710 drives the fuel supply pump 125_1 to open the fuel supply valve 124_1. Also, the fuel supply controller 2710 · When the fuel supply permission from the poly - tank determination unit 1506 is transmitted from the transmission unit 2801, the fuel supply controller 2710 drives the fuel supply pump 125_1 to open the fuel supply valve 124_1. Further, · When the fuel supply non - permission from any of the first determination logic 1503 to the third determination logic 1505 is transmitted from the transmission unit 2801, or · When the fuel supply non - permission from the fire determination unit 1507 is transmitted from the transmission unit 2801, it stops the fuel supply by stopping the drive of the fuel supply pump 125_1, or prohibits the fuel supply by stopping the drive of the fuel supply pump 125_1 and closing the fuel supply valve 124_1. Note that the definitions of fuel supply stop and fuel supply prohibition here are only examples, and the fuel supply stop and fuel supply prohibition may be defined by other definitions. Alternatively, without distinguishing between fuel supply stop and fuel supply prohibition, the entire state where fuel supply cannot be performed may be referred to as fuel supply stop or prohibition.

[0403] In addition, the transmission unit 2801 notifies the cause information acquisition unit 1509 of the determination results of fueling permission and fueling non - permission. When the transmission unit 2801 notifies of fueling non - permission, it also notifies the cause information acquisition unit 1509 of the information indicating the cause determined as fueling non - permission.

[0404] Note that when any of the first determination logic 1503 to the third determination logic 1505 is unable to perform the determination of fueling permission or non - permission, the transmission unit 2801 may be configured to output "determination impossible" to the cause information acquisition unit 1509. When "determination impossible" is output from the transmission unit 2801, the cause information acquisition unit 1509 may display "determination impossible" on, for example, the display device 912 which is a display device for the staff of the service station 110'. According to such a configuration, even when the determination of fueling permission or non - permission cannot be performed, the staff of the service station 110' can operate the fuel controller 2710 to control fueling.

[0405] In addition, the transmission unit 2801 may be configured to monitor the communication status with the fuel controller 2710 and output "communication abnormality" to the cause information acquisition unit 1509 when a communication failure occurs. When "communication abnormality" is output from the transmission unit 2801, the cause information acquisition unit 1509 may display "communication abnormality" on, for example, the display device 912 which is a display device for the staff of the service station 110'. According to such a configuration, even when the determination result of fueling permission or non - permission cannot be transmitted to the fuel controller 2710, the staff of the service station 110' can operate the fuel controller 2710 to control fueling.

[0406] FIG. 28B is a diagram showing an example of the functional configuration of the second fueling monitoring unit of the information processing device. Since the functional configuration of the second fueling monitoring unit 1009_2' shown in FIG. 28B is the same as that of the first fueling monitoring unit 1009_1' shown in FIG. 28A, detailed description is omitted here.

[0407] <Summary> As is clear from the above description, in the second embodiment, the functions of the information processing apparatus 160 in the first embodiment are realized by the information processing apparatus 160' and the fuel supply controller 2710. Thus, even when the information processing apparatus 160' and the fuel supply controller 2710 are configured separately, according to the second embodiment, the same effects as those in the first embodiment can be achieved.

[0408] [Third Embodiment] In the first embodiment, the case where the period from when the fuel supply nozzle is removed from the nozzle holder until it is returned to the nozzle holder is divided into three sections was described. However, the number of sections to be divided is not limited to three, and may be four or more.

[0409] Also, in the first embodiment, it was described that one determination logic is associated with each section. However, the relationship between the number of sections and the number of determination logics is not limited to this. For example, one determination logic may be associated with two consecutive sections.

[0410] Also, in the first embodiment, the case where there are two types of fuel supply was described, but the number of types of fuel supply may be three or more. Note that when there are three or more types of fuel supply, three or more learned models are also generated.

[0411] Also, in the first embodiment, for the first fuel supply type, the recognition targets were a vehicle, a container, a fuel supplier, a fuel supply nozzle, etc. However, the recognition targets for the first fuel supply type are not limited to these.

[0412] Similarly, in the first embodiment, for the second fuel supply type, the recognition targets were a container, a fuel supplier, a fuel supply nozzle, etc. However, the recognition targets for the second fuel supply type are not limited to these.

[0413] In the first embodiment described above, the first to third determination logics for the first fuel supply type were described with reference to FIGS. 16A to 18B. However, the processing contents of the first to third determination logics are not limited to these.

[0414] Similarly, in the first embodiment described above, the first to third determination logics for the second fuel supply type were described with reference to FIGS. 20A to 22B. However, the processing contents of the first to third determination logics are not limited to these.

[0415] In the first embodiment described above, the processing after displaying the cause information, for example, on the display device 912 for the staff of the service station 110' was not mentioned. However, after displaying the cause information, it may be switched to manual operation by the staff.

[0416] In the first embodiment described above, the number of executions of the learning process was not mentioned. However, the learning process may be executed multiple times. For example, in the image recognition process using the first learned model and the second learned model generated by the first learning process, if sufficient recognition accuracy cannot be obtained, or if the recognition accuracy decreases, additional learning processes may be configured to be performed.

[0417] For example, the learning device 150 may be configured to perform additional learning processes by collecting image data and signals indicating the states of the fuel nozzles again from a plurality of service stations and generating learning data.

[0418] Note that the learning device 150 may be configured to be able to collect image data and signals indicating the states of the fuel nozzles at any timing from a plurality of service stations. Also, a device that monitors the recognition accuracy, analyzes the cause when the recognition accuracy decreases, and notifies the learning device 150 may be connected to the network 180.

[0419] In the first embodiment described above, the learning device 150 and the information processing device 160 are configured as separate bodies, but the learning device 150 and the information processing device 160 may be configured as one body.

[0420] Note that the configurations and the like described in the above embodiments are not limited to the configurations shown here, such as combinations with other elements. Regarding these points, it is possible to make changes without departing from the spirit of the present invention, and they can be appropriately determined according to the application form.

Explanation of Reference Numerals

[0421] 110, 111: Service station 110S': Fuel supply system 120_1: First fuel dispenser 120_2: Second fuel dispenser 121_1, 121_2: Fuel nozzles 122_1, 122_2: Voice output devices 123_1, 123_3: Fueling data displays 124_1, 124_2: Fueling valves 125_1, 125_2: Fueling pumps 130_1, 130_2: Cameras 170_1, 170_2: Thermal sensors 150: Learning device 160: Information processing device 160': Information processing device 210: First area 220: Second area 211: Vehicle 221: Fueler 231: Container 241: Fueler 500A: Learning data generation unit 500B: Learning unit 600: First learning data 700: Second learning data 912: Display device 921_1: First fueling sensor 921_2: Second fueling sensor 1001: First fuel type image data acquisition unit 1002: Second fuel type image data acquisition unit 1003: First fueling sensor data acquisition unit 1004: Second fueling sensor data acquisition unit 1005: Image recognition unit 1006, 1007: Interval determination unit 1009_1: First fueling monitoring unit 1009_1': First fueling monitoring unit 1009_2: Second fueling monitoring unit 1009_2': Second fueling monitoring unit 1010: Output control unit 1302: First trained model 1312: Second trained model 1503, 1513: First judgment logic 1504, 1514: Second judgment logic 1505, 1515: Third judgment logic 2710: Fueling controller 2801, 2811: Transmission unit

Claims

1. A fuel dispenser, and an information processing device, comprising a fuel supply system, wherein the information processing device determines the state of the fuel nozzle and the state of fuel supply based on a notification from the fuel dispenser, and when it is determined that refueling is not permitted based on a determination logic according to which the refueling operation of the refueler is included in any one of a first section before the start of refueling, a second section during refueling, and a third section after the end of refueling, or based on a determination logic according to which it is included in any two consecutive sections, the fuel dispenser prohibits or stops refueling. A fuel supply system.

2. further comprising a state determination unit that determines the state of the fuel nozzle and the state of fuel supply based on a notification from the fuel dispenser, wherein when the state determination unit determines that the fuel nozzle has been removed from the nozzle holder, it determines that the first section has started, when it is determined that refueling is permitted based on the determination logic corresponding to the first section, and when the state determination unit determines that refueling has started, it determines that the second section has been entered, when the state determination unit determines that no fueling has been performed for a predetermined time, it determines that the third section has been entered, and when the state determination unit determines that the fuel nozzle has been returned to the nozzle holder, it determines that the third section has ended. The fuel supply system according to claim 1.

3. further comprising a camera that photographs the area including the fuel dispenser, wherein when the fuel type is gasoline or light oil, the determination logic performs recognition processing on the image data photographed by the camera, a recognition result indicating that a vehicle has been recognized, a recognition result indicating that a refueler holding the fuel nozzle in an appropriate posture has been recognized, a recognition result indicating that a refueler not holding the fuel nozzle in an appropriate posture has been recognized, when output, determines whether refueling is permitted based on the recognition result. The fuel supply system according to claim 1.

4. The first determination logic corresponding to the first section is such that when the recognition result satisfies the permission conditions for a certain period of time, it determines that refueling is permitted, when the predetermined non-permission conditions are satisfied for a certain period of time, it determines that refueling is not permitted, and then, when the permission conditions are satisfied for a certain period of time, it determines that refueling is permitted, when other non-permission conditions are satisfied for a certain period of time, it determines that refueling is not permitted, The second determination logic corresponding to the second section is such that when the recognition result satisfies the non-permission conditions for a certain period of time, it outputs a warning, and when the non-permission conditions are satisfied for a certain period of time even after the warning has been output, it determines that refueling is not permitted. If the permission conditions are satisfied for a certain period of time after a warning is output, it is determined that refueling is permitted, The third determination logic corresponding to the third section is such that the recognition result is If the permission conditions are satisfied for a certain period of time, it is determined that refueling is permitted, If the non-permission conditions are satisfied for a certain period of time, it is determined that refueling is not permitted, and then, if the permission conditions are satisfied for a certain period of time, it is determined that refueling is permitted, The refueling system according to claim 3.

5. The determination logic further When recognition processing is performed on the image data captured by the camera and a recognition result indicating that a predetermined container has been recognized is output, the refueling system according to claim 3 includes a process of determining that refueling is not permitted.

6. Further having a thermal sensor, The determination logic further Determines whether the temperature measured by the thermal sensor is equal to or higher than a predetermined threshold value, and if it is determined that the temperature is equal to or higher than the predetermined threshold value, determines that refueling is not permitted. The refueling system according to claim 1.

7. In a first region that is part of the image data captured by the camera, a recognition result indicating that a vehicle has been recognized is output, In a second region that is part of the image data captured by the camera and is different from the first region, a recognition result indicating that a fueler holding the fuel nozzle in an appropriate posture has been recognized is output. The refueling system according to claim 3.

8. Further having a thermal sensor, The determination logic Determines whether the temperature measured by the thermal sensor is equal to or higher than a predetermined threshold value in a region that does not include the first region. The refueling system according to claim 7.

9. When the fuel type is kerosene, the determination logic Performs recognition processing on the image data captured by the camera, A recognition result indicating that a predetermined container has been recognized, A recognition result indicating that a fueler holding the fuel nozzle in an appropriate posture has been recognized, A recognition result indicating that a fueler not holding the fuel nozzle in an appropriate posture has been recognized, When output, based on the recognition result, it is determined whether refueling is permitted. The refueling system according to claim 3.

10. The first determination logic corresponding to the first section is such that the recognition result is If the permission conditions are satisfied for a certain period of time, it is determined that refueling is permitted, If a predetermined non-permission condition is satisfied for a certain period of time, it is determined that refueling is not permitted, and then, if the permission conditions are satisfied for a certain period of time, it is determined that refueling is permitted, If another non-permission condition is satisfied for a certain period of time, it is determined that refueling is not permitted, The second determination logic corresponding to the second interval is such that when the recognition result satisfies the non-permission condition for a certain period of time, a warning is output, when the non-permission condition is satisfied for a certain period of time even after the warning is output, it is determined that refueling is not permitted, when the permission condition is satisfied for a certain period of time after the warning is output, it is determined that refueling is permitted, The third determination logic corresponding to the third interval is such that when the recognition result satisfies the permission condition for a certain period of time, it is determined that refueling is permitted, when the non-permission condition is satisfied for a certain period of time, it is determined that refueling is not permitted, and then, when the permission condition is satisfied for a certain period of time, it is determined that refueling is permitted, The fuel supply system according to claim 9.

11. further having a thermal sensor, the determination logic further determines whether the temperature measured by the thermal sensor is equal to or higher than a predetermined threshold value, and when it is determined that the temperature is equal to or higher than the predetermined threshold value, determines that refueling is not permitted. The fuel supply system according to claim 9.

12. refueling is prohibited or stopped by stopping the fuel pump or by closing the fuel valve installed in the fuel supply pipe, The fuel supply system according to claim 1.

13. further having a fuel supply controller that communicates with the information processing device, the fuel supply controller prohibits or stops the fuel supply by the fuel dispenser when it is determined based on the determination logic that refueling is not permitted. The fuel supply system according to any one of claims 1 to 12.

14. Based on a notification from the fuel dispenser, the state of the fuel nozzle and the state of fuel supply are determined, when it is determined based on the determination logic according to which the refueling act of the refueler is included in any one of the first interval before refueling start, the second interval during refueling, and the third interval after refueling end, or based on the determination logic according to which the refueling act of the refueler is included in any two consecutive intervals, that refueling is not permitted, the fuel supply by the fuel dispenser is prohibited or stopped, Information processing device.

15. Based on a notification from the fuel dispenser, the state of the fuel nozzle and the state of fuel supply are determined, when it is determined based on the determination logic according to which the refueling act of the refueler is included in any one of the first interval before refueling start, the second interval during refueling, and the third interval after refueling end, or based on the determination logic according to which the refueling act of the refueler is included in any two consecutive intervals, that refueling is not permitted, the fuel supply by the fuel dispenser is prohibited or stopped, A fuel supply control method executed by a computer for processing.

16. Based on the notification from the fuel dispenser, determine the state of the fuel nozzle and the state of fueling, When it is determined that refueling is not permitted based on the determination logic according to which the refueling act of the refueler is included in any one of the first section before the start of refueling, the second section during refueling, and the third section after the end of refueling, or based on the determination logic according to which it is included in any consecutive two sections, prohibit or stop the refueling by the fuel dispenser, A fueling control program for causing a computer to execute the process.

Images