Fuel supply system, information processing device, fuel supply control method, and fuel supply control program

Abstract

To provide a fuel supply system capable of executing processing in accordance with a fuel supply scene.SOLUTION: A fuel supply system comprises a fuel dispenser and an information processing device. The information processing device determines the state of the fuel nozzle and the fuel supply state based on notifications from the fuel dispenser. If it is determined that fueling is not permitted based on a determination logic corresponding to whether the fuel supplying action by a fuel supplier is included in any one of a first section before the start of fueling, a second section during fueling, or a third section after the end of fueling, or based on a determination logic corresponding to whether it is included in any two consecutive sections, then fueling by the fuel dispenser is prohibited or stopped.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present disclosure relates to a fueling system, an information processing device, a fueling control method, and a fueling control program.

Background Art

[0002] As a fueling system for service stations, development of a fueling system that monitors the fueling behavior of customers who are fuelers and determines whether to allow fueling by utilizing AI or the like is underway.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, fueling scenes at service stations are diverse, and in developing a fueling system, it is desirable to adopt a configuration in which the method for determining whether to allow fueling, the method for outputting the determination result, etc. are changed according to the fueling scene.

[0005] In one aspect, an object is to provide a fueling system capable of executing processing according to a fueling scene.

Means for Solving the Problems

[0006] According to one aspect, the fueling system is a fuel dispenser and an information processing device, and is a fueling system having the information processing device determines the state of the fueling nozzle and the state of fueling based on a notification from the fuel dispenser, Based on the determination logic according to which the refueling operation of the refueler is included in any one of the first section before the start of refueling, the second section during refueling, and the third section after the end of refueling, or based on the determination logic according to which it is included in any two consecutive sections, if it is determined that refueling is not permitted, the refueling by the fuel dispenser is prohibited or stopped.

Advantages of the Invention

[0007] It is possible to provide a refueling system capable of executing processing according to the refueling scene.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14A

Figure 14B

Figure 15A

Figure 15B

Figure 16A

Figure 16B

Figure 17A

Figure 17B

Figure 17C

Figure 17D

Figure 18A

Figure 18B

Figure 19

Figure 20A

Figure 20B

Figure 21A

Figure 21B

Figure 21C

Figure 21D

Figure 22A

Figure 22B

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28A

Figure 28B

Mode for Carrying Out the Invention

[0009] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In the present specification and the drawings, for components having substantially the same functional configuration, the same reference numerals are given and redundant descriptions are omitted.

[0010] [First Embodiment] [Learning Process] First, the learning process for constructing the fueling system according to the first embodiment will be described. FIG. 1 is a diagram showing the learning process for constructing the fueling system according to the first embodiment. As shown in FIG. 1, when constructing the fueling system according to the first embodiment, image data collected at a plurality of service stations (for example, service stations 110, 111,...) is used. Also, at one service station, when different types of fuel can be supplied, image data collected for each type of fuel is used. Here, the different types of fuel referred to here mean fuel oil for vehicles (hereinafter referred to as "fuel oil of the first fuel type") such as gasoline and light oil, and fuel oil for uses other than vehicles (hereinafter referred to as "fuel oil of the second fuel type") such as kerosene.

[0011] The example of FIG. 1 shows the case where service station 110 can supply both the fuel oil of the first fuel type and the fuel oil of the second fuel type. Therefore, in the example of FIG. 1, image data obtained by photographing the fueling scene 110_1 for the first fuel type and image data obtained by photographing the fueling scene 110_2 for the second fuel type are collected.

[0012] The fueling scene 110_1 for the first fuel type includes a fueling act by a fueler (not shown). A camera 130_1 that captures the area including the first fuel dispenser 120_1 as a video captures the first fuel dispenser 120_1 and the fueling act by the fueler, and image data with a predetermined frame period is collected by the image data collection device 140.

[0013] As shown in FIG. 1, a fuel nozzle 121_1 is connected to the first fuel dispenser 120_1, and the fuel oil of the first fuel type stored in the tank 126_1 is configured to be supplied to the fuel nozzle 121_1 through a fuel supply pipe by a fuel supply pump 125_1. Also, as shown in FIG. 1, a fuel supply valve 124_1 is installed in the fuel supply pipe. Further, in the first fuel dispenser 120_1, as a notification device, a voice output device 122_1 and a fuel supply data display 123_1 (a device that displays fuel supply data and messages from the information processing device 160) are installed.

[0014] Although not shown in FIG. 1, similar image data for the fuel supply scene 110_2 for the second fuel type is also collected by the image data collection device 140.

[0015] When collecting image data, the image data collection device 140 also collects a signal indicating the removal of the fuel nozzle 121_1 from the first fuel dispenser 120_1 and the return of the fuel nozzle 121_1 to the first fuel dispenser 120_1 (a signal indicating the state of the fuel nozzle).

[0016] The image data collected by the image data collection device 140 and the signal indicating the state of the fuel nozzle are transmitted to the learning device 150 via the network 180, and learning data is generated in the learning device 150. Note that the learning data is generated based on the image data collected at a plurality of service stations (for example, service stations 110, 111,...) and the signal indicating the state of the fuel nozzle. However, the learning data is generated separately for the first fuel type and the second fuel type.

[0017] The learning device 150 is installed at a location different from a plurality of service stations (for example, service stations 110, 111, ···) (for example, a server on the cloud). The learning device 150 uses the learning data for the first fuel type (referred to as the first learning data) to perform learning of the first model and generate the first learned model. Further, the learning device 150 uses the learning data for the second fuel type (referred to as the second learning data) to perform learning of the second model and generate the second learned model.

[0018] The first learned model and the second learned model generated by the learning device 150 are transmitted to a plurality of service stations (for example, service stations 110', 111', ···) via the network 180. Thereby, they are stored in the information processing device of each service station (for example, the information processing device 160 of the service station 110').

[0019] As shown in FIG. 1, the fueling system 110S' according to the first embodiment applied to the service station 110' includes, for the first fuel type, · a camera 130_1, · a first fuel dispenser 120_1, · a thermal sensor 170_1. The first fuel dispenser 120_1 is connected to a fueling nozzle 121_1 to which the fuel oil of the first fuel type stored in the tank 126_1 is supplied by a fueling pump 125_1 through a fueling pipe. A fueling valve 124_1 is installed in the fueling pipe. Further, the first fuel dispenser 120_1 is provided with a voice output device 122_1 and a fueling data display 123_1 as notification devices.

[0020] Also, the fueling system 110S' according to the first embodiment applied to the service station 110' has a similar configuration for the second fuel type. However, when the measurement area of the thermal sensor 170_1 includes both the fueling scene 110_1 of the first fuel type and the fueling scene 110_2 of the second fuel type, the thermal sensor 170_1 may be used for both the first fuel type and the second fuel type.

[0021] Furthermore, as shown in FIG. 1, the fueling system 110S' according to the first embodiment applied to the service station 110' has an information processing device 160. The information processing device 160 is · Cameras for the first fuel type and the second fuel type (cameras 130_1, 130_2), · Fuel dispensers (first fuel dispenser 120_1, second fuel dispenser 120_2), · Thermal sensors (thermal sensors 170_1, 170_2), · Fuel pumps (fuel pumps 125_1, 125_2), and is communicably connected.

[0022] In the example of FIG. 1, the case where the fueling system 110S' according to the first embodiment is applied to the service station 110' is shown. However, when it is applied to other service stations (for example, service stations 111', ···), the configuration is the same.

[0023] <Description of the fueling scene> Next, a specific example of the fueling scene at the service station 110 will be described. FIGS. 2A and 2B are the first and second figures showing an example of the fueling scene at the service station.

[0024] Among them, FIG. 2A shows an example of the fueling scene for the first fuel type at the service station 110. As shown in FIG. 2A, the service station 110 includes a first area 210 where the vehicle 211 stops and a second area 220 where the first fuel dispenser 120_1 for fueling the first fuel type is installed and where the fueling operation by the fueler 221 is performed.

[0025] In the case of the fueling scene shown in FIG. 2A, with the vehicle 211 stopped in the first area 210, the fueler 221 removes the fuel nozzle 121_1 from the nozzle holder of the first fuel dispenser 120_1 and inserts the fuel nozzle 121_1 into the fuel filler neck of the vehicle 211. Also, the fueler 221 starts fueling by operating the lever of the fuel nozzle 121_1 and continues fueling until the desired fuel quantity is reached. Then, when the desired fuel quantity is reached, the fueler 221 stops fueling by returning the lever of the fuel nozzle 121_1, removes the fuel nozzle 121_1 from the fuel filler neck, and then returns it to the nozzle holder of the first fuel dispenser 120_1 to end the fueling operation.

[0026] On the other hand, FIG. 2B shows an example of a fueling scene for the second fuel type at the service station 110. As shown in FIG. 2B, the service station 110 includes an area where the container 231 (poly tank) is placed and an area where the second fuel dispenser 120_2 for performing fueling of the second fuel type is installed, and an area where the fueling operation by the fueler 241 is performed.

[0027] In the case of the fueling scene shown in FIG. 2B, with the container 231 placed, the fueler 241 removes the fuel nozzle 121_2 from the nozzle holder of the second fuel dispenser 120_2 and inserts the fuel nozzle 121_2 into the fuel filler neck of the container 231. Also, the fueler 241 starts fueling by operating the lever of the fuel nozzle 121_2 and continues fueling until the desired fuel quantity is reached. Then, when the desired fuel quantity is reached, the fueler 241 stops fueling by returning the lever of the fuel nozzle 121_2, removes the fuel nozzle 121_2 from the fuel filler neck, and then returns it to the nozzle holder of the second fuel dispenser 120_2 to end the fueling operation.

[0028] <Processing for generating learning data> Next, an overview of the process from when the camera 130_1 captures the area including the first area 210 and the second area 220 as a video until the learning device 150 generates the first learning data based on the video will be described. FIG. 3A is a diagram showing the flow of the process for generating the first learning data.

[0029] As shown in FIG. 3A, the image data collection device 140 collects image data with a predetermined frame period, which is captured as a video by the camera 130_1. Further, as shown in FIG. 3A, the image data collection device 140 collects a signal indicating the state of the fuel nozzle 121_1 from the first fuel dispenser 120_1.

[0030] The learning device 150 extracts, from the image data collected by the image data collection device 140, the image data during the period from when the fuel nozzle 121_1 is removed from the nozzle mounting of the first fuel dispenser 120_1 to when it is returned to the nozzle mounting.

[0031] Further, the learning device 150 performs image recognition processing on the extracted image data, and recognizes each recognition target (such as a vehicle, a container, a fueler, a fuel nozzle, etc.) included in the extracted image data. Furthermore, the learning device 150 assigns an annotation to the recognized recognition target (such as a vehicle, a container, a fueler, a fuel nozzle, etc.).

[0032] Specifically, when a vehicle is recognized in the image data and is recognized within the first region 210 that is a part of the image data, an annotation indicating that it is a vehicle is assigned to the circumscribed rectangle (bounding box) of the vehicle.

[0033] Also, when a container is recognized in the image data, an annotation indicating that it is a container is assigned to the circumscribed rectangle (bounding box) of the container.

[0034] Also, when a fueler is recognized in the image data, · the fueler is recognized within the second region 220, and · the fuel nozzle is held in an appropriate posture, in this case, an annotation indicating that it is a fueler holding the fuel nozzle in an appropriate posture is assigned to the circumscribed rectangle (bounding box) of the fueler.

[0035] Also, in the case where the refueler is recognized in the image data, · the refueler is recognized within the second region 220, and · the fueling nozzle being held is not recognized, in this case, an annotation (fueling nozzle not recognizable) indicating that the refueler is not holding the fueling nozzle in an appropriate posture is assigned to the bounding box of the refueler.

[0036] Also, in the case where the refueler is not recognized within the second region 220 in the image data, · for example, when the fueling nozzle in a state of being inserted into the fuel inlet is recognized, in this case, an annotation (refueler not recognizable) indicating that the refueler is not holding the fueling nozzle in an appropriate posture is assigned to the bounding box of the recognized fueling nozzle.

[0037] The learning device 150 generates first learning data using the image data with any annotation among the extracted image data.

[0038] Subsequently, an overview of the process from when the camera 130_2 that captures the region including the second fuel dispenser 120_2 as a video captures the second fuel dispenser 120_2 and the placed container until the learning device 150 generates second learning data based on the video will be described. FIG. 3B is a diagram showing the flow of the process for generating the second learning data.

[0039] As shown in FIG. 3B, the image data collection device 140 collects image data with a predetermined frame period captured as a video by the camera 130_2. Also, as shown in FIG. 3B, the image data collection device 140 collects a signal indicating the state of the fueling nozzle 121_2 from the second fuel dispenser 120_2.

[0040] The learning device 150 extracts, from the image data collected by the image data collection device 140, the image data during the period from when the fueling nozzle 121_2 is removed from the nozzle mount of the second fuel dispenser 120_2 until it is returned to the nozzle mount.

[0041] Also, the learning device 150 performs image recognition processing on the extracted image data, and recognizes each recognition target (container, fueler, fueling nozzle, etc.) included in the extracted image data. Further, the learning device 150 assigns an annotation to the recognized recognition target (container, fueler, fueling nozzle, etc.).

[0042] Specifically, when a container is recognized in the image data, an annotation indicating that it is a container is assigned to the circumscribed rectangle (bounding box) of the recognized container.

[0043] Also, when a fueler is recognized in the image data, · if the fueler is holding the fueling nozzle in an appropriate posture, then an annotation indicating that the fueler is holding the fueling nozzle in an appropriate posture is assigned to the circumscribed rectangle (bounding box) of the fueler.

[0044] Also, when a fueler is recognized in the image data, · if the fueling nozzle held by the fueler cannot be recognized, then an annotation (fueling nozzle recognition impossible) indicating that the fueler is not holding the fueling nozzle in an appropriate posture is assigned to the circumscribed rectangle (bounding box) of the fueler.

[0045] Also, when a fueler is not recognized in the image data, · for example, if a fueling nozzle inserted into the fuel inlet is recognized, In this case, an annotation (fueler unrecognizable) indicating that the fueler is not holding the fuel nozzle in an appropriate posture is attached to the circumscribed rectangle (bounding box) of the fuel nozzle.

[0046] The learning device 150 generates second learning data using the image data to which any annotation has been attached among the extracted image data.

[0047] <Hardware Configuration of Learning Device> Next, the hardware configuration of the learning device 150 will be described. FIG. 4 is a diagram showing an example of the hardware configuration of the learning device.

[0048] As shown in FIG. 4, the learning device 150 includes a processor 401, a memory 402, an auxiliary storage device 403, an I / F (Interface) device 404, a communication device 405, and a drive device 406. The processor 401, the memory 402, the auxiliary storage device 403, the I / F device 404, the communication device 405, and the drive device 406 of the learning device 150 are interconnected via a bus 407.

[0049] The processor 401 includes various arithmetic devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 401 reads out and executes various programs (for example, a learning program, etc.) on the memory 402.

[0050] The memory 402 includes main storage devices such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processor 401 and the memory 402 form a so-called computer, and the computer realizes various functions by the processor 401 executing various programs read out on the memory 402.

[0051] The auxiliary storage device 403 stores various programs and various information used when the various programs are executed by the processor 401.

[0052] The I / F device 404 is a connection device for connecting the operation device 411, which is an example of an external device, the display device 412, and the learning device 150.

[0053] The communication device 405 is a communication device for communicating with, for example, the image data collection device 140 or the information processing device 160 via a network.

[0054] The drive device 406 is a device for setting the recording medium 413. The recording medium 413 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. Further, the recording medium 413 may include semiconductor memories that record information electrically, such as ROMs and flash memories.

[0055] Note that various programs installed in the auxiliary storage device 403 are installed, for example, by setting the distributed recording medium 413 in the drive device 406 and reading out the various programs recorded on the recording medium 413. Alternatively, various programs installed in the auxiliary storage device 403 may be installed by being downloaded from a network via the communication device 405.

[0056] <Functional Configuration of the Learning Device> Next, the functional configuration of the learning device 150 will be described. FIG. 5 is a diagram showing an example of the functional configuration of the learning device. As described above, a learning program is installed in the learning device 150, and when the program is executed, the learning device 150 functions as a learning data generation unit 500A and a learning unit 500B.

[0057] As shown in FIG. 5, the learning data generation unit 500A further includes a collected data acquisition unit 510, an image recognition unit 520, an annotation adding unit 530, and a learning data generation unit 540.

[0058] The collection data acquisition unit 510 acquires, from the image data collection device 140, the image data captured by the camera 130_1 and the signal indicating the state of the fuel nozzle 121_1. Further, the collection data acquisition unit 510 acquires, from the image data collection device 140, the image data captured by the camera 130_2 and the signal indicating the state of the fuel nozzle 121_2.

[0059] The image recognition unit 520 extracts, from the image data captured by the camera 130_1, the image data during the period from when the fuel nozzle 121_1 is removed from the nozzle holder of the first fuel dispenser 120_1 until it is returned to the nozzle holder of the first fuel dispenser 120_1. Further, the image recognition unit 520 performs image recognition processing on the extracted image data. Thereby, the image recognition unit 520 recognizes the recognition targets (such as vehicles, containers, fuelers, fuel nozzles, etc.).

[0060] Also, the image recognition unit 520 extracts, from the image data captured by the camera 130_2, the image data during the period from when the fuel nozzle 121_2 is removed from the nozzle holder of the second fuel dispenser 120_2 until it is returned to the nozzle holder of the second fuel dispenser 120_2. Further, the image recognition unit 520 performs image recognition processing on the extracted image data. Thereby, the image recognition unit 520 recognizes the recognition targets (such as containers, fuelers, fuel nozzles, etc.).

[0061] The annotation adding unit 530 adds an annotation to the bounding box of the recognition target recognized by the image recognition unit 520. Since the method of adding the annotation has been described with reference to FIGS. 3A and 3B, the description is omitted here.

[0062] The learning data generation unit 540 generates first learning data and second learning data using the image data annotated by the annotation adding unit 530 as input data and the added annotation as correct data. The learning data generation unit 540 stores the generated first learning data and second learning data in the learning data storage unit 500C.

[0063] Further, as shown in FIG. 5, the learning unit 500B further includes a learning data reading unit 550, a first model 560_1, a comparison / modification unit 570_1, a second model 560_2, and a comparison / modification unit 570_2.

[0064] The learning data reading unit 550 reads the first learning data from the learning data storage unit 500C, inputs the input data included in the first learning data into the first model 560_1, and inputs the correct answer data into the comparison / modification unit 570_1.

[0065] Also, the learning data reading unit 550 reads the second learning data from the learning data storage unit 500C, inputs the input data included in the second learning data into the second model 560_2, and inputs the correct answer data into the comparison / modification unit 570_2.

[0066] The first model 560_1 is a machine learning model that performs image recognition processing. When the input data included in the first learning data is input, the first model 560_1 performs image recognition processing and outputs a recognition result.

[0067] The comparison / modification unit 570_1 compares the recognition result output from the first model 560_1 with the correct answer data input by the learning data reading unit 550. Also, the comparison / modification unit 570_1 updates the model parameters of the first model 560_1 so that the recognition result output from the first model 560_1 matches the correct answer data. In this way, the learning process for the first model 560_1 is performed, and the first learned model is generated.

[0068] The second model 560_2 is a machine learning model that performs image recognition processing. When the input data included in the second learning data is input, the second model 560_2 performs image recognition processing and outputs a recognition result.

[0069] The comparison / modification unit 570_2 compares the recognition result output from the second model 560_2 with the correct data input by the learning data reading unit 550. Further, the comparison / modification unit 570_2 updates the model parameters of the second model 560_2 so that the recognition result output from the second model 560_2 matches the correct data. In this way, the learning process for the second model 560_2 is performed, and the second learned model is generated.

[0070] Note that the first learned model and the second learned model generated by the learning unit 500B are transmitted to the information processing apparatus 160 and the like.

[0071] <Specific example of learning data> Next, a specific example of the learning data generated by the learning data generation unit 500A of the learning apparatus 150 will be described.

[0072] (1) Specific example of the first learning data FIG. 6 is a diagram showing a specific example of the first learning data. As shown in FIG. 6, the first learning data 600 includes "input data" and "correct data" as information items. Further, the "input data" further includes an "image data ID".

[0073] The "image data ID" stores an identifier for identifying the image data annotated by the annotation adding unit 530. In the example of FIG. 6, the image data 610 is the image data with the image data ID = ID001, the image data 620 is the image data with the image data ID = ID00L, the image data 630 is the image data with the image data ID = ID00M, and the image data 640 is the image data with the image data ID = ID00N.

[0074] "Correct data" further includes "regions" and "annotation data". The "regions" store the first region 210 and the second region 220, and the "annotation data" stores the annotation data assigned to the bounding box of the recognition target recognized in the corresponding region.

[0075] In the example of FIG. 6, for the bounding box 611 of the vehicle recognized in the first region 210 which is a part of the image data 610, "Annotation data indicating that it is a vehicle" is shown as being assigned. Also, in the example of FIG. 6, for the bounding box 612 of the fueler recognized in the second region 220 which is a part of the image data 610, "Annotation indicating that it is a fueler holding the fuel nozzle in an appropriate posture" is shown as being assigned.

[0076] Similarly, in the example of FIG. 6, for the bounding box 621 of the vehicle recognized in the first region 210 which is a part of the image data 620, "Annotation data indicating that it is a vehicle" is shown as being assigned. Also, in the example of FIG. 6, for the bounding box 622 of the fueler recognized in the second region 220 which is a part of the image data 620, "Annotation indicating that it is a fueler not holding the fuel nozzle in an appropriate posture (fuel nozzle recognition impossible)" is shown as being assigned.

[0077] Similarly, in the example of FIG. 6, for the bounding box 631 of the vehicle recognized in the first region 210 which is a part of the image data 630, "Annotation data indicating that it is a vehicle" is shown as being assigned. Also, in the example of FIG. 6, for the bounding box 632 of the fuel nozzle recognized in the second region 220 which is a part of the image data 630, "Annotation indicating that the fueler is not holding the fuel nozzle in an appropriate posture (fueler unrecognizable)" indicates that it has been added.

[0078] Similarly, in the example of FIG. 6, for the bounding box 641 of the container recognized in the first region 210 which is a part of the image data 640, "Annotation data indicating that it is a container" indicates that it has been added. Also, in the example of FIG. 6, for the bounding box 642 of the fueler recognized in the second region 220 which is a part of the image data 640, "Annotation indicating that the fueler is holding the fuel nozzle in an appropriate posture" indicates that it has been added.

[0079] (2) Specific examples of the second learning data FIG. 7 is a diagram showing a specific example of the second learning data. As shown in FIG. 7, the second learning data 700 includes, as information items, "input data" and "correct answer data". Further, the "input data" includes an "image data ID".

[0080] The "image data ID" stores an identifier for identifying the image data to which an annotation has been added by the annotation adding unit 530. In the example of FIG. 7, the image data 710 is the image data with the image data ID = ID101, the image data 720 is the image data with the image data ID = ID10L, the image data 730 is the image data with the image data ID = ID10M, and the image data 740 is the image data with the image data ID = ID10N.

[0081] The "correct answer data" further includes "annotation data". The "annotation data" stores the annotation data attached to the recognized bounding box of the recognition target.

[0082] The example in FIG. 7 shows that for the bounding box 711 of the container recognized in the image data 710, "annotation data indicating that it is a container" has been assigned. Also, the example in FIG. 7 shows that for the bounding box 712 of the fueler recognized in the image data 710, "annotation indicating that it is a fueler holding the fuel nozzle in an appropriate posture" has been assigned.

[0083] Similarly, the example in FIG. 7 shows that for the bounding box 721 of the container recognized in the image data 720, "annotation data indicating that it is a container" has been assigned. Also, the example in FIG. 7 shows that for the bounding box 722 of the fueler recognized in the image data 720, "annotation indicating that it is a fueler not holding the fuel nozzle in an appropriate posture (fuel nozzle unrecognizable)" has been assigned.

[0084] Similarly, the example in FIG. 7 shows that for the bounding box 731 of the container recognized in the image data 730, "annotation data indicating that it is a container" has been assigned. Also, the example in FIG. 7 shows that for the bounding box 732 of the fuel nozzle recognized in the image data 730, "annotation indicating that it is a fueler not holding the fuel nozzle in an appropriate posture (fueler unrecognizable)" has been assigned.

[0085] Similarly, the example in FIG. 7 shows that for the bounding box 742 of the fueler recognized in the image data 740, "annotation indicating that it is a fueler holding the fuel nozzle in an appropriate posture" has been assigned.

[0086] <Flow of Learning Process by Learning Device> Next, the flow of the learning process by the learning device 150 will be described. FIG. 8 is an example of a flowchart showing the flow of the learning process.

[0087] In step S801, the learning device 150 acquires image data and a signal indicating the state of the fuel nozzle from the image data collection device 140.

[0088] In step S802, the learning device 150 extracts the image data during the period from when the fuel nozzle is removed from the nozzle holder until it is returned to the nozzle holder, and performs image recognition processing on the extracted image data.

[0089] In step S803, the learning device 150 identifies the first region and the second region in the image data used for generating the first learning data among the image data on which the image recognition processing has been performed.

[0090] In step S804, the learning device 150 assigns annotations to the image data used for generating the first learning data. Specifically, the learning device 150 assigns annotations to the bounding box of the recognition target included in the identified first region and also assigns annotations to the bounding box of the recognition target included in the identified second region.

[0091] Also, the learning device 150 assigns annotations to the image data used for generating the second learning data. Specifically, the learning device 150 assigns annotations to the bounding box of the recognition target.

[0092] In step S805, the learning device 150 generates first learning data for the first fuel type and second learning data for the second fuel type based on the image data with annotations.

[0093] In step S806, the learning device 150 performs learning processing on the first model using the first learning data to generate a first learned model. Further, the learning device 150 performs learning processing on the second model using the second learning data to generate a second learned model.

[0094] In step S807, the learning device 150 transmits the generated first learned model and second learned model to the information processing device 160 etc. and stores them in the information processing device 160 etc.

[0095] <Hardware Configuration of Information Processing Device> Next, the hardware configuration of the information processing device 160 will be described. FIG. 9 is a diagram showing an example of the hardware configuration of the information processing device.

[0096] As shown in FIG. 9, the information processing device 160 includes a processor 901, a memory 902, an auxiliary storage device 903, a connection device 904, an I / F device 905, a communication device 906, and a drive device 907. Note that the processor 901, memory 902, auxiliary storage device 903, connection device 904, I / F device 905, communication device 906, and drive device 907 of the information processing device 160 are interconnected via a bus 908.

[0097] The processor 901 includes various arithmetic devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 901 reads out and executes various programs (for example, a fuel supply control program etc.) on the memory 902.

[0098] The memory 902 includes main memory devices such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processor 901 and the memory 902 form a so-called computer, and by the processor 901 executing various programs read out on the memory 902, the computer realizes various functions.

[0099] The auxiliary storage device 903 stores various programs and various information used when the various programs are executed by the processor 901.

[0100] The connection device 904 is a connection device for connecting the first fuel dispenser 120_1, the second fuel dispenser 120_2, and the information processing device 160. The connection device 904 receives signals indicating the state of the fuel nozzle, signals indicating the fuel quantity, etc. from the first fuel dispenser 120_1 and the second fuel dispenser 120_2. Signals indicating the state of the fuel nozzle, signals indicating the fuel quantity, etc. are measured by the first fuel sensor 921_1 and the second fuel sensor 921_2. Further, the connection device 904 transmits messages for voice output to the first fuel dispenser 120_1 and the second fuel dispenser 120_2 via the voice output devices 122_1 and 122_2. Also, the connection device 904 transmits display information to be displayed to the first fuel dispenser 120_1 and the second fuel dispenser 120_2 via the fueling data displays 123_1 and 123_2. Further, the connection device 904 transmits signals for instructing the opening and closing operations of the fueling valves 124_1 and 124_2 to the first fuel dispenser 120_1 and the second fuel dispenser 120_2. Furthermore, the connection device 904 transmits signals for instructing the start / stop of driving to the fuel pumps 125_1 and 125_2.

[0101] Also, the connection device 904 receives thermal sensor data from the thermal sensors 170_1 and 170_2.

[0102] The I / F device 905 is a connection device for connecting an operation device 911 and a display device 912, which are examples of external devices, and the information processing device 160. Note that the operation device 911 is operated by, for example, the staff of the service station 110', and the display device 912 is displayed for, for example, the staff of the service station 110'.

[0103] The communication device 906 communicates with the cameras 130_1 and 130_2 via a network and receives image data. Also, the communication device 906 communicates with the learning device 150 and receives the first learned model and the second learned model.

[0104] The drive device 907 is a device for setting the recording medium 913. The recording medium 913 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, magneto-optical disks, etc. Further, the recording medium 913 may include semiconductor memories that record information electrically, such as ROMs, flash memories, etc.

[0105] Note that various programs installed in the auxiliary storage device 903 are installed, for example, when the distributed recording medium 913 is set in the drive device 907 and various programs recorded on the recording medium 913 are read out. Alternatively, various programs installed in the auxiliary storage device 903 may be installed by being downloaded from a network via the communication device 906.

[0106] <Functional Configuration of Information Processing Device> Next, the functional configuration of the information processing device 160 will be described. FIG. 10 is a diagram showing an example of the functional configuration of the information processing device. As described above, a fuel supply control program is installed in the information processing device 160, and when the program is executed, the information processing device 160 · First fuel supply type image data acquisition unit 1001, second fuel supply type image data acquisition unit 1002, · First fuel supply sensor data acquisition unit 1003, second fuel supply sensor data acquisition unit 1004, · Image recognition unit 1005, · Section determination unit 1006, section determination unit 1007, · Thermal sensor data acquisition unit 1008, · First fuel supply monitoring unit 1009_1, second fuel supply monitoring unit 1009_2, · Output control unit 1010, has.

[0107] The first fuel supply type image data acquisition unit 1001 acquires the image data captured by the camera 130_1 and notifies the image recognition unit 1005.

[0108] The second fuel supply type image data acquisition unit 1002 acquires the image data captured by the camera 130_2 and notifies the image recognition unit 1005.

[0109] The first fuel supply sensor data acquisition unit 1003 acquires the signal from the first fuel supply sensor 921_1 and notifies the section determination unit 1006.

[0110] The second fuel supply sensor data acquisition unit 1004 acquires the signal from the second fuel supply sensor 921_2 and notifies the section determination unit 1007.

[0111] Based on the signal from the first fuel supply sensor 921_1, the section determination unit 1006 classifies the section from when the fuel supply nozzle 121_1 is removed from the nozzle holder of the first fuel dispenser 120_1 until it is returned to the nozzle holder into three sections. Among the three sections, the section before fuel supply start is referred to as the first section, the section during fuel supply is referred to as the second section, and the section after fuel supply end is referred to as the third section.

[0112] Based on the signal from the second fuel supply sensor 921_2, the section determination unit 1007 classifies the section from when the fuel supply nozzle 121_2 is removed from the nozzle holder of the second fuel dispenser 120_2 until it is returned to the nozzle holder into three sections. For the second fuel supply type as well, similar to the first fuel supply type, among the three sections, the section before fuel supply start is referred to as the first section, the section during fuel supply is referred to as the second section, and the section after fuel supply end is referred to as the third section.

[0113] The image recognition unit 1005 extracts the image data included in any one of the first to third sections from the image data notified by the first fuel supply type image data acquisition unit 1001. Further, the image recognition unit 1005 performs image recognition processing on the extracted image data using the first learned model and notifies the recognition result to the first fuel supply monitoring unit 1009_1.

[0114] In addition, the image recognition unit 1005 extracts the image data included in any one of the first to third intervals from the image data notified by the second fuel supply type image data acquisition unit 1002. Further, the image recognition unit 1005 performs image recognition processing on the extracted image data using the second learned model, and notifies the recognition result to the second fuel supply monitoring unit 1009_2.

[0115] The thermal sensor data acquisition unit 1008 acquires the thermal sensor data measured by the thermal sensor 170_1 and notifies it to the first fuel supply monitoring unit 1009_1. Further, the thermal sensor data acquisition unit 1008 acquires the thermal sensor data measured by the thermal sensor 170_2 and notifies it to the second fuel supply monitoring unit 1009_2.

[0116] The first fuel supply monitoring unit 1009_1 · the recognition result notified by the image recognition unit 1005, · the section determination result notified by the section determination unit 1006, · the thermal sensor data of the thermal sensor 170_1 notified by the thermal sensor data acquisition unit 1008, is used to determine whether fuel supply for the first fuel supply type is permitted. Further, the first fuel supply monitoring unit 1009_1 controls the fuel supply for the first fuel supply type based on the determination result of whether fuel supply is permitted. Specifically, the first fuel supply monitoring unit 1009_1 controls the fuel supply for the first fuel supply type by controlling the fuel valve 124_1 or the fuel pump 125_1.

[0117] In addition, the first fuel supply monitoring unit 1009_1 notifies the output control unit 1010 of the determination result of whether fuel supply is permitted. Note that when the first fuel supply monitoring unit 1009_1 determines that fuel supply is not permitted, it also notifies the output control unit 1010 of the information indicating the cause of non-permission of fuel supply.

[0118] The second fuel supply monitoring unit 1009_2 · the recognition result notified by the image recognition unit 1005, · the section determination result notified by the section determination unit 1007, · Using the thermal sensor data of the thermal sensor 170_2 notified by the thermal sensor data acquisition unit 1008, it determines whether refueling is permitted for the second refueling type. Further, the second refueling monitoring unit 1009_2 controls the refueling for the second refueling type based on the determination result of whether refueling is permitted. Specifically, the second refueling monitoring unit 1009_2 controls the refueling for the second refueling type by controlling the fueling valve 124_2 or the fueling pump 125_2.

[0119] In addition, the second refueling monitoring unit 1009_2 notifies the output control unit 1010 of the determination result of whether refueling is permitted. Note that when the second refueling monitoring unit 1009_2 determines that refueling is not permitted, it also notifies the output control unit 1010 of the information indicating the cause of the non-permission of refueling.

[0120] When the output control unit 1010 is notified of the information indicating the cause of the non-permission of refueling from the first refueling monitoring unit 1009_1, · It controls so that a message corresponding to the information indicating the cause is output as voice via the voice output device 122_1, · It controls so that the display information corresponding to the information indicating the cause is displayed and output on the fueling data display 123_1 of the first fueling machine 120_1, · It controls so that the information indicating the cause is displayed and output on the display device 912 of the information processing device 160.

[0121] Also, when the output control unit 1010 is notified of the information indicating the cause of the non-permission of refueling from the second refueling monitoring unit 1009_2, · It controls so that a message corresponding to the information indicating the cause is output as voice via the voice output device 122_2, · It controls so that the display information corresponding to the information indicating the cause is displayed and output on the fueling data display 123_2 of the second fueling machine 120_2, · It controls so that the information indicating the cause is displayed and output on the display device 912 of the information processing device 160.

[0122] <Details of each part of the information processing device> Next, details of each part of the information processing apparatus 160 (here, the section determination units 1006 and 1007, the image recognition unit 1005, the first fuel supply monitoring unit 1009_1, the second fuel supply monitoring unit 1009_2, and the output control unit 1010) will be described.

[0123] (1) Details of the section determination units 1006 and 1007 First, as details of the section determination units 1006 and 1007, the functional configurations of the section determination units 1006 and 1007 and specific examples of the processing of the section determination units 1006 and 1007 will be described.

[0124] (1-1) Details of the functional configurations of the section determination units 1006 and 1007 FIG. 11 is a diagram showing an example of the functional configuration of the section determination unit of the information processing apparatus. Since the functional configurations of the section determination unit 1006 and the section determination unit 1007 are the same, they will be described together using FIG. 11.

[0125] As shown in FIG. 11, the section determination units 1006 and 1007 · nozzle state determination unit 1101, · fuel supply amount determination unit 1102, · first section start determination unit 1103, second section transition determination unit 1104, third section transition determination unit 1105, third section end determination unit 1106, · section determination result output unit 1107, have.

[0126] The nozzle state determination unit 1101 acquires a signal indicating the state of the fuel supply nozzle 121_1 (or 121_2) from the first fuel supply sensor data acquisition unit 1003 (or the second fuel supply sensor data acquisition unit 1004).

[0127] When the nozzle state determination unit 1101 receives a signal indicating that the fuel supply nozzle 121_1 (or 121_2) has been removed from the nozzle hook of the first fuel dispenser 120_1 (or the second fuel dispenser 120_2), it notifies the first section start determination unit 1103.

[0128] Further, when the nozzle state determination unit 1101 receives a signal indicating that the fuel supply nozzle 121_1 (or 121_2) has been returned to the nozzle mount of the first fuel dispenser 120_1 (or the second fuel dispenser 120_2), it notifies the third section end determination unit 1106.

[0129] The fuel quantity determination unit 1102 acquires a signal indicating the fuel quantity from the first fuel supply sensor data acquisition unit 1003 (or the second fuel supply sensor data acquisition unit 1004).

[0130] When the signal indicating the fuel quantity acquired by the fuel quantity determination unit 1102 from the first fuel supply sensor data acquisition unit 1003 (or the second fuel supply sensor data acquisition unit 1004) changes from zero to a predetermined threshold value or more, the fuel quantity determination unit 1102 notifies the second section transition determination unit 1104.

[0131] Further, when the signal indicating the fuel quantity acquired by the fuel quantity determination unit 1102 from the first fuel supply sensor data acquisition unit 1003 (or the second fuel supply sensor data acquisition unit 1004) changes from a predetermined threshold value or more to zero, the fuel quantity determination unit 1102 notifies the third section transition determination unit 1105.

[0132] When the first section start determination unit 1103 is notified from the nozzle state determination unit 1101 of a signal indicating that the fuel supply nozzle 121_1 (or 121_2) has been removed, it determines that the first section has started. Further, the first section start determination unit 1103 notifies the second section transition determination unit 1104 and the section determination result output unit 1107 that the first section has started.

[0133] After being notified that the first section has started, when the second section transition determination unit 1104 is notified from the fuel quantity determination unit 1102 that the fuel quantity has changed from zero to a predetermined threshold value or more, it determines that the transition has been made from the first section to the second section. Further, the second section transition determination unit 1104 notifies the third section transition determination unit 1105 and the section determination result output unit 1107 that the transition has been made from the first section to the second section.

[0134] After being notified that the transition to the second section has occurred, when the fuel injection amount determination unit 1102 notifies that the fuel injection amount has changed from equal to or greater than a predetermined threshold value to zero, the third section transition determination unit 1105 determines that the transition from the second section to the third section has occurred. Further, the third section transition determination unit 1105 notifies the third section end determination unit 1106 and the section determination result output unit 1107 that the transition from the second section to the third section has occurred.

[0135] After being notified that the transition to the third section has occurred, when the nozzle state determination unit 1101 notifies a signal indicating that the fuel injection nozzle 121_1 (or 121_2) has been retracted, the third section end determination unit 1106 determines that the third section has ended. Further, the third section end determination unit 1106 notifies the section determination result output unit 1107 that the third section has ended.

[0136] When being notified by the first section start determination unit 1103 that the first section has started, the section determination result output unit 1107 starts outputting information indicating that it is during the first section. Further, when being notified by the second section transition determination unit 1104 that the transition to the second section has occurred, the section determination result output unit 1107 stops outputting information indicating that it is during the first section and starts outputting information indicating that it is during the second section. Further, when being notified by the third section transition determination unit 1105 that the transition to the third section has occurred, the section determination result output unit 1107 stops outputting information indicating that it is during the second section and starts outputting information indicating that it is during the third section. Furthermore, when being notified by the third section end determination unit 1106 that the third section has ended, the section determination result output unit 1107 stops outputting information indicating that it is during the third section.

[0137] (1-2) Specific examples of the processing of the section determination units 1006 and 1007 FIG. 12 is a diagram showing a specific example of the processing by the section determination unit of the information processing apparatus. As shown in FIG. 12(a), when a signal indicating that the fuel nozzle 121_1 has been removed from the nozzle mount is acquired as a signal indicating the state of the fuel nozzle 121_1, the section determination unit 1006 starts outputting information indicating that it is in the first section. Also, when a signal indicating that the signal indicating the fuel supply amount of the first fuel dispenser 120_1 has changed from zero to a predetermined threshold value or more is acquired, the section determination unit 1006 starts outputting information indicating that it is in the second section. Further, when a signal indicating that the signal indicating the fuel supply amount of the first fuel dispenser 120_1 has changed from a predetermined threshold value or more to zero is acquired, the section determination unit 1006 starts outputting information indicating that it is in the third section. Furthermore, when a signal indicating that the fuel nozzle 121_1 has been returned to the nozzle mount is acquired as a signal indicating the state of the fuel nozzle 121_1, the section determination unit 1006 stops outputting information indicating that it is in the third section.

[0138] Similarly, as shown in FIG. 12(b), when a signal indicating that the fuel nozzle 121_2 has been removed from the nozzle mount is acquired as a signal indicating the state of the fuel nozzle 121_2, the section determination unit 1007 starts outputting information indicating that it is in the first section. Also, when a signal indicating that the signal indicating the fuel supply amount of the second fuel dispenser 120_2 has changed from zero to a predetermined threshold value or more is acquired, the section determination unit 1007 starts outputting information indicating that it is in the second section. Further, when a signal indicating that the signal indicating the fuel supply amount of the second fuel dispenser 120_2 has changed from a predetermined threshold value or more to zero is acquired, the section determination unit 1007 starts outputting information indicating that it is in the third section. Furthermore, when a signal indicating that the fuel nozzle 121_2 has been returned to the nozzle mount is acquired as a signal indicating the state of the fuel nozzle 121_2, the section determination unit 1007 stops outputting information indicating that it is in the third section.

[0139] (2) Details of the Image Recognition Unit 1005 Next, as details of the image recognition unit 1005, the functional configuration of the image recognition unit 1005 and a specific example of the processing of the image recognition unit 1005 will be described.

[0140] (2-1) Detailed Functional Configuration of Image Recognition Unit 1005 FIG. 13 is a diagram showing an example of the functional configuration of the image recognition unit of the information processing apparatus. As shown in FIG. 13, the image recognition unit 1005 includes a recognition section acquisition unit 1301, a first learned model 1302, and a region determination unit 1303. Further, the image recognition unit 1005 includes a recognition section acquisition unit 1311 and a second learned model 1312.

[0141] The recognition section acquisition unit 1301 extracts the image data included in any one of the first to third sections notified from the section determination unit 1006 from the image data notified from the first fuel type image data acquisition unit 1001, and inputs the extracted image data to the first learned model 1302.

[0142] The first learned model 1302 is the first learned model generated in the learning device 150. When image data is input, it recognizes the recognition target included in the image data and outputs the recognition result.

[0143] The region determination unit 1303 determines whether the recognition result output by the first learned model 1302 is included in the first region 210 or the second region 220. Further, the region determination unit 1303 outputs the recognition result determined to be included in the first region 210 or the second region 220 to the first fuel monitoring unit 1009_1.

[0144] The recognition section acquisition unit 1311 extracts the image data included in any one of the first to third sections notified from the section determination unit 1007 from the image data notified from the second fuel type image data acquisition unit 1002, and inputs the extracted image data to the second learned model 1312.

[0145] The second learned model 1312 is the second learned model generated in the learning device 150. When image data is input, it recognizes the recognition target included in the image data and outputs the recognition result to the second fuel monitoring unit 1009_2.

[0146] (2-2) Specific Example of the Processing of the Image Recognition Unit 1005 (in the Case of the First Fueling Type) FIG. 14A is a first diagram showing a specific example of the processing by the image recognition unit of the information processing apparatus. As shown in FIG. 14A, the image recognition unit 1005 extracts the image data included in the section (the first to third sections) from when the fueling nozzle 121_1 is removed from the nozzle holder until it is returned, among the image data captured by the camera 130_1.

[0147] Also, as shown in FIG. 14A, the image recognition unit 1005 performs image recognition processing on the extracted image data using the first pre-trained model 1302, and by recognizing the recognition targets (such as vehicles, containers, fuelers, fueling nozzles, etc.), · Vehicles, · Containers, · Fuelers who hold the fueling nozzle in an appropriate posture, · Fuelers who do not hold the fueling nozzle in an appropriate posture (fueling nozzle unrecognizable), · Fuelers who do not hold the fueling nozzle in an appropriate posture (fueler unrecognizable), are output as recognition results.

[0148] Also, the image recognition unit 1005 outputs the recognition results determined to be included in the first area 210 or the second area 220. The recognition results 1401 to 1404 in FIG. 14A show output examples of the recognition results.

[0149] In the example of FIG. 14A, the recognition result 1401 output by the image recognition unit 1005 is · In the first area 210, a bounding box in which a vehicle is recognized, · In the second area 220, a bounding box in which a fueler who holds the fueling nozzle in an appropriate posture is recognized, as shown.

[0150] In the example of FIG. 14A, the recognition result 1402 output by the image recognition unit 1005 is · In the first area 210, a bounding box in which a vehicle is recognized, · In the second area 220, a bounding box (refueling nozzle unrecognizable) in which a refueler not holding the refueling nozzle in an appropriate posture is recognized. is shown.

[0151] Also, in the example of FIG. 14A, the recognition result 1403 output by the image recognition unit 1005 is · A bounding box in which a vehicle is recognized in the first area 210, · In the second area 220, a bounding box (refueler unrecognizable) in which a refueler not holding the refueling nozzle in an appropriate posture is recognized. is shown.

[0152] Furthermore, in the example of FIG. 14A, the recognition result 1404 output by the image recognition unit 1005 is · A bounding box in which a container is recognized in the first area 210, · In the second area 220, a bounding box in which a refueler holding the refueling nozzle in an appropriate posture is recognized. is shown.

[0153] (2-3) Specific example of the processing of the image recognition unit 1005 (in the case of the second refueling type) FIG. 14B is a second diagram showing a specific example of the processing by the image recognition unit of the information processing apparatus. As shown in FIG. 14B, the image recognition unit 1005 extracts the image data included in the section (the first to the third sections) from when the refueling nozzle 121_2 is removed from the nozzle holder until it is returned, among the image data captured by the camera 130_2.

[0154] Also, as shown in FIG. 14B, the image recognition unit 1005 performs image recognition processing on the extracted image data using the second learned model 1312, and by recognizing the recognition targets (container, refueler, refueling nozzle, etc.), · Container, · A refueler holding the refueling nozzle in an appropriate posture, · A refueler not holding the refueling nozzle in an appropriate posture (refueling nozzle unrecognizable), · A fueler who is not holding the fuel nozzle in an appropriate posture (fueler not recognizable), is output as a recognition result. The recognition results 1411 to 1414 in Fig. 14B show examples of the output of the recognition results.

[0155] In the example of Fig. 14B, the recognition result 1411 output by the image recognition unit 1005 is · The bounding box in which the container is recognized, · The bounding box in which a fueler holding the fuel nozzle in an appropriate posture is recognized. is shown.

[0156] Also, in the example of Fig. 14B, the recognition result 1412 output by the image recognition unit 1005 is · The bounding box in which the container is recognized, · The bounding box in which a fueler not holding the fuel nozzle in an appropriate posture is recognized (fuel nozzle not recognizable). is shown.

[0157] Also, in the example of Fig. 14B, the recognition result 1413 output by the image recognition unit 1005 is · The bounding box in which the container is recognized, · The bounding box in which a fueler not holding the fuel nozzle in an appropriate posture is recognized (fueler not recognizable). is shown.

[0158] Furthermore, in the example of Fig. 14B, the recognition result 1414 output by the image recognition unit 1005 is · The bounding box in which a fueler holding the fuel nozzle in an appropriate posture is recognized. is shown.

[0159] (3) Details of the first fueling monitoring unit 1009_1 and the second fueling monitoring unit 1009_2 Next, as the details of the first fuel supply monitoring unit 1009_1 and the second fuel supply monitoring unit 1009_2, the functional configuration and processing flow of the first fuel supply monitoring unit 1009_1 and the functional configuration and processing flow of the second fuel supply monitoring unit 1009_2 will be described.

[0160] (3-1) Details of the functional configuration of the first fuel supply monitoring unit 1009_1 and the second fuel supply monitoring unit 1009_2 FIG. 15A is a diagram showing an example of the functional configuration of the first fuel supply monitoring unit of the information processing apparatus. As shown in FIG. 15A, the first fuel supply monitoring unit 1009_1 includes a recognition result acquisition unit 1501, a determination logic switching unit 1502, a poly-tank determination unit 1506, a first determination logic 1503, a second determination logic 1504, a third determination logic 1505, and a fire determination unit 1507. The first fuel supply monitoring unit 1009_1 also includes a fuel supply control unit 1508 and a cause information acquisition unit 1509. The first determination logic 1503 to the third determination logic 1505, the poly-tank determination unit 1506, and the fire determination unit 1507 are collectively referred to as the first determination unit.

[0161] The recognition result acquisition unit 1501 acquires the recognition results for the image data included in the first section to the third section from the image recognition unit 1005. The recognition result acquisition unit 1501 notifies the acquired recognition results to the determination logic switching unit 1502 and the poly-tank determination unit 1506.

[0162] The determination logic switching unit 1502 inputs the recognition result notified from the recognition result acquisition unit 1501 to any one of the first determination logic 1503 to the third determination logic 1505. Which one of the first determination logic 1503 to the third determination logic 1505 to input is determined by the information indicating which of the first to third sections is notified from the section determination unit 1006. For example, while the information indicating that it is within the first section is notified, the determination logic switching unit 1502 inputs the recognition result to the first determination logic 1503. Also, while the information indicating that it is within the second section is notified, the determination logic switching unit 1502 inputs the recognition result to the second determination logic 1504. Also, while the information indicating that it is within the third section is notified, the determination logic switching unit 1502 inputs the recognition result to the third determination logic 1505.

[0163] Also, the determination logic switching unit 1502 notifies the poly tank determination unit 1506 and the firearm determination unit 1507 of the information indicating that it is within any one of the first to third sections.

[0164] When the poly tank determination unit 1506 is notified of the information indicating that it is within any one of the first to third sections and the recognition result is notified from the recognition result acquisition unit 1501, the poly tank determination unit 1506 determines whether a container is included in the notified recognition result. When the poly tank determination unit 1506 determines that a container is included, it notifies the fuel supply control unit 1508 of fuel supply non - permission.

[0165] The first determination logic 1503 determines whether fuel supply is permitted based on the recognition result in the first section. Also, the first determination logic 1503 notifies the fuel supply control unit 1508 of the determination result of whether fuel supply is permitted in the first section. When the first determination logic 1503 determines non - permission of fuel supply, when notifying the determination result to the fuel supply control unit 1508, it also notifies the information indicating the reason for determining non - permission of fuel supply.

[0166] The second determination logic 1504 determines whether refueling is permitted or not based on the recognition result during the second period. Further, the second determination logic 1504 notifies the refueling control unit 1508 of the determination result of whether refueling is permitted or not during the second period. In addition, when the second determination logic 1504 determines that refueling is not permitted, it notifies the refueling control unit 1508 of the determination result and also notifies information indicating the reason for determining that refueling is not permitted.

[0167] The third determination logic 1505 determines whether refueling is permitted or not based on the recognition result during the third period. Further, the third determination logic 1505 notifies the refueling control unit 1508 of the determination result of whether refueling is permitted or not during the third period. In addition, when the third determination logic 1505 determines that refueling is not permitted, it notifies the refueling control unit 1508 of the determination result and also notifies information indicating the reason for determining that refueling is not permitted.

[0168] The fire determination unit 1507 acquires the thermal sensor data measured by the thermal sensor 170_1 from the thermal sensor data acquisition unit 1008. Further, when information indicating that it is in any one of the first to third periods is notified, the fire determination unit 1507 determines whether or not a temperature region equal to or higher than a predetermined threshold is included. When the fire determination unit 1507 determines that a temperature region equal to or higher than a predetermined threshold is included, it notifies the refueling control unit 1508 that refueling is not permitted.

[0169] The refueling control unit 1508 controls the fueling valve 124_1 or the fuel pump 125_1 based on whether refueling is permitted or not, thereby controlling the fueling for the first fuel type. For example, when it is notified that refueling is permitted, the refueling control unit 1508 drives the fuel pump 125_1 to open the fueling valve 124_1. Further, the refueling control unit 1508 · When notified of refueling permission from the poly-tank determination unit 1506, drives the fuel pump 125_1 to open the fueling valve 124_1. Further, · When notified of refueling non-permission from any one of the first determination logic 1503 to the third determination logic 1505, or · When a notification of refueling prohibition is received from the fire determination unit 1507, refueling is stopped by stopping the drive of the fuel pump 125_1, or refueling is prohibited by stopping the drive of the fuel pump 125_1 and closing the fuel valve 124_1. Note that the definitions of refueling stop and refueling prohibition here are only examples, and refueling stop and refueling prohibition may be defined by other definitions. Alternatively, without distinguishing between refueling stop and refueling prohibition, the entire state where refueling cannot be performed may be referred to as refueling stop or prohibition.

[0170] In addition, the fuel supply control unit 1508 notifies the cause information acquisition unit 1509 of fuel supply permission and fuel supply non - permission. When the fuel supply control unit 1508 notifies fuel supply non - permission, it also notifies the cause information acquisition unit 1509 of information indicating the cause for which fuel supply non - permission was determined.

[0171] The cause information acquisition unit 1509 outputs fuel supply permission and fuel supply non - permission to the output control unit 1010. When the cause information acquisition unit 1509 outputs fuel supply non - permission, it also outputs information indicating the cause of fuel supply non - permission (first to fourth information, warning information, confirmation request information) notified from the fuel supply control unit 1508 to the output control unit 1010.

[0172] FIG. 15B is a diagram showing an example of the functional configuration of the second fuel supply monitoring unit of the information processing apparatus. The difference from the functional configuration of the first fuel supply monitoring unit 1009_1 shown in FIG. 15A is that the functional configuration of the second fuel supply monitoring unit 1009_2 does not include a functional unit corresponding to the poly - tank determination unit 1506. Since the other functional units of the second fuel supply monitoring unit 1009_2 are the same as those of each functional unit of the first fuel supply monitoring unit 1009_1, the description is omitted here.

[0173] Note that what was called the first determination unit in the first fuel supply monitoring unit 1009_1 is collectively referred to as the second determination unit in the second fuel supply monitoring unit 1009_2, including the first determination logic 1513 to the third determination logic 1515 and the fire determination unit 1517.

[0174] (3-2) Flow of processing using the first determination logic 1503 of the first fuel supply monitoring unit 1009_1 Next, the flow of processing using the first determination logic 1503 of the first fuel supply monitoring unit 1009_1 will be described with reference to FIGS. 16A and 16B. FIGS. 16A and 16B are examples of the first and second flowcharts showing the flow of processing using the first determination logic of the first fuel supply monitoring unit.

[0175] In step S1601, the first determination logic 1503 determines whether the currently input recognition result is a recognition result determined to be fuel supply permission (whether a predetermined permission condition is satisfied). Specifically, it is determined whether a vehicle is recognized in the first area 210 and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0176] If it is determined in step S1601 that the recognition result is a recognition result determined to be fuel supply permission (if YES in step S1601), the process proceeds to step S1602.

[0177] In step S1602, the first determination logic 1503 determines whether the recognition result determined to be fuel supply permission has continued for a predetermined time T1 (whether a predetermined permission condition has been satisfied for a certain time). If it is determined in step S1602 that it has continued for the predetermined time T1 (if YES in step S1602), the process proceeds to step S1603.

[0178] In step S1603, the first determination logic 1503 determines fuel supply permission and proceeds to step S1614 in FIG. 16B.

[0179] On the other hand, if it is determined in step S1601 that the recognition result is not a recognition result determined to be fuel supply permission (if NO in step S1601), the process proceeds to step S1604. Or, if it is determined in step S1602 that it has not continued for the predetermined time T1 (if NO in step S1602), the process proceeds to step S1604.

[0180] In step S1604, the first determination logic 1503 determines whether a state where the recognition result is not determined to be refueling permission has continued for a predetermined time T2 (however, T2 > T1).

[0181] In step S1604, if it is determined that the continuation has not occurred for the predetermined time T2 (if NO in step S1604), the process returns to step S1601. On the other hand, in step S1604, if it is determined that the continuation has occurred for the predetermined time T2 (if YES in step S1604), the process proceeds to step S1605.

[0182] In step S1605, the first determination logic 1503 determines whether the currently input recognition result is a recognition result determined to be refueling not permitted (whether a predetermined non - permission condition is satisfied). Specifically, it is determined whether a vehicle is recognized in the first area 210 and whether a refueler who is not holding the fuel nozzle in an appropriate posture is recognized (whether the fuel nozzle cannot be recognized) in the second area.

[0183] In step S1605, if it is determined that the recognition result is a recognition result determined to be refueling not permitted (if YES in step S1605), it is determined that the predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S1606.

[0184] In step S1606, the first determination logic 1503 determines refueling not permitted, acquires the first information, and then proceeds to step S1611 in FIG. 16B.

[0185] On the other hand, in step S1605, if it is determined that the recognition result is another recognition result determined to be refueling not permitted (if NO in step S1605), the process proceeds to step S1617 in FIG. 16B.

[0186] In step S1611, the first determination logic 1503 determines whether the currently input recognition result is a recognition result determined to be refueling permission (whether it satisfies a predetermined permission condition). Specifically, it determines whether a vehicle is recognized in the first area 210 and whether a refueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0187] In step S1611, if it is determined that the recognition result is not a recognition result determined to be refueling permission (if NO in step S1611), the first refueling monitoring unit 1009_1 ends the process using the first determination logic 1503.

[0188] On the other hand, in step S1611, if it is determined that the recognition result is a recognition result determined to be refueling permission (if YES in step S1611), the process proceeds to step S1612.

[0189] In step S1612, the first determination logic 1503 determines whether the recognition result determined to be refueling permission has continued for a predetermined time T1 (whether the predetermined permission condition has been satisfied for a certain time). In step S1612, if it is determined that it has continued for the predetermined time T1 (if YES in step S1612), the process proceeds to step S1613.

[0190] In step S1613, the first determination logic 1503 determines that refueling is permitted and proceeds to step S1614.

[0191] In step S1614, the determination logic switching unit 1502 determines whether it is within the first section. In step S1614, if it is determined that it is within the first section, it is determined that the process by the first determination logic 1503 is not ended (determined as NO in step S1614), and the process returns to step S1601 in FIG. 16A.

[0192] On the other hand, in step S1614, when it is determined that it is not during the first period, it is determined to end the process by the first determination logic 1503 (determined as YES in step S1614). Then, the first fueling monitoring unit 1009_1 ends the process using the first determination logic 1503.

[0193] On the other hand, in step S1612, when it is determined that the predetermined time T1 has not continued (when NO in step S1612), the process proceeds to step S1615.

[0194] In step S1615, the first determination logic 1503 determines whether a state where the recognition result is not a fueling permission has continued for a predetermined time T3 (however, T3 > T1).

[0195] In step S1615, when it is determined that the predetermined time T3 has not continued (when NO in step S1615), the process returns to step S1611. On the other hand, in step S1615, when it is determined that the predetermined time T3 has continued (when YES in step S1615), it is determined that the predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S1616.

[0196] In step S1616, the first determination logic 1503 determines non - fueling permission and acquires the first information. After that, the first fueling monitoring unit 1009_1 ends the process using the first determination logic 1503.

[0197] On the other hand, when the process proceeds from step S1605 in FIG. 16A to step S1617 in FIG. 16B, the first determination logic 1503 determines that the predetermined non - permission condition has been satisfied for a certain period of time. Specifically, the first determination logic 1503 determines that the vehicle has not been recognized in the first area 210. Alternatively, the first determination logic 1503 determines that a fueler who is not holding the fuel nozzle in an appropriate posture has been recognized (fueler recognition impossible) in the second area 220.

[0198] In step S1618, the first determination logic 1503 determines no refueling permission. After acquiring the second information, the first refueling monitoring unit 1009_1 ends the process using the first determination logic 1503.

[0199] (3-3) Flow of the process using the second determination logic 1504 of the first refueling monitoring unit 1009_1 Next, the flow of the process using the second determination logic 1504 of the first refueling monitoring unit 1009_1 will be described with reference to FIGS. 17A to 17D. FIGS. 17A to 17D are examples of the first to fourth flowcharts showing the flow of the process using the second determination logic of the first refueling monitoring unit.

[0200] In step S1701, the second determination logic 1504 determines whether the currently input recognition result is a recognition result determined as refueling permission (whether a predetermined permission condition is satisfied). Specifically, it is determined whether a vehicle is recognized in the first area 210 and whether a refueler holding the fuel nozzle in an appropriate posture is recognized in the second area.

[0201] If it is determined in step 1701 that the recognition result is a recognition result determined as refueling permission (YES in step S1701), it is determined that the predetermined permission condition is satisfied, and the process proceeds to step S1702.

[0202] In step S1702, the second determination logic 1504 determines refueling permission and proceeds to step S1734 in FIG. 17D.

[0203] On the other hand, if it is determined in step S1701 that the recognition result is not a recognition result determined as refueling permission (NO in step S1701), the process proceeds to step S1703.

[0204] In step S1703, the second determination logic 1504 determines whether the state where the recognition result is not a recognition result determined as refueling permission has continued for a predetermined time T4.

[0205] In step S1703, if it is determined that the predetermined time T4 has not elapsed continuously (if NO in step S1703), the process returns to step S1701. On the other hand, in step S1703, if it is determined that the predetermined time T4 has elapsed continuously (if YES in step S1703), the process proceeds to step S1704.

[0206] In step S1704, the second determination logic 1504 determines whether the currently input recognition result is a recognition result determined as refueling not permitted (whether a predetermined non - permission condition is satisfied). Specifically, it determines whether a vehicle is recognized in the first area 210 and whether a refueler who is not holding the fuel nozzle in an appropriate posture is recognized in the second area (whether the fuel nozzle cannot be recognized).

[0207] In step S1704, if it is determined that the recognition result is another recognition result determined as refueling not permitted (if NO in step S1704), the process proceeds to step S1721 in FIG. 17C.

[0208] On the other hand, in step S1704, if it is determined that the recognition result is a recognition result determined as refueling not permitted (if YES in step S1704), it is determined that the predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S1705.

[0209] In step S1705, the second determination logic 1504 acquires warning information.

[0210] In step S1706, the second determination logic 1504 determines whether the currently input recognition result is a recognition result determined as refueling permitted (whether a predetermined permission condition is satisfied). Specifically, it determines whether a vehicle is recognized in the first area 210 and whether a refueler who is holding the fuel nozzle in an appropriate posture is recognized in the second area.

[0211] If it is determined that the recognition result is the one that permits refueling in step S1706 (if YES in step S1706), the process proceeds to step S1707.

[0212] In step S1707, the second determination logic 1504 determines whether the recognition result that permits refueling has continued for a predetermined time T1. If it is determined in step S1707 that it has continued for the predetermined time T1 (if YES in step S1707), it is determined that the predetermined permission conditions have been satisfied for a certain period of time, and the process proceeds to step S1708.

[0213] In step S1708, the second determination logic 1504 determines that refueling is permitted and proceeds to step S1734 in FIG. 17D.

[0214] On the other hand, if it is determined in step S1706 that the recognition result is not the one that permits refueling (if NO in step S1706), the process proceeds to step S1709. Or, if it is determined in step S1707 that it has not continued for the predetermined time T1 (if NO in step S1707), the process proceeds to step S1709.

[0215] In step S1709, the second determination logic 1504 determines whether the state where the recognition result is not the one that permits refueling has continued for a predetermined time T2 (where T2 > T1).

[0216] If it is determined in step S1709 that it has not continued for the predetermined time T2 (if NO in step S1709), the process returns to step S1706. On the other hand, if it is determined in step S1709 that it has continued for the predetermined time T2 (if YES in step S1709), it is determined that the predetermined non - permission conditions have been satisfied for a certain period of time, and the process proceeds to step S1711 in FIG. 17B.

[0217] In step S1711, the second determination logic 1504 determines that refueling is not permitted, acquires the first information, and then proceeds to step S1712.

[0218] In step S1712, the second determination logic 1504 determines whether, within a predetermined time T2, the currently input recognition result has become a recognition result determined to be fueling permission (whether a predetermined permission condition is satisfied). Specifically, it is determined whether, within the predetermined time T2, a vehicle is recognized in the first area 210 and a fueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0219] In step S1712, if it is determined that the currently input recognition result has become a recognition result determined to be fueling permission within the predetermined time T2 (if YES in step S1712), it is determined that the predetermined permission condition is satisfied, and the process proceeds to step S1713.

[0220] In step S1713, the second determination logic 1504 determines fueling permission and proceeds to step S1734 in FIG. 17D.

[0221] On the other hand, in step S1712, if it is determined that the recognition result has not become a recognition result determined to be fueling permission within the predetermined time T2 (if NO in step S1712), the first fueling monitoring unit 1009_1 ends the process using the second determination logic 1504.

[0222] On the other hand, when the process proceeds from step S1704 to step S1721 in FIG. 17C, the second determination logic 1504 determines that a predetermined non - permission condition has been satisfied for a certain period of time. Specifically, the second determination logic 1504 determines that no vehicle is recognized in the first area 210. Alternatively, the second determination logic 1504 determines that a fueler who does not hold the fuel nozzle in an appropriate posture (fueler recognition impossible) is recognized in the second area 220.

[0223] In step S1722, the second determination logic 1504 acquires warning information.

[0224] In step S1723, the second determination logic 1504 determines whether the currently input recognition result is a recognition result determined to be refueling permission (whether it satisfies a predetermined permission condition). Specifically, it determines whether a vehicle is recognized in the first area 210 and whether a refueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0225] In step S1723, if it is determined that the recognition result is a recognition result determined to be refueling permission (if YES in step S1723), the process proceeds to step S1724.

[0226] In step S1724, the second determination logic 1504 determines whether the recognition result determined to be refueling permission has continued for a predetermined time T1. In step S1724, if it is determined that it has continued for the predetermined time T1 (if YES in step S1724), it is determined that the predetermined permission condition has been satisfied for a certain period of time, and the process proceeds to step S1725.

[0227] In step S1725, the second determination logic 1504 determines that refueling is permitted and proceeds to step S1734 in FIG. 17D.

[0228] On the other hand, in step S1723, if it is determined that the recognition result is not a recognition result determined to be refueling permission (if NO in step S1723), the process proceeds to step S1726. Or, in step S1724, if it is determined that it has not continued for the predetermined time T1 (if NO in step S1724), the process proceeds to step S1726.

[0229] In step S1726, the second determination logic 1504 determines whether the state where the recognition result is not a recognition result determined to be refueling permission has continued for a predetermined time T2 (where T2>T1).

[0230] In step S1726, if it is determined that the predetermined time T2 has not elapsed continuously (if NO in step S1726), the process returns to step S1723. On the other hand, in step S1726, if it is determined that the predetermined time T2 has elapsed continuously (if YES in step S1726), it is determined that the predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S1731 in FIG. 17D.

[0231] In step S1731, the second determination logic 1504 determines non - fueling permission. After acquiring the second information, the process proceeds to step S1732.

[0232] In step S1732, the second determination logic 1504 determines whether the currently input recognition result has become a recognition result for which fueling permission is determined (whether the predetermined permission condition has been satisfied) within the predetermined time T2. Specifically, it is determined whether, within the predetermined time T2, the vehicle is recognized in the first area 210 and a fueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0233] In step S1732, if it is determined that the currently input recognition result has become a recognition result for which fueling permission is determined within the predetermined time T2 (if YES in step S1732), it is determined that the predetermined permission condition has been satisfied, and the process proceeds to step S1733.

[0234] In step S1733, the second determination logic 1504 determines fueling permission and proceeds to step S1734.

[0235] In step S1734, the determination logic switching unit 1502 determines whether it is during the second section. In step S1734, if it is determined that it is during the second section, it is determined that the process by the second determination logic 1504 is not terminated (if determined NO in step S1734), and the process returns to step S1701 in FIG. 17A.

[0236] On the other hand, in step S1734, if it is determined that it is not within the second interval, it is determined to end the process by the second determination logic 1504 (determined as YES in step S1734). Then, the first fuel supply monitoring unit 1009_1 ends the process using the second determination logic 1504.

[0237] Also, in step S1732, if it is not determined that the recognition result is a determination of fuel supply permission within the predetermined time T2 (if NO in step S1732), the first fuel supply monitoring unit 1009_1 ends the process using the second determination logic 1504.

[0238] (3-4) Flow of the process using the third determination logic 1505 of the first fuel supply monitoring unit 1009_1 Next, the flow of the process using the third determination logic 1505 of the first fuel supply monitoring unit 1009_1 will be described with reference to FIGS. 18A and 18B. FIGS. 18A and 18B are examples of the first and second flowcharts showing the flow of the process using the third determination logic of the first fuel supply monitoring unit.

[0239] In step S1801, the third determination logic 1505 determines whether the currently input recognition result is a recognition result determined as fuel supply permission (whether the predetermined permission conditions are satisfied). Specifically, it is determined whether a vehicle is recognized in the first area 210 and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0240] In step 1801, if it is determined that the recognition result is a recognition result determined as fuel supply permission (if YES in step S1801), it is determined that the predetermined permission conditions are satisfied, and the process proceeds to step S1802.

[0241] In step S1802, the third determination logic 1505 determines fuel supply permission and proceeds to step S1813 in FIG. 18B.

[0242] On the other hand, if it is determined in step S1801 that the recognition result is not a refueling permission determination (if NO in step S1801), the process proceeds to step S1803.

[0243] In step S1803, the third determination logic 1505 determines whether or not a state where the recognition result is not a refueling permission determination has continued for a predetermined time T5.

[0244] In step S1803, if it is determined that it has not continued for the predetermined time T5 (if NO in step S1803), the process returns to step S1801. On the other hand, in step S1803, if it is determined that it has continued for the predetermined time T5 (if YES in step S1803), the process proceeds to step S1804.

[0245] In step S1804, the third determination logic 1505 determines whether the currently input recognition result is a recognition result determined as refueling not permitted (whether or not a predetermined non - permission condition is satisfied). Specifically, it is determined whether a vehicle is recognized in the first area 210 and whether a refueler who is not holding the fuel nozzle in an appropriate posture is recognized (whether the fuel nozzle is unrecognizable) in the second area.

[0246] In step S1804, if it is determined that the recognition result is another recognition result determined as refueling not permitted (if NO in step S1804), the process proceeds to step S1806.

[0247] In step S1806, the third determination logic 1505 determines that a predetermined non - permission condition has been satisfied for a certain period of time. Specifically, the third determination logic 1505 determines that no vehicle is recognized in the first area 210. Alternatively, the third determination logic 1505 determines that a refueler who is not holding the fuel nozzle in an appropriate posture is recognized (the refueler is unrecognizable) in the second area 220.

[0248] In step S1807, the third determination logic 1505 determines that refueling is not permitted. After acquiring the second information, it proceeds to step S1811 in FIG. 18B.

[0249] On the other hand, if it is determined in step S1804 that the recognition result is a recognition result indicating that refueling is not permitted (if YES in step S1804), it is determined that a predetermined non-permission condition has been satisfied for a certain period of time, and the process proceeds to step S1805.

[0250] In step S1805, the third determination logic 1505 determines that refueling is not permitted. After acquiring the first information, it proceeds to step S1811 in FIG. 18B.

[0251] In step S1811, the third determination logic 1505 determines whether the currently input recognition result has become a recognition result indicating that refueling is permitted (whether a predetermined permission condition has been satisfied) within a predetermined time T2. Specifically, it is determined whether, within the predetermined time T2, a vehicle is recognized in the first area 210 and a refueler who holds the fuel nozzle in an appropriate posture is recognized in the second area.

[0252] If it is determined in step S1811 that the currently input recognition result has become a recognition result indicating that refueling is permitted within the predetermined time T2 (if YES in step S1811), it is determined that a predetermined permission condition has been satisfied, and the process proceeds to step S1812.

[0253] In step S1812, the third determination logic 1505 determines that refueling is permitted and proceeds to step S1813.

[0254] In step S1813, the determination logic switching unit 1502 determines whether it is during the third section. If it is determined in step S1813 that it is during the third section, it is determined that the process by the third determination logic 1505 is not terminated (if determined NO in step S1813), and the process returns to step S1801 in FIG. 18A.

[0255] On the other hand, in step S1813, if it is determined that it is not during the third interval, it is determined to end the process by the third determination logic 1505 (determined as YES in step S1813). Then, the first fuel supply monitoring unit 1009_1 ends the process using the third determination logic 1505.

[0256] On the other hand, in step S1811, if it is determined that the currently input recognition result has not become a recognition result determined as fuel supply permission within the predetermined time T2 (if NO in step S1811), the process proceeds to step S1814.

[0257] In step S1814, the third determination logic 1505 determines whether the state of not being a recognition result determined as fuel supply permission has continued for the predetermined time T6. In step S1814, if it is determined that the state of not being a recognition result determined as fuel supply permission has not continued for the predetermined time T6 (if NO in step S1814), it waits until it continues for the predetermined time T6.

[0258] On the other hand, in step S1814, if it is determined that the state of not being a recognition result determined as fuel supply permission has continued for the predetermined time T6 (if YES in step S1814), the process proceeds to step S1815.

[0259] In step S1815, the third determination logic 1505 acquires confirmation request information, and the first fuel supply monitoring unit 1009_1 ends the process using the third determination logic 1505.

[0260] (3-5) Flow of processing using other determination logics of the first fuel supply monitoring unit 1009_1 Next, the flow of processing using other determination logics of the first fuel supply monitoring unit 1009_1 (processing using the poly-tank determination unit 1506 and processing using the fire determination unit 1507) will be described with reference to FIG. 19. FIG. 19 is an example of a flowchart showing the flow of processing using the poly-tank determination unit of the first fuel supply monitoring unit and the flow of processing using the fire determination unit.

[0261] First, the processing flow using the poly-tank determination unit 1506 of the first fuel supply monitoring unit 1009_1 will be described.

[0262] In step S1901 of FIG. 19(a), the poly-tank determination unit 1506 determines whether a container is included (whether a predetermined non-permission condition is satisfied) in the currently input recognition result. If it is determined in step S1901 that a container is included (if YES in step S1901), it is determined that the predetermined non-permission condition is satisfied, and the process proceeds to step S1902.

[0263] In step S1902, the poly-tank determination unit 1506 determines that fuel supply is not permitted, acquires the third information, and then proceeds to step S1903.

[0264] On the other hand, if it is determined in step S1901 that a container is not included (if NO in step S1901), the process directly proceeds to step S1903.

[0265] In step S1903, the poly-tank determination unit 1506 determines whether to end the process. If it corresponds to any of the first section to the third section, in step S1903, the poly-tank determination unit 1506 determines to continue the process (determines NO in step S1903), and returns to step S1901.

[0266] On the other hand, if it does not correspond to any of the first section to the third section, in step S1903, the first fuel supply monitoring unit 1009_1 ends the process using the poly-tank determination unit 1506.

[0267] Subsequently, the processing flow using the poly-tank determination unit 1506 of the second fuel supply monitoring unit 1009_2 will be described.

[0268] In step S1911 of FIG. 19(b), the fire determination unit 1507 determines whether a temperature equal to or higher than a predetermined threshold has been measured in a region other than the first region 210 in the currently input thermal sensor data (whether a predetermined non-permissible condition is satisfied). Note that the region other than the first region 210 is set to avoid misjudging fire detection because the hood of the vehicle included in the first region 210 may be at a high temperature.

[0269] If it is determined in step S1911 that a temperature equal to or higher than the predetermined threshold has been measured (if YES in step S1911), it is determined that the predetermined non-permissible condition is satisfied, and the process proceeds to step S1912.

[0270] In step S1912, the fire determination unit 1507 determines that refueling is not permitted, acquires the fourth information, and then proceeds to step S1913.

[0271] On the other hand, if it is determined in step S1911 that a temperature equal to or higher than the predetermined threshold has not been measured (if NO in step S1911), the process directly proceeds to step S1913.

[0272] In step S1913, the fire determination unit 1507 determines whether to end the process. If it corresponds to any of the first to third intervals, in step S1913, the fire determination unit 1507 determines to continue the process (determines NO in step S1913) and returns to step S1911.

[0273] On the other hand, if it does not correspond to any of the first to third intervals, in step S1913, the first fuel supply monitoring unit 1009_1 ends the process using the fire determination unit 1507.

[0274] (3-6) Flow of processing using the first determination logic 1513 of the second fuel supply monitoring unit 1009_2 Next, the processing flow using the first determination logic 1513 of the second fuel supply monitoring unit 1009_2 will be described with reference to FIGS. 20A and 20B. FIGS. 20A and 20B are examples of first and second flowcharts showing the processing flow using the first determination logic of the second fuel supply monitoring unit.

[0275] In step S2001, the first determination logic 1513 determines whether the currently input recognition result is a recognition result determined to be fuel supply permitted (whether it satisfies a predetermined permission condition). Specifically, it determines whether a container is recognized and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized.

[0276] If it is determined in step S2001 that the recognition result is a recognition result determined to be fuel supply permitted (if YES in step S2001), the process proceeds to step S2002.

[0277] In step S2002, the first determination logic 1513 determines whether the recognition result determined to be fuel supply permitted has continued for a predetermined time T1 (whether the predetermined permission condition has been satisfied for a certain time). If it is determined in step S2002 that it has continued for the predetermined time T1 (if YES in step S2002), the process proceeds to step S2003.

[0278] In step S2003, the first determination logic 1513 determines that fuel supply is permitted and proceeds to step S2014 in FIG. 20B.

[0279] On the other hand, if it is determined in step S2001 that the recognition result is not a recognition result determined to be fuel supply permitted (if NO in step S2001), the process proceeds to step S2004. Or, if it is determined in step S2002 that it has not continued for the predetermined time T1 (if NO in step S2002), the process proceeds to step S2004.

[0280] In step S2004, the first determination logic 1513 determines whether a state where the recognition result is not determined as fueling permission has continued for a predetermined time T2 (where T2 > T1).

[0281] In step S2004, if it is determined that the predetermined time T2 has not continued (if NO in step S2004), the process returns to step S2001. On the other hand, in step S2004, if it is determined that the predetermined time T2 has continued (if YES in step S2004), the process proceeds to step S2005.

[0282] In step S2005, the first determination logic 1513 determines whether the currently input recognition result is a recognition result determined as fueling not permitted (whether a predetermined non - permission condition is satisfied). Specifically, it determines whether a fueler who has a recognized container and is not holding the fueling nozzle in an appropriate posture (whether the fueling nozzle is not recognized) is recognized.

[0283] In step S2005, if it is determined that the recognition result is a recognition result determined as fueling not permitted (if YES in step S2005), it is determined that the predetermined non - permission condition has been satisfied for a certain time, and the process proceeds to step S2006.

[0284] In step S2006, after the first determination logic 1513 determines fueling not permitted and acquires the first information, the process proceeds to step S2011 in FIG. 20B.

[0285] On the other hand, in step S2005, if it is determined that the recognition result is another recognition result determined as fueling not permitted (if NO in step S2005), the process proceeds to step S2017 in FIG. 20B.

[0286] In step S2011, the first determination logic 1513 determines whether the currently input recognition result is a recognition result determined as refueling permission (whether it satisfies a predetermined permission condition). Specifically, it determines whether a container is recognized and whether a refueler who holds the fuel nozzle in an appropriate posture is recognized.

[0287] If it is determined in step S2011 that the recognition result is not a recognition result determined as refueling permission (if NO in step S2011), the process proceeds to step S2017.

[0288] On the other hand, if it is determined in step S2011 that the recognition result is a recognition result determined as refueling permission (if YES in step S2011), the second fueling monitoring unit 1009_2 ends the process using the first determination logic 1513.

[0289] In step S2012, the first determination logic 1513 determines whether the recognition result determined as refueling permission has continued for a predetermined time T1 (whether the predetermined permission condition has been satisfied for a certain time). If it is determined in step S2012 that it has continued for the predetermined time T1 (if YES in step S2012), the process proceeds to step S2013.

[0290] In step S2013, the first determination logic 1513 determines that refueling is permitted and proceeds to step S2014.

[0291] In step S2014, the determination logic switching unit 1512 determines whether it is within the first section. If it is determined in step S2014 that it is within the first section, it is determined that the process by the first determination logic 1513 is not ended (determined as NO in step S2014), and the process returns to step S2001 in FIG. 20A.

[0292] On the other hand, in step S2014, when it is determined that it is not within the first period, it is determined to end the process by the first determination logic 1513 (determined as YES in step S2014). Then, the second fuel supply monitoring unit 1009_2 ends the process using the first determination logic 1513.

[0293] On the other hand, in step S2012, when it is determined that the predetermined time T1 has not continued (when NO in step S2012), the process proceeds to step S2015.

[0294] In step S2015, the first determination logic 1513 determines whether a state where the recognition result is not a fuel supply permission has continued for a predetermined time T3 (where T3 > T1).

[0295] In step S2015, when it is determined that the predetermined time T3 has not continued (when NO in step S2015), the process returns to step S2011. On the other hand, in step S2015, when it is determined that the predetermined time T3 has continued (when YES in step S2015), it is determined that the predetermined non - permission condition has been satisfied for a certain period. Then, the second fuel supply monitoring unit 1009_2 ends the process using the first determination logic 1513.

[0296] On the other hand, when the process proceeds from step S2005 in FIG. 20A to step S2017 in FIG. 20B, the first determination logic 1513 determines that the predetermined non - permission condition has been satisfied for a certain period. Specifically, the first determination logic 1513 determines that the container has not been recognized. Alternatively, the first determination logic 1513 determines that a fueler who is not holding the fuel nozzle in an appropriate posture has been recognized (fueler recognition impossible).

[0297] In step S2018, the first determination logic 1513 determines fuel supply non - permission, acquires second information, and the second fuel supply monitoring unit 1009_2 ends the process using the first determination logic 1513.

[0298] (3-7) Flow of processing using the second determination logic 1514 of the second fuel supply monitoring unit 1009_2 Next, the flow of processing using the second determination logic 1514 of the second fuel supply monitoring unit 1009_2 will be described with reference to FIGS. 21A to 21D. FIGS. 21A to 21D are examples of the first to fourth flowcharts showing the flow of processing using the second determination logic of the second fuel supply monitoring unit.

[0299] In step S2101, the second determination logic 1514 determines whether the currently input recognition result is a recognition result determined to be fuel supply permission (whether it satisfies a predetermined permission condition). Specifically, it determines whether a container is recognized and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized.

[0300] If it is determined in step 2101 that the recognition result is a recognition result determined to be fuel supply permission (if YES in step S2101), it is determined that the predetermined permission condition is satisfied, and the process proceeds to step S2102.

[0301] In step S2102, the second determination logic 1514 determines fuel supply permission and proceeds to step S2134 in FIG. 21D.

[0302] On the other hand, if it is determined in step S2101 that the recognition result is not a recognition result determined to be fuel supply permission (if NO in step S2101), the process proceeds to step S2103.

[0303] In step S2103, the second determination logic 1514 determines whether the state where the recognition result is not a recognition result determined to be fuel supply permission has continued for a predetermined time T4.

[0304] If it is determined in step S2103 that it has not continued for the predetermined time T4 (if NO in step S2103), the process returns to step S2101. On the other hand, if it is determined in step S2103 that it has continued for the predetermined time T4 (if YES in step S2103), the process proceeds to step S2104.

[0305] In step S2104, the second determination logic 1514 determines whether the currently input recognition result is a recognition result determined as refueling not permitted (whether a predetermined non-permission condition is satisfied). Specifically, it determines whether a refueler who has a recognized container and does not hold the fuel nozzle in an appropriate posture is recognized (whether the fuel nozzle cannot be recognized).

[0306] In step S2104, if it is determined that the recognition result is another recognition result determined as refueling not permitted (if NO in step S2104), the process proceeds to step S2121 in FIG. 21C.

[0307] On the other hand, in step S2104, if it is determined that the recognition result is a recognition result determined as refueling not permitted (if YES in step S2104), it is determined that a predetermined non-permission condition has been satisfied for a certain period of time, and the process proceeds to step S2105.

[0308] In step S2105, the second determination logic 1514 acquires warning information.

[0309] In step S2106, the second determination logic 1514 determines whether the currently input recognition result is a recognition result determined as refueling permitted (whether a predetermined permission condition is satisfied). Specifically, it determines whether a refueler who has a recognized container and holds the fuel nozzle in an appropriate posture is recognized.

[0310] In step S2106, if it is determined that the recognition result is a recognition result determined as refueling permitted (if YES in step S2106), the process proceeds to step S2107.

[0311] In step S2107, the second determination logic 1514 determines whether the recognition result determined as fueling permission has continued for a predetermined time T1. In step S2107, if it is determined that the continuation has been for the predetermined time T1 (if YES in step S2107), it is determined that the predetermined permission conditions have been satisfied for a certain period of time, and the process proceeds to step S2108.

[0312] In step S2108, the second determination logic 1514 determines fueling permission and proceeds to step S2134 in FIG. 21D.

[0313] On the other hand, in step S2106, if it is determined that the recognition result is not the one determined as fueling permission (if NO in step S2106), the process proceeds to step S2109. Or, in step S2107, if it is determined that the continuation has not been for the predetermined time T1 (if NO in step S2107), the process proceeds to step S2109.

[0314] In step S2109, the second determination logic 1514 determines whether the state where the recognition result is not the one determined as fueling permission has continued for a predetermined time T2 (where T2 > T1).

[0315] In step S2109, if it is determined that the continuation has not been for the predetermined time T2 (if NO in step S2109), the process returns to step S2106. On the other hand, in step S2109, if it is determined that the continuation has been for the predetermined time T2 (if YES in step S2109), it is determined that the predetermined non - permission conditions have been satisfied for a certain period of time, and the process proceeds to step S2111 in FIG. 21B.

[0316] In step S2111, the second determination logic 1514 determines non - fueling permission, and after acquiring the first information, proceeds to step S2112.

[0317] In step S2112, the second determination logic 1514 determines whether, within a predetermined time T2, the currently input recognition result has become a recognition result determined to be fueling permission (whether a predetermined permission condition has been satisfied). Specifically, it determines whether, within the predetermined time T2, the container has been recognized and a fueler who holds the fueling nozzle in an appropriate posture has been recognized.

[0318] In step S2112, if it is determined that, within the predetermined time T2, the currently input recognition result has become a recognition result determined to be fueling permission (if YES in step S2112), it is determined that the predetermined permission condition has been satisfied, and the process proceeds to step S2113.

[0319] In step S2113, the second determination logic 1514 determines that fueling is permitted and proceeds to step S2134 in FIG. 21D.

[0320] On the other hand, in step S2112, if it is determined that, within the predetermined time T2, the recognition result has not become a recognition result determined to be fueling permission (if NO in step S2112), the second fueling monitoring unit 1009_2 ends the process using the second determination logic 1514.

[0321] On the other hand, when proceeding from step S2104 to step S2121 in FIG. 21C, the second determination logic 1514 determines that a predetermined non - permission condition has been satisfied for a certain period of time. Specifically, the second determination logic 1514 determines that the container has not been recognized. Alternatively, the second determination logic 1514 determines that a fueler who does not hold the fueling nozzle in an appropriate posture has been recognized (the fueler cannot be recognized).

[0322] In step S2122, the second determination logic 1504 acquires warning information.

[0323] In step S2123, the second determination logic 1514 determines whether the currently input recognition result is a recognition result determined to be fueling permission (whether it satisfies a predetermined permission condition). Specifically, it determines whether a container is recognized and whether a fueler who holds the fuel nozzle in an appropriate posture is recognized.

[0324] In step S2123, if it is determined that the recognition result is a recognition result determined to be fueling permission (if YES in step S2123), the process proceeds to step S2124.

[0325] In step S2124, the second determination logic 1514 determines whether the recognition result determined to be fueling permission has continued for a predetermined time T1. In step S2124, if it is determined that it has continued for the predetermined time T1 (if YES in step S2124), it is determined that the predetermined permission condition has been satisfied for a certain period of time, and the process proceeds to step S2125.

[0326] In step S2125, the second determination logic 1514 determines fueling permission and proceeds to step S2134 in FIG. 21D.

[0327] On the other hand, in step S2123, if it is determined that the recognition result is not a recognition result determined to be fueling permission (if NO in step S2123), the process proceeds to step S2126. Or, in step S2124, if it is determined that it has not continued for the predetermined time T1 (if NO in step S2124), the process proceeds to step S2126.

[0328] In step S2126, the second determination logic 1514 determines whether the state where the recognition result is not a recognition result determined to be fueling permission has continued for a predetermined time T2 (where T2 > T1).

[0329] In step S2126, if it is determined that the predetermined time T2 has not continued (if NO in step S2126), the process returns to step S2123. On the other hand, in step S2126, if it is determined that the predetermined time T2 has continued (if YES in step S2126), it is determined that the predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S2131 in FIG. 21D.

[0330] In step S2131, the second determination logic 1514 determines non - fueling permission, and after acquiring the second information, proceeds to step S2132.

[0331] In step S2132, the second determination logic 1514 determines whether the currently input recognition result has become a recognition result determined as fueling permission (whether the predetermined permission condition has been satisfied) within the predetermined time T2. Specifically, it determines whether, within the predetermined time T2, the container has been recognized and a fueler holding the fuel nozzle in an appropriate posture has been recognized.

[0332] In step S2132, if it is determined that the currently input recognition result has become a recognition result determined as fueling permission within the predetermined time T2 (if YES in step S2132), it is determined that the predetermined permission condition has been satisfied, and the process proceeds to step S2133.

[0333] In step S2133, the second determination logic 1514 determines fueling permission and proceeds to step S2134.

[0334] In step S2134, the determination logic switching unit 1512 determines whether it is within the second section. In step S2134, if it is determined that it is within the second section, it is determined that the process by the second determination logic 1514 is not terminated (if determined as NO in step S2134), and the process returns to step S2101 in FIG. 21A.

[0335] On the other hand, in step S2134, if it is determined that it is not during the second period, it is determined to end the process by the second determination logic 1514 (determined as YES in step S2134). Then, the second fuel supply monitoring unit 1009_2 ends the process using the second determination logic 1514.

[0336] Also, in step S2132, if it is not determined that the recognition result is a determination of fuel supply permission within the predetermined time T2 (NO in step S2132), the second fuel supply monitoring unit 1009_2 ends the process using the second determination logic 1514.

[0337] (3-8) Flow of the process using the third determination logic 1515 of the second fuel supply monitoring unit 1009_2 Next, the flow of the process using the third determination logic 1515 of the second fuel supply monitoring unit 1009_2 will be described with reference to FIGS. 22A and 22B. FIGS. 22A and 22B are examples of the first and second flowcharts showing the flow of the process using the third determination logic of the second fuel supply monitoring unit.

[0338] In step S2201, the third determination logic 1515 determines whether the currently input recognition result is a recognition result determined as fuel supply permission (whether it satisfies a predetermined permission condition). Specifically, it determines whether a container is recognized and whether a fueler holding the fuel nozzle in an appropriate posture is recognized.

[0339] In step 2201, if it is determined that the recognition result is a recognition result determined as fuel supply permission (YES in step S2201), it is determined that the predetermined permission condition is satisfied, and the process proceeds to step S2202.

[0340] In step S2202, the third determination logic 1515 determines fuel supply permission and proceeds to step S2213 in FIG. 22B.

[0341] On the other hand, if it is determined in step S2201 that the recognition result is not the one for which refueling is permitted (if NO in step S2201), the process proceeds to step S2203.

[0342] In step S2203, the third determination logic 1515 determines whether the state where the recognition result is not the one for which refueling is permitted has continued for a predetermined time T5.

[0343] In step S2203, if it is determined that it has not continued for the predetermined time T5 (if NO in step S2203), the process returns to step S2201. On the other hand, in step S2203, if it is determined that it has continued for the predetermined time T5 (if YES in step S2203), the process proceeds to step S2204.

[0344] In step S2204, the third determination logic 1515 determines whether the currently input recognition result is a recognition result for which refueling is not permitted (whether a predetermined non - permission condition is satisfied). Specifically, it determines whether a container is recognized and whether a refueler who is not holding the fuel nozzle in an appropriate posture is recognized (whether the fuel nozzle cannot be recognized).

[0345] In step S2204, if it is determined that it is another recognition result for which refueling is not permitted (if NO in step S2204), the process proceeds to step S2206.

[0346] In step S2206, the third determination logic 1515 determines that a predetermined non - permission condition has been satisfied for a certain period of time. Specifically, the third determination logic 1515 determines that the container has not been recognized. Alternatively, the third determination logic 1515 determines that a refueler who is not holding the fuel nozzle in an appropriate posture has been recognized (the refueler cannot be recognized).

[0347] In step S2207, the third determination logic 1515 determines that refueling is not permitted, acquires the second information, and then proceeds to step S2211 in FIG. 22B.

[0348] On the other hand, in step S2204, if it is determined that the recognition result is a refueling non - permission determination result (if YES in step S2204), it is determined that a predetermined non - permission condition has been satisfied for a certain period of time, and the process proceeds to step S2205.

[0349] In step S2205, the third determination logic 1515 determines refueling non - permission, acquires the first information, and then proceeds to step S2211 in FIG. 22B.

[0350] In step S2211, the third determination logic 1515 determines whether the currently input recognition result has become a recognition result for which refueling is permitted (whether a predetermined permission condition has been satisfied) within a predetermined time T2. Specifically, it is determined whether, within the predetermined time T2, the container has been recognized and a refueler who holds the fuel nozzle in an appropriate posture has been recognized.

[0351] In step S2211, if it is determined that the currently input recognition result has become a recognition result for which refueling is permitted within the predetermined time T2 (if YES in step S2211), it is determined that a predetermined permission condition has been satisfied, and the process proceeds to step S2212.

[0352] In step S2212, the third determination logic 1515 determines refueling permission and proceeds to step S2213.

[0353] In step S2213, the determination logic switching unit 1512 determines whether it is within the third section. In step S2213, if it is determined that it is within the third section, it is determined that the process by the third determination logic 1515 is not terminated (if determined NO in step S2213), and the process returns to step S2201 in FIG. 22A.

[0354] On the other hand, in step S2213, when it is determined that it is not during the third period, it is determined to end the process by the third determination logic 1515 (determined as YES in step S2213). Then, the second fuel supply monitoring unit 1009_2 ends the process using the third determination logic 1515.

[0355] On the other hand, in step S2211, when it is determined that the currently input recognition result has not become a recognition result determined as fuel supply permission within the predetermined time T2 (when it is NO in step S2211), the process proceeds to step S2214.

[0356] In step S2214, the third determination logic 1515 determines whether the state of not being a recognition result determined as fuel supply permission has continued for the predetermined time T6. In step S2214, when it is determined that the state of not being a recognition result determined as fuel supply permission has not continued for the predetermined time T6 (when it is NO in step S2214), it waits until it continues for the predetermined time T6.

[0357] On the other hand, in step S2214, when it is determined that the state of not being a recognition result determined as fuel supply permission has continued for the predetermined time T6 (when it is YES in step S2214), the process proceeds to step S2215.

[0358] In step S2215, the third determination logic 1515 acquires confirmation request information, and the second fuel supply monitoring unit 1009_2 ends the process using the third determination logic 1515.

[0359] (3-9) Flow of processing using other determination logic of the second fuel supply monitoring unit 1009_2 Next, the flow of processing using other determination logic of the second fuel supply monitoring unit 1009_2 (processing using the fire determination unit 1517) will be described with reference to FIG. 23. FIG. 23 is an example of a flowchart showing the flow of processing using the fire determination unit of the second fuel supply monitoring unit.

[0360] In step S2301, the fire determination unit 1517 determines whether a temperature equal to or higher than a predetermined threshold has been measured for the currently input thermal sensor data (whether a predetermined non-permissible condition is satisfied).

[0361] In step S2301, if it is determined that a temperature equal to or higher than the predetermined threshold has been measured (if YES in step S2301), it is determined that the predetermined non-permissible condition is satisfied, and the process proceeds to step S2302.

[0362] In step S2302, the fire determination unit 1517 determines no fuel supply is allowed, acquires the fourth information, and then proceeds to step S2303.

[0363] On the other hand, in step S2301, if it is determined that a temperature equal to or higher than the predetermined threshold has not been measured (if NO in step S2301), the process directly proceeds to step S2303.

[0364] In step S2303, the fire determination unit 1517 determines whether to end the process. If it corresponds to any of the first to third intervals, in step S2303, the fire determination unit 1517 determines to continue the process (determines NO in step S2303) and returns to step S2301.

[0365] On the other hand, if it does not correspond to any of the first to third intervals, in step S2303, the second fuel supply monitoring unit 1009_2 ends the process using the fire determination unit 1517.

[0366] (4) Details of the output control unit 1010 Next, as details of the output control unit 1010, the functional configuration and specific examples of the processes of the output control unit 1010 will be described.

[0367] (4-1) Details of the functional configuration of the output control unit 1010 FIG. 24 is a diagram showing an example of the functional configuration of the output control unit of the information processing apparatus. As shown in FIG. 24, the output control unit 1010 includes an output information acquisition unit 2401 and a transmission unit 2402.

[0368] The output information acquisition unit 2401 obtains from the first fueling monitoring unit 1009_1, · information indicating fueling permission, · information indicating non - fueling permission and the first to fourth information, · warning information, · confirmation request information, and acquires them.

[0369] Also, the output information acquisition unit 2401 obtains from the second fueling monitoring unit 1009_2, · information indicating fueling permission, · information indicating non - fueling permission and the first, second, and fourth information, · warning information, · confirmation request information, and acquires them.

[0370] Also, the output information acquisition unit 2401 notifies the transmission unit 2402 of the information obtained from the first fueling monitoring unit 1009_1. At this time, the output information acquisition unit 2401 refers to the output information storage unit 2403, · cause information (information indicating the cause of non - fueling permission) corresponding to the first to fourth information, · messages corresponding to the first to fourth information, · display information corresponding to the first to fourth information, · messages corresponding to the warning information, · display information corresponding to the warning information, · cause information corresponding to the confirmation request information, · messages corresponding to the confirmation request information, · display information corresponding to the confirmation request information, and acquires them. Then, the output information acquisition unit 2401 converts the first to fourth information, warning information, and confirmation request information into the corresponding cause information, messages, and display information, and then notifies the transmission unit 2402.

[0371] Similarly, the output information acquisition unit 2401 notifies the information obtained from the second fueling monitoring unit 1009_2 to the transmission unit 2402. At this time, the output information acquisition unit 2401 refers to the output information storage unit 2403 and · Cause information corresponding to the first information, the second information, and the fourth information (information indicating the cause of refueling not permitted), and · Messages corresponding to the first information, the second information, and the fourth information, and · Display information corresponding to the first information, the second information, and the fourth information, and · Messages corresponding to the warning information, and · Display information corresponding to the warning information, and · Cause information corresponding to the confirmation request information, and · Messages corresponding to the confirmation request information, and · Display information corresponding to the confirmation request information, and acquires them. Then, the output information acquisition unit 2401 converts the first information, the second information, the fourth information, the warning information, and the confirmation request information into the corresponding cause information, messages, and display information, and then notifies the transmission unit 2402.

[0372] The transmission unit 2402 transmits the information notified by the output information acquisition unit 2401 to · The display device 912, which is a display device for the staff of the service station 110', · The voice output device 122_1 of the first fuel dispenser 120_1 or the voice output device 122_2 of the second fuel dispenser 120_2, · The fueling data display 123_1 of the first fuel dispenser 120_1 or the fueling data display 123_2 of the second fuel dispenser 120_2. Specifically, the transmission unit 2402 controls the information obtained from the first fueling monitoring unit 1009_1,

[0373] · Information indicating fueling permission, information indicating fueling non - permission, cause information, and confirmation request information to be displayed on the display device 912, · Messages to be voice - output by the voice output device 122_1 of the first fuel dispenser 120_1, · Messages to be voice - output by the voice output device 122_2 of the second fuel dispenser 120_2, ·Control so that the indication information is displayed on the fueling data display 123_1 of the first fuel dispenser 120_1.

[0374] Similarly, the transmission unit 2402 controls so that the information acquired from the second fuel monitoring unit 1009_2, ·information indicating fueling permission, information indicating non - permission for fueling, cause information, and confirmation request information are displayed on the display device 912, ·a message is output as voice by the voice output device 122_2 of the second fuel dispenser 120_2, ·control so that the display information is displayed on the fueling data display 123_2 of the second fuel dispenser 120_2.

[0375] (4 - 2) Specific example of the processing of the output control unit 1010 Next, as a specific example of the processing of the output control unit 1010, specific examples of the cause information, message, and display information converted by the output control unit 1010 will be described. FIG. 25 is a diagram showing an example of the output information.

[0376] In FIG. 25, the output information 2500 includes cause information, a message, and display information corresponding to the first to fourth information, warning information, and confirmation request information acquired from the first fuel monitoring unit 1009_1. Also, the output information 2510 includes cause information, a message, and display information corresponding to the first information, second information, fourth information, warning information, and confirmation request information acquired from the second fuel monitoring unit 1009_2.

[0377] As shown in FIG. 25, the output information 2500 and 2510 include, as information items, "information type", "cause information", "message output to the customer", and "display information on the fueling data display of the fuel dispenser".

[0378] For "information type", in the case of the output information 2500, any one of the first to fourth information is stored, and in the case of the output information 2510, any one of the first information, second information, and fourth information is stored.

[0379] In the case of output information 2500, any one of "nozzle indeterminate", "judgment impossible", "refueling from outside the vehicle", and "fire detection" is stored in the "cause information", and in the case of output information 2510, any one of "nozzle indeterminate", "judgment impossible", and "fire detection" is stored.

[0380] In the case of output information 2500, messages corresponding to the first to fourth information respectively, which are output as voice from the voice output device 122_1, are stored in the "message output to the customer". Also, in the case of output information 2510, messages corresponding to the first, second, and fourth information respectively, which are output as voice from the voice output device 122_2, are stored.

[0381] In the case of output information 2500, display information corresponding to the first to fourth information respectively, which is displayed on the fueling data display 123_1 of the first fuel dispenser 120_1, is stored in the "display information of the fueling data display of the fuel dispenser". Also, in the case of output information 2510, display information corresponding to the first, second, and fourth information respectively, which is displayed on the fueling data display 123_2 of the second fuel dispenser 120_2, is stored.

[0382] <Flow of fueling control process by information processing device> Next, the flow of the fueling control process by the fueling system 110S' according to the first embodiment will be described. FIG. 26 is an example of a flowchart showing the flow of the fueling control process by the fueling system according to the first embodiment.

[0383] In step S2601, the information processing device 160 acquires the image data captured by the cameras 130_1 and 130_2.

[0384] In step S2602, the information processing device 160 acquires the signals from the first fueling sensor 921_1 and the second fueling sensor 921_2.

[0385] In step S2603, the information processing device 160 acquires the thermal sensor data measured by the thermal sensors 170_1 and 170_2.

[0386] In step S2604, the information processing apparatus 160 determines which of the first section to the third section it corresponds to.

[0387] In step S2605, the information processing apparatus 160 determines whether to determine the fueling permission or not. In step S2605, if it is determined that it does not correspond to any of the first section to the third section, it is determined not to determine the fueling permission (determined as NO in step S2605), and the process proceeds to step S2610.

[0388] On the other hand, in step 2605, if it is determined that it corresponds to any of the first section to the third section, it is determined to determine the fueling permission (determined as YES in step S2605), and the process proceeds to step S2606.

[0389] In step S2606, the information processing apparatus 160 performs image recognition processing on the image data using the first learned model or the second learned model.

[0390] In step S2607, the information processing apparatus 160 determines the fueling permission or not from the recognition result using the determination logic corresponding to the determined section.

[0391] In step S2608, the information processing apparatus 160 controls the fueling of the first fuel dispenser 120_1 and the second fuel dispenser 120_2 based on the determination result of the fueling permission or not. Note that when the determination result is non - fueling permission, the information processing apparatus 160 controls the prohibition or stop of fueling by the first fuel dispenser 120_1 or the second fuel dispenser 120_2 according to the cause information of the non - fueling permission.

[0392] In step S2609, the information processing apparatus 160 outputs the determination result of the fueling permission or not. Further, when the determination result is non - fueling permission, the information processing apparatus 160 outputs the cause information of the non - fueling permission and the like.

[0393] In step S2610, the information processing apparatus 160 determines whether to end the fuel supply control process. If it is determined in step S2610 to continue the fuel supply control process (if NO in step S2610), the process returns to step S2601. On the other hand, if it is determined in step S2610 to end the fuel supply control process (if YES in step S2610), the fuel supply control process ends.

[0394] <Summary> As is clear from the above description, the fuel supply system 110S' according to the first embodiment divides the period from when the fuel supply nozzle is removed from the nozzle holder until it is returned to the nozzle holder into three sections, and determines whether fuel supply is permitted using determination logic corresponding to each section. Thus, according to the first embodiment, it is possible to provide a fuel supply system capable of executing processing according to the fuel supply scene.

[0395] Also, the fuel supply system 110S' according to the first embodiment is configured to acquire cause information for non-permission of fuel supply and output a message corresponding to the cause information as voice when determining non-permission of fuel supply based on the recognition result for the recognition target. Thus, according to the first embodiment, it is possible to provide a fuel supply system capable of executing processing according to the fuel supply scene.

[0396] Also, the fuel supply system 110S' according to the first embodiment is configured to use a learned model corresponding to the fuel supply type when performing image recognition processing on the recognition target. Thus, according to the first embodiment, it is possible to provide a fuel supply system capable of executing processing according to the fuel supply scene.

[0397] [Second Embodiment] In the above-described first embodiment, the information processing apparatus 160 has been described as performing fueling control for the first fuel dispenser 120_1, the second fuel dispenser 120_2, and the like. However, the fueling control for the first fuel dispenser 120_1, the second fuel dispenser 120_2, and the like may be configured to be performed by a device separate from the information processing apparatus 160 (for example, a fueling controller such as a self-service console provided in a service station). Hereinafter, the second embodiment will be described centering on the differences from the first embodiment.

[0398] <Functional Configuration of Information Processing Apparatus> FIG. 27 is a diagram showing another example of the functional configuration of the information processing apparatus. The difference from the functional configuration of the information processing apparatus 160 shown in FIG. 10 is that, in the case of the information processing apparatus 160' shown in FIG. 27, the functions of the first fueling monitoring unit 1009_1' and the second fueling monitoring unit 1009_2' are different from the functions of the first fueling monitoring unit 1009_1 and the second fueling monitoring unit 1009_2. Further, the difference from the functional configuration of the information processing apparatus 160 shown in FIG. 10 is that, in the case of the information processing apparatus 160' shown in FIG. 27, the first fueling monitoring unit 1009_1' and the second fueling monitoring unit 1009_2' are communicably connected to the fueling controller 2710.

[0399] The fueling controller 2710 is a self-service console (SSC: Self Service Console), and controls the fueling of the first fuel dispenser 120_1 based on the determination result of whether fueling is permitted transmitted from the first fueling monitoring unit 1009_1'. Further, the fueling controller 2710 controls the fueling of the second fuel dispenser 120_2 based on the determination result of whether fueling is permitted transmitted from the second fueling monitoring unit 1009_2'.

[0400] <Details of First Fueling Monitoring Unit and Second Fueling Monitoring Unit> Next, the details of the first fuel supply monitoring unit 1009_1' and the second fuel supply monitoring unit 1009_2' will be described. FIG. 28A is a diagram showing another example of the functional configuration of the first fuel supply monitoring unit of the information processing apparatus. In the above first embodiment, the difference from the functional configuration of the first fuel supply monitoring unit 1009_1 of the information processing apparatus 160 described with reference to FIG. 15A is that the first fuel supply monitoring unit 1009_1' of the information processing apparatus 160' in FIG. 28A has a transmission unit 2801. Further, in the case of the first fuel supply monitoring unit 1009_1' of the information processing apparatus 160' in FIG. 28A, the transmission unit 2801 is communicably connected to the fuel supply controller 2710.

[0401] The transmission unit 2801 transmits the determination result of fuel supply permission or fuel supply non - permission to the fuel supply controller 2710. Thereby, the fuel supply controller 2710 controls the fuel supply for the first fuel supply type by controlling the fuel supply valve 124_1 or the fuel supply pump 125_1.

[0402] For example, when the fuel supply permission is transmitted from the transmission unit 2801, the fuel supply controller 2710 drives the fuel supply pump 125_1 to open the fuel supply valve 124_1. Also, the fuel supply controller 2710 · when the fuel supply permission from the poly - tank determination unit 1506 is transmitted from the transmission unit 2801, drives the fuel supply pump 125_1 to open the fuel supply valve 124_1. Further, · when the fuel supply non - permission from any of the first determination logic 1503 to the third determination logic 1505 is transmitted from the transmission unit 2801, or · when the fuel supply non - permission from the fire determination unit 1507 is transmitted from the transmission unit 2801, it stops the fuel supply by stopping the drive of the fuel supply pump 125_1, or stops the drive of the fuel supply pump 125_1 and closes the fuel supply valve 124_1 to prohibit the fuel supply. Note that the definitions of fuel supply stop and fuel supply prohibition here are only examples, and the fuel supply stop and fuel supply prohibition may be defined by other definitions. Alternatively, without distinguishing between fuel supply stop and fuel supply prohibition, the entire state where fuel supply cannot be performed may be referred to as fuel supply stop or prohibition.

[0403] Further, the transmission unit 2801 notifies the fueling permission determination result and the fueling non - permission determination result to the cause information acquisition unit 1509. When notifying fueling non - permission, the transmission unit 2801 also notifies the cause information acquisition unit 1509 of information indicating the cause determined as fueling non - permission.

[0404] Note that when any of the first determination logic 1503 to the third determination logic 1505 cannot perform the determination of fueling permission or non - permission, the transmission unit 2801 may be configured to output "determination impossible" to the cause information acquisition unit 1509. When "determination impossible" is output from the transmission unit 2801, the cause information acquisition unit 1509 may display "determination impossible" on, for example, the display device 912 which is a display device for the staff of the service station 110'. According to such a configuration, even when the determination of fueling permission or non - permission cannot be performed, the staff of the service station 110' can operate the fueling controller 2710 to control fueling.

[0405] Also, the transmission unit 2801 may be configured to monitor the communication status with the fueling controller 2710 and output "communication abnormality" to the cause information acquisition unit 1509 when a communication failure occurs. When "communication abnormality" is output from the transmission unit 2801, the cause information acquisition unit 1509 may display "communication abnormality" on, for example, the display device 912 which is a display device for the staff of the service station 110'. According to such a configuration, even when the determination result of fueling permission or non - permission cannot be transmitted to the fueling controller 2710, the staff of the service station 110' can operate the fueling controller 2710 to control fueling.

[0406] FIG. 28B is a diagram showing an example of the functional configuration of the second fueling monitoring unit of the information processing device. Since the functional configuration of the second fueling monitoring unit 1009_2' shown in FIG. 28B is the same as the functional configuration of the first fueling monitoring unit 1009_1' shown in FIG. 28A, detailed description is omitted here.

[0407] <Summary> As is clear from the above description, in the second embodiment, the functions of the information processing apparatus 160 in the first embodiment are realized by the information processing apparatus 160' and the fuel supply controller 2710. Thus, even when the information processing apparatus 160' and the fuel supply controller 2710 are configured separately, according to the second embodiment, the same effects as those in the first embodiment can be achieved.

[0408] [Third Embodiment] In the first embodiment, the case where the period from when the fuel supply nozzle is removed from the nozzle holder until it is returned to the nozzle holder is divided into three sections has been described. However, the number of sections to be divided is not limited to three, and may be four or more.

[0409] Also, in the first embodiment, it has been described that one determination logic is associated with each section. However, the relationship between the number of sections and the number of determination logics is not limited to this. For example, one determination logic may be associated with two consecutive sections.

[0410] Also, in the first embodiment, the case where there are two types of fuel supply has been described, but the number of types of fuel supply may be three or more. Note that when there are three or more types of fuel supply, three or more learned models are also generated.

[0411] Also, in the first embodiment, as recognition targets for the first fuel supply type, vehicles, containers, fuel suppliers, fuel supply nozzles, etc. have been used. However, the recognition targets for the first fuel supply type are not limited to these.

[0412] Similarly, in the first embodiment, as recognition targets for the second fuel supply type, containers, fuel suppliers, fuel supply nozzles, etc. have been used. However, the recognition targets for the second fuel supply type are not limited to these.

[0413] In the first embodiment described above, the first to third determination logics for the first fuel supply type were described with reference to FIGS. 16A to 18B. However, the processing contents of the first to third determination logics are not limited to these.

[0414] Similarly, in the first embodiment described above, the first to third determination logics for the second fuel supply type were described with reference to FIGS. 20A to 22B. However, the processing contents of the first to third determination logics are not limited to these.

[0415] Also, in the first embodiment described above, the processing after displaying the cause information, for example, on the display device 912 for the staff of the service station 110' was not mentioned. However, after displaying the cause information, it may be switched to manual operation by the staff.

[0416] Also, in the first embodiment described above, the number of executions of the learning process was not mentioned. However, the learning process may be executed multiple times. For example, in the image recognition process using the first learned model and the second learned model generated by the first learning process, if sufficient recognition accuracy cannot be obtained, or if the recognition accuracy decreases, additional learning processes may be configured to be performed.

[0417] For example, the learning device 150 may be configured to perform additional learning processes by collecting image data and signals indicating the states of the fuel nozzles again from a plurality of service stations and generating learning data.

[0418] Note that the learning device 150 may be configured to be able to collect image data and signals indicating the states of the fuel nozzles at any timing from a plurality of service stations. Also, a device that monitors the recognition accuracy, analyzes the cause when the recognition accuracy decreases, and notifies the learning device 150 may be connected to the network 180.

[0419] In addition, in the above-described first embodiment, the learning device 150 and the information processing device 160 are configured as separate bodies, but the learning device 150 and the information processing device 160 may be configured as an integral body.

[0420] Note that the configurations and the like described in the above embodiments are not limited to the configurations shown herein, such as combinations with other elements. Regarding these points, it is possible to make changes without departing from the spirit of the present invention, and it can be appropriately determined according to the application form.

Explanation of Reference Numerals

[0421] 110, 111: Service Station 110S': Fueling System 120_1: First Fueling Machine 120_2: Second Fueling Machine 121_1, 121_2: Fueling Nozzle 122_1, 122_2: Voice Output Device 123_1, 123_3: Fueling Data Display 124_1, 124_2: Fueling Valve 125_1, 125_2: Fueling Pump 130_1, 130_2: Camera 170_1, 170_2: Thermal Sensor 150: Learning Device 160: Information Processing Device 160': Information Processing Device 210: First Region 220: Second Region 211: Vehicle 221: Fueler 231: Container 241: Fueler 500A: Learning Data Generation Unit 500B: Learning Unit 600: First Learning Data 700: Second Learning Data 912: Display Device 921_1: First Fueling Sensor 921_2: Second Fueling Sensor 1001: First fuel type image data acquisition unit 1002: Second fuel type image data acquisition unit 1003: First fueling sensor data acquisition unit 1004: Second fueling sensor data acquisition unit 1005: Image recognition unit 1006, 1007: Interval determination unit 1009_1: First fueling monitoring unit 1009_1': First fueling monitoring unit 1009_2: Second fueling monitoring unit 1009_2': Second fueling monitoring unit 1010: Output control unit 1302: First trained model 1312: Second trained model 1503, 1513: First judgment logic 1504, 1514: Second judgment logic 1505, 1515: Third judgment logic 2710: Fueling controller 2801, 2811: Transmitter

Claims

1. A fuel tanker and A refueling system having an information processing device, The information processing device includes: determining a state of the refueling nozzle and a state of refueling based on the notification from the refueling aircraft; A refueling system that prohibits or stops refueling by a refueling machine when it is determined that refueling is not permitted based on a judgment logic according to whether the refueling action of a refueler falls in one of the following three sections: a first section before refueling begins, a second section during refueling, or a third section after refueling is completed, or based on a judgment logic according to which two consecutive sections the refueling action falls in.

2. A state determination unit that determines a state of the refueling nozzle and a state of refueling based on a notification from the refueling machine, When the state determination unit determines that the fuel nozzle has been removed from the nozzle hanger, it is determined that the first section has started; When it is determined that refueling is permitted based on a determination logic corresponding to the first section and the state determination unit determines that refueling has started, it is determined that the vehicle has transitioned to the second section; When the state determination unit determines that refueling has not been performed for a predetermined time, it is determined that the state has entered the third section, and when the state determination unit determines that the refueling nozzle has been returned to the nozzle hook, it is determined that the third section has ended. The fueling system of claim 1 .

3. a camera for photographing an area including the tanker aircraft; When the fuel type is gasoline or diesel, the determination logic is A recognition process is performed on the image data captured by the camera, A recognition result that a vehicle is recognized, A recognition result that a fuel dispenser holding a fuel nozzle in an appropriate posture is recognized, A recognition result that a fuel dispenser who is not holding the fuel dispenser nozzle in an appropriate position is recognized, The refueling system according to claim 1 , wherein, when the recognition result is output, it is determined whether or not refueling is permitted based on the recognition result.

4. The first determination logic according to the first section is If the permission conditions are met for a certain period of time, refueling is permitted. When a predetermined non-permission condition is satisfied for a certain period of time, it is determined that refueling is not permitted, and when a permission condition is satisfied for a certain period of time thereafter, it is determined that refueling is permitted; If other refusal conditions are met for a certain period of time, refueling is not permitted. The second determination logic according to the second section is If the disallowance condition is met for a certain period of time, a warning is output. If the disapproval conditions are met for a certain period of time after the warning is output, it is determined that refueling is not permitted. If the permission conditions are met for a certain period of time after the warning is output, it is determined that refueling is permitted. The third determination logic according to the third section is If the permission conditions are met for a certain period of time, refueling is permitted. When the non-permission condition is satisfied for a certain period of time, it is determined that refueling is not permitted, and when the permission condition is satisfied for a certain period of time thereafter, it is determined that refueling is permitted.

4. The fueling system of claim 3.

5. The decision logic further comprises: The refueling system of claim 3, further comprising a process for performing a recognition process on image data captured by the camera, and determining that refueling is not permitted if a recognition result indicating that a specified container has been recognized is output.

6. Further comprising a thermal sensor; The decision logic further comprises:

2. The refueling system according to claim 1, further comprising: a determination as to whether or not the temperature measured by the thermal sensor is equal to or higher than a predetermined threshold; and if it is determined that the temperature is equal to or higher than the predetermined threshold, it is determined that refueling is not permitted.

7. A recognition result indicating that a vehicle has been recognized in a first area that is a part of the image data captured by the camera is output; A recognition result indicating that a fuel dispenser holding a fuel dispenser nozzle in an appropriate posture has been recognized in a second area, which is a part of the image data captured by the camera and is different from the first area, is output.

4. The fueling system of claim 3.

8. Further comprising a thermal sensor; The determination logic includes: The fuel supply system according to claim 7 , further comprising: determining whether or not the temperature measured by the thermal sensor is equal to or higher than a predetermined threshold in an area not including the first area.

9. When the fuel type is kerosene, the judgment logic is as follows: A recognition process is performed on the image data captured by the camera, A recognition result that a specific container has been recognized, A recognition result that a fuel dispenser holding a fuel nozzle in an appropriate posture is recognized, A recognition result that a fuel dispenser who is not holding the fuel dispenser nozzle in an appropriate position is recognized, The refueling system according to claim 3 , wherein, when the recognition result is output, it is determined whether or not refueling is permitted based on the recognition result.

10. The first determination logic according to the first section is If the permission conditions are met for a certain period of time, refueling is permitted. When a predetermined non-permission condition is satisfied for a certain period of time, it is determined that refueling is not permitted, and when a permission condition is satisfied for a certain period of time thereafter, it is determined that refueling is permitted; If other refusal conditions are met for a certain period of time, refueling is not permitted. The second determination logic according to the second section is If the disallowance condition is met for a certain period of time, a warning is output. If the disapproval conditions are met for a certain period of time after the warning is output, it is determined that refueling is not permitted. If the permission conditions are met for a certain period of time after the warning is output, it is determined that refueling is permitted. The third determination logic according to the third section is If the permission conditions are met for a certain period of time, refueling is permitted. When the non-permission condition is satisfied for a certain period of time, it is determined that refueling is not permitted, and when the permission condition is satisfied for a certain period of time thereafter, it is determined that refueling is permitted.

10. The fueling system of claim 9.

11. Further comprising a thermal sensor; The decision logic further comprises:

10. The refueling system according to claim 9, further comprising: a determination as to whether or not the temperature measured by the thermal sensor is equal to or higher than a predetermined threshold; and if it is determined that the temperature is equal to or higher than the predetermined threshold, it is determined that refueling is not permitted.

12. Prohibiting or stopping refueling by stopping the refueling pump or closing the refueling valve installed in the refueling pipe, The fueling system of claim 1 .

13. A refueling controller communicating with the information processing device, The refueling controller includes: The refueling system according to claim 1 , wherein, when it is determined based on the determination logic that refueling is not permitted, refueling by the refueling aircraft is prohibited or stopped.

14. Based on the notification from the refueling aircraft, the state of the refueling nozzle and the state of refueling are determined; When it is determined that refueling is not permitted based on a determination logic according to whether the refueling action of the refueler is included in a first section before the start of refueling, a second section during refueling, or a third section after the end of refueling, or based on a determination logic according to whether the refueling action is included in two consecutive sections, refueling by the refueling aircraft is prohibited or stopped; Information processing device.

15. Based on the notification from the refueling aircraft, the state of the refueling nozzle and the state of refueling are determined; When it is determined that refueling is not permitted based on a determination logic according to whether the refueling action of the refueler is included in a first section before the start of refueling, a second section during refueling, or a third section after the end of refueling, or based on a determination logic according to whether the refueling action is included in two consecutive sections, refueling by the refueling aircraft is prohibited or stopped; A fuel supply control method in which processing is executed by a computer.

16. Based on the notification from the refueling aircraft, the state of the refueling nozzle and the state of refueling are determined; When it is determined that refueling is not permitted based on a determination logic according to whether the refueling action of the refueler is included in a first section before the start of refueling, a second section during refueling, or a third section after the end of refueling, or based on a determination logic according to whether the refueling action is included in two consecutive sections, refueling by the refueling aircraft is prohibited or stopped; A fuel supply control program for causing a computer to execute processing.

Images