Fuel Monitoring System

Abstract

To provide a fueling monitoring system capable of reducing an error in determining whether or not self-fueling can be permitted, taking into account situations of a vehicle targeted for fueling and a fuel supplying person.SOLUTION: A monitoring device 110 determines whether or not there is a hindrance in using a learning model for a four-wheel car based on a camera video image. When there is no hindrance, the device uses the learning model for the four-wheel car and executes AI fueling determination processing. When there is a hindrance, the device makes a terminal device 120 display an indication that a visual determination is necessary on whether the self-fueling can be permitted, thus prompting an operator of a fueling station to carry out the visual determination.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a fuel supply monitoring system that is applied to a gas station where self-service fuel supply is possible. [Background technology]

[0002] At conventional gas stations, a worker (supervisor) inside the gas station building would visually or via a surveillance monitor monitor the behavior of the customer (fuel dispenser) performing self-service refueling, and would then operate a fuel dispensing management device to issue a fuel dispenser permission. In recent years, there has been progress in the development of systems that use AI (Artificial Intelligence) to analyze images from surveillance cameras, and these systems are beginning to be applied to the monitoring of self-service refueling.

[0003] Here, the following are examples of prior art in the technical field related to the present invention: For example, Patent Document 1 discloses an invention in which, based on an image of the fuel dispenser, the behavior of the fuel dispenser is judged as abnormal, normal, or unknown, and if the behavior at each stage of the fuel dispenser process is judged as abnormal or unknown, fuel dispenser is stopped or prohibited, and an image serving as the basis for the judgment is displayed on a management terminal. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-091460 Summary of the Invention [Problem to be solved by the invention]

[0005] Systems are being developed that perform AI refueling judgment processing, which uses AI based on a predetermined learning model to analyze camera footage and determine whether self-service refueling is permitted. However, depending on the type of vehicle, the accuracy of the AI ​​analysis may be reduced, making it impossible to perform the AI ​​refueling judgment processing properly. For example, when refueling a motorcycle, the fuel tank opening is more likely to be obscured by the person refueling (the camera cannot capture the fuel tank opening), compared to refueling a car, resulting in reduced accuracy of the AI ​​analysis. Furthermore, if the person refueling is leaning forward, for example, the fuel tank opening will be obscured by the person refueling, making it difficult to perform the AI ​​analysis. Performing the AI ​​refueling judgment processing in situations where AI analysis of camera footage is difficult increases the likelihood of an incorrect judgment result.

[0006] The present invention has been made in consideration of the above-mentioned conventional circumstances, and aims to provide a refueling monitoring system that can reduce errors in determining whether or not self-refueling is permitted, taking into account the situation of the vehicle being refueled and the person refueling. [Means for solving the problem]

[0007] In order to achieve the above object, a refueling monitoring system according to one aspect of the present invention is configured as follows: That is, the refueling monitoring system according to the present invention includes a camera installed at a gas station, and a monitoring device that executes a refueling determination process that determines whether refueling of a vehicle captured by the camera is permitted by analyzing the camera image using a pre-prepared learning model, and the monitoring device determines based on the camera image whether there is a problem with using the learning model, and if there is a problem with using the learning model, executes the refueling determination process using the learning model, and if there is a problem with using the learning model, controls so that the determination of whether refueling of the vehicle is permitted is performed by a different method.

[0008] Here, in the refueling monitoring system of the present invention, the monitoring device can be configured to determine that there is a problem with using the learning model when an image is captured in which the body of a refueler overlaps the fuel filler opening of a vehicle.

[0009] In addition, in the refueling monitoring system of the present invention, the monitoring device can be configured to determine that there is a problem with using the learning model if the vehicle's fuel filler opening is outside the field of view of the camera image.

[0010] In addition, in the refueling monitoring system of the present invention, the monitoring device can be configured to determine that there is a problem with using the learning model when video of the refueler in a specific posture is captured.

[0011] In addition, in the refueling monitoring system of the present invention, the learning model is a first learning model for vehicles whose vehicle type corresponds to a first type, and the monitoring device can be configured to determine the type of vehicle contained in the camera image and, if the determined vehicle type is not the first type, to determine that there is a problem with using the learning model.

[0012] In addition, in the refueling monitoring system of the present invention, if the monitoring device determines that there is a problem with the use of the learning model, it can be configured to notify a gas station employee to visually determine whether or not refueling of the vehicle is permitted.

[0013] In addition, in the refueling monitoring system of the present invention, the learning model further includes a second learning model for vehicles that fall into a second type different from the first type, and the monitoring device can be configured to perform a refueling judgment process using the second learning model when the type of vehicle identified is not the first type and therefore there is a problem in using the first learning model, but when the type of vehicle identified is the second type.

[0014] A refueling monitoring system according to another aspect of the present invention is configured as follows: That is, the refueling monitoring system according to the present invention includes a camera installed at a gas station, and a monitoring device that executes a refueling determination process that determines whether refueling of a vehicle captured by the camera is permitted by analyzing the camera image using a pre-prepared learning model, the learning models including a plurality of learning models each corresponding to a vehicle type, and the monitoring device determines the type of vehicle included in the camera image and executes the refueling determination process using the learning model corresponding to the determined vehicle type.

[0015] A refueling monitoring system according to yet another aspect of the present invention is configured as follows: That is, the refueling monitoring system according to the present invention includes a camera installed at a gas station, and a monitoring device that executes a refueling determination process that determines whether refueling of a vehicle captured by the camera is permitted by analyzing the camera's video using a pre-prepared learning model, wherein the learning models include a first learning model for vehicles that fall into a first type and a second learning model for vehicles that fall into a second type different from the first type, and the monitoring device executes the refueling determination process using the first learning model, and if the refueling determination process fails or if the refueling determination process determines that refueling is not permitted, re-executes the refueling determination process using the second learning model.

[0016] In addition, in the refueling monitoring system of the present invention, if it is determined as a result of the refueling determination process that the vehicle is not in a state in which refueling is permitted, the monitoring device can be configured to notify a gas station worker to visually determine whether the vehicle is in a state in which refueling is permitted. [Effects of the Invention]

[0017] According to the present invention, a refueling monitoring system can be provided that can reduce errors in determining whether or not self-refueling is permitted, taking into account the conditions of the vehicle being refueled and the person refueling. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a diagram illustrating an example of the configuration of a refueling monitoring system according to an embodiment of the present invention. [Figure 2] FIG. 3 is a diagram showing a refueling determination flow according to the first embodiment. [Figure 3] FIG. 10 is a diagram showing a refueling determination flow according to a second embodiment. [Figure 4] FIG. 10 is a diagram showing a refueling determination flow according to a third embodiment. [Figure 5] FIG. 10 is a diagram showing a refueling determination flow according to a fourth embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of a display on a terminal device. [Figure 7] FIG. 10 is a diagram showing another example of display on the terminal device. DETAILED DESCRIPTION OF THE INVENTION

[0019] An embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows an example of the configuration of a refueling monitoring system 100 according to an embodiment of the present invention. The refueling monitoring system 100 of this example is a system that has the function of executing an AI refueling determination process that analyzes camera footage using AI and determines whether or not self-refueling is permitted. The refueling monitoring system 100 includes a monitoring device 110, a terminal device 120, a refueling management device 130, a weighing machine 135, a monitoring camera 140, and a sensor 150.

[0020] The surveillance camera 140 is installed at any position within the gas station, captures images of the self-service refueling, and outputs the obtained image data (camera video) to the monitoring device 110. The surveillance camera 140 is installed for each vehicle stopping area (i.e., for each fueling lane) so that it can capture images of vehicles and fuel dispensers (users) in the vehicle stopping area adjacent to the weighing machine 135. The surveillance camera 140 is installed, for example, in a position and orientation that overlooks the area including the area between the weighing machine 135 and the vehicles from above.

[0021] The sensor 150 detects sound, heat, odor, light, oil leakage, or the like generated by the actions of the fuel dispenser, and outputs the detection results to the monitoring device 110. The sensor 150 also includes a sensor that detects when the fuel nozzle is removed from the metering machine 135. The sensor 150 is installed on the metering machine 135 or in its vicinity (for example, around the vehicle parking area).

[0022] The monitoring device 110 is installed in a location away from the metering machine 135 (for example, inside a gas station building). The monitoring device 110 includes an image analysis device 111, a control device 112, a wireless device 113, and a learning model storage unit 114. The learning model storage unit 114 stores a learning model prepared in advance for AI analysis of camera footage. The image analysis device 111 analyzes the camera footage using the learning model in the learning model storage unit 114, and executes an AI refueling determination process to determine whether refueling of a vehicle photographed by the monitoring camera 140 is permitted. The AI ​​refueling determination process also takes into account the detection results of the sensor 150. The wireless device 113 wirelessly communicates with the terminal device 120. The control device 112 is interposed between the monitoring camera 140, the sensor 150, the fuel supply management device 130, the image analysis device 111, and the wireless device 113, and comprehensively controls the operations of these devices / equipment. The monitoring device 110 is, for example, a computer equipped with hardware resources such as a processor and memory, and is configured so that the processor executes a program for realizing each function according to the present invention.

[0023] The AI ​​refueling determination process checks whether there are any problems with multiple check items based on camera footage captured by surveillance camera 140 and detection results from sensor 150. For example, it checks whether the fuel nozzle of metering device 135 is properly inserted (fully inserted) into the vehicle's fuel filler opening, whether there is no heat source such as a cigarette, whether there is a portable can, and whether multiple people are attempting to refuel. If it is confirmed from the camera footage that there are no problems with any of these check items, it determines that refueling is permitted (i.e., a refueling permission-enabled state); if not, it determines that refueling is not permitted (i.e., a refueling permission-prohibited state).

[0024] The terminal device 120 is a device used by a gas station worker (monitor). The terminal device 120 has a control unit 121, a communication unit 122, a display unit 123, and an operation unit 124. The control unit 121 comprehensively controls the operation of each unit of the terminal device 120. The communication unit 122 performs wireless communication with the monitoring device 110. The display unit 123 displays various information including the results of the AI ​​refueling determination process performed by the monitoring device 110. The operation unit 124 accepts various operations including permission or suspension of self-service refueling. The display unit 123 and the operation unit 124 may be integrated into a touch panel. The terminal device 120 is, for example, a computer equipped with hardware resources such as a processor and memory, and is configured so that the processor executes programs for realizing each function according to the present invention.

[0025] The terminal device 120 receives and displays the results of the AI ​​refueling determination process from the monitoring device 110. The terminal device 120 also displays the camera footage along with the results of the AI ​​refueling determination process. The worker considers whether to permit or prohibit (not permit) refueling after referring to the information displayed on the terminal device 120, and if the worker determines that refueling should be permitted, performs an operation on the terminal device 120 accordingly. Furthermore, if the worker determines that refueling should be stopped after permitting refueling, the worker performs an operation on the terminal device 120 accordingly.

[0026] When the terminal device 120 receives the above-described operation, it transmits a control signal indicating that operation (for example, a refueling permission signal or a refueling stop signal) to the refueling management device 130. The control signal is accompanied by information identifying the target metering device 135 (for example, a metering device ID). The control signal corresponding to the operation of the terminal device 120 may be transmitted to the refueling management device 130 via the monitoring device 110, or may be transmitted directly to the refueling management device 130 without going through the monitoring device 110. Here, an example will be described in which the terminal device 120 is a portable terminal such as a tablet that can be carried by an operator, but it may also be a stationary terminal that is installed (fixed) in a predetermined location.

[0027] The fuel supply management device 130 is also referred to as a self-service controller (SSC), and is installed, for example, in a location remote from the metering device 135 (for example, inside a gas station building). The fuel supply management device 130 controls the operation of the metering device 135 based on a control signal transmitted in response to an operation of the terminal device 120. For example, when the terminal device 120 receives an operation to permit refueling, a fuel supply permission signal is transmitted from the terminal device 120 to the fuel supply management device 130. Upon receiving the fuel supply permission signal, the fuel supply management device 130 drives a pump inside the metering device 135 corresponding to the fuel supply permission signal, thereby enabling the metering device 135 to perform refueling. Furthermore, for example, when the terminal device 120 receives an operation to stop refueling, a fuel supply stop signal is transmitted from the terminal device 120 to the metering device 135. The fuel supply management device 130 that has received the fuel supply stop signal stops the pump inside the metering device 135 that corresponds to the fuel supply stop signal, and as a result, the metering device 135 is in a state where it cannot supply fuel.

[0028] The main feature of the refueling monitoring system 100 of this example is that it is equipped with a mechanism that reduces errors in determining whether or not self-refueling is permitted, taking into consideration the conditions of the vehicle being refueled and the person refueling. This mechanism will be explained below using several examples.

[0029] [First Example] 2 shows a refueling determination flow according to the first embodiment. In the first embodiment, a learning model for four-wheeled vehicles is stored in the learning model storage unit 114 as the learning model used in the AI ​​refueling determination process. The learning model storage unit 114 may also store another learning model for the vehicle type discrimination process.

[0030] First, suppose that a user (fueler) intending to perform self-service refueling gets into a vehicle, arrives at a gas station, stops the vehicle in a vehicle stopping area adjacent to the weighing machine 135, gets out of the vehicle, and begins to take action to refuel. The surveillance camera 140 captures an image of the vehicle and the fueler to be refueled, and transmits the camera image to the monitoring device 110.

[0031] In the monitoring device 110, the control device 112 provides the camera image received from the monitoring camera 140 to the image analysis device 111. Based on the provided camera image, the image analysis device 111 executes a vehicle type determination process to determine the type of vehicle photographed by the monitoring camera 140, i.e., the type of vehicle to be refueled (step S101). The vehicle type determination is performed at any time while the vehicle is captured in the camera image, and the determination result is confirmed, for example, when a signal indicating that the fuel nozzle has been removed from the metering machine 135 is received from the sensor 150.

[0032] Here, the vehicle type discrimination process determines whether the vehicle type is a "four-wheeled vehicle" or a "two-wheeled vehicle." The vehicle type discrimination process may be performed by image analysis based on multiple discrimination learning models prepared for each vehicle type, or may be performed by image analysis based on a single discrimination learning model that is independent of vehicle type. Alternatively, the vehicle type may be discriminated using other methods that do not use a learning model.

[0033] If the vehicle type is determined to be a "four-wheeled vehicle" as a result of the vehicle type determination process (step S102; Yes), the image analysis device 111 executes an AI refueling determination process using the learning model for four-wheeled vehicles in the learning model storage unit 114 (step S103). If it is determined that refueling is permitted (step S104; Yes), the control device 112 transmits a notification signal indicating that refueling is permitted to the terminal device 120 via the wireless device 113 (step S105).

[0034] Upon receiving this notification signal, the terminal device 120 displays on the display unit 123 the fact that refueling is permitted. FIG. 6 shows a display example 200 by the terminal device 120. The display example 200 in the figure has a display area 211 that displays a camera image of the target refueling lane, a display area 212 that displays the fact that refueling is permitted, and a permission button 214 that is operated to permit refueling. When the terminal device 120 receives an operation to permit refueling (i.e., pressing the permission button 214) (step S107), it transmits a refueling permission signal to the refueling management device 130 (step S108). As a result, the metering machine 135 becomes capable of refueling under the control of the refueling management device 130.

[0035] On the other hand, if the vehicle type determination process determines that the vehicle type is not a "four-wheeled vehicle" (step S102; No), that is, if the vehicle type is a "two-wheeled vehicle," the use of the learning model for four-wheeled vehicles is hindered, so the AI ​​refueling determination process using the learning model for four-wheeled vehicles is not executed, and a notification signal indicating that a visual determination is required is transmitted to the terminal device 120 via the wireless device 113 (step S106). Also, if the AI ​​refueling determination process (step S103) determines that the refueling permission is prohibited (step S104; No), a notification signal indicating that a visual determination is required is transmitted to the terminal device 120 via the wireless device 113 (step S106).

[0036] Upon receiving these notification signals, the terminal device 120 displays on the display unit 123 a message that a visual inspection is required. FIG. 7 shows a display example 200′ by the terminal device 120. The display example 200′ in the figure has a display area 211 that displays a camera image of the target fueling lane, a display area 213 that displays a message that a visual inspection is required, and a permission button 214 that is operated to permit fueling. The display area 213 may display not only the message that a visual inspection is required but also the reason why a visual inspection is required. To achieve this, the reason why a visual inspection is required may be transmitted by including it in the notification signal.

[0037] When the worker sees the display indicating that a visual judgment is required, he visually checks the vehicle to be refueled and the person refueling, and checks whether or not refueling is permitted. If it is determined that refueling is permitted, the worker operates the terminal device 120 to permit refueling (i.e., presses the permission button 214). When the terminal device 120 receives the operation to permit refueling (step S107), it transmits a refueling permission signal to the refueling management device 130 (step S108). As a result, the metering machine 135 becomes capable of refueling under the control of the refueling management device 130.

[0038] As described above, the monitoring device 110 of the first embodiment is configured to determine whether there is a problem with using the learning model for four-wheeled vehicles based on camera footage, and if there is no problem, to execute the AI ​​refueling determination process using the learning model for four-wheeled vehicles, and if there is a problem, to display on the terminal device 120 that a visual determination is required as to whether the condition is such that self-refueling is permitted, thereby prompting the gas station employee to make a visual determination. In this way, if there is a problem with using the learning model for four-wheeled vehicles, by using a method other than the AI ​​refueling determination process using the learning model for four-wheeled vehicles, it is possible to reduce errors in determining whether the condition is such that self-refueling is permitted.

[0039] [Second Example] 3 shows a refueling determination flow according to the second embodiment. In the second embodiment, a learning model for four-wheeled vehicles and a learning model for two-wheeled vehicles are stored in the learning model storage unit 114 as learning models used in the AI ​​refueling determination process. The learning model storage unit 114 may also store another learning model for the vehicle type discrimination process.

[0040] First, suppose that a user (fueler) intending to perform self-service refueling gets into a vehicle, arrives at a gas station, stops the vehicle in a vehicle stopping area adjacent to the weighing machine 135, gets out of the vehicle, and begins to take action to refuel. The surveillance camera 140 captures an image of the vehicle and the fueler to be refueled, and transmits the camera image to the monitoring device 110.

[0041] In the monitoring device 110, the control device 112 provides the camera image received from the monitoring camera 140 to the image analysis device 111. Based on the provided camera image, the image analysis device 111 executes a vehicle type determination process to determine the type of vehicle photographed by the monitoring camera 140, i.e., the type of vehicle to be refueled (step S201). The vehicle type determination is performed at any time while the vehicle is captured in the camera image, and the determination result is confirmed, for example, when a signal indicating that the fuel nozzle has been removed from the metering machine 135 is received from the sensor 150.

[0042] Here, the vehicle type discrimination process determines whether the vehicle type is a "four-wheeled vehicle" or a "two-wheeled vehicle." The vehicle type discrimination process may be performed by image analysis based on multiple discrimination learning models prepared for each vehicle type, or may be performed by image analysis based on a single discrimination learning model that is independent of vehicle type. Alternatively, the vehicle type may be discriminated using other methods that do not use a learning model.

[0043] If the vehicle type is determined to be a "four-wheeled vehicle" as a result of the vehicle type determination process (step S202; Yes), the image analysis device 111 executes an AI refueling determination process using the learning model for four-wheeled vehicles stored in the learning model storage unit 114 (step S203). On the other hand, if the vehicle type is determined to be not a "four-wheeled vehicle" (step S202; No), that is, if the vehicle type is a "two-wheeled vehicle," the image analysis device 111 executes an AI refueling determination process using the learning model for two-wheeled vehicles (step S204), rather than the AI ​​refueling determination process using the learning model for four-wheeled vehicles. Then, if the result of these AI refueling determination processes determines that refueling is permitted (step S205; Yes), the control device 112 transmits a notification signal indicating that refueling is permitted to the terminal device 120 via the wireless device 113 (step S206).

[0044] Upon receiving this notification signal, the terminal device 120 displays on the display unit 123 that refueling is permitted (see, for example, FIG. 6). After that, when the terminal device 120 receives an operation to permit refueling (step S208), it transmits a refueling permission signal to the refueling management device 130 (step S209). As a result, under the control of the refueling management device 130, the metering machine 135 becomes capable of refueling.

[0045] On the other hand, if the result of the AI ​​refueling judgment process using a learning model for four-wheeled vehicles (step S203) or the AI ​​refueling judgment process using a learning model for two-wheeled vehicles (step S204) determines that refueling is prohibited (step S205; No), a notification signal indicating that a visual judgment is required is sent to the terminal device 120 via the wireless device 113 (step S207).

[0046] Upon receiving this notification signal, the terminal device 120 displays on the display unit 123 that a visual determination is required (see, for example, FIG. 7). The worker who sees this display visually checks the vehicle to be refueled and the person refueling, and checks whether or not refueling is permitted. If it is determined that refueling is permitted, the worker operates the terminal device 120 to permit refueling. When the terminal device 120 receives the operation to permit refueling (step S208), it transmits a refueling permission signal to the refueling management device 130 (step S209). As a result, the metering machine 135 becomes capable of refueling under the control of the refueling management device 130.

[0047] As described above, the monitoring device 110 of the second embodiment is configured to determine whether there is a problem with using the learning model for four-wheeled vehicles based on camera footage, and if there is no problem, to execute the AI ​​refueling judgment process using the learning model for four-wheeled vehicles, and if there is a problem, to execute the AI ​​refueling judgment process using the learning model for two-wheeled vehicles. In this way, if there is a problem with using the learning model for four-wheeled vehicles, by using a method other than the AI ​​refueling judgment process using the learning model for four-wheeled vehicles, it is possible to reduce errors in determining whether self-refueling is permitted.

[0048] [Third Example] 4 shows a refueling judgment flow according to Example 3. In Example 4, the learning model storage unit 114 stores a learning model for four-wheeled vehicles and a learning model for two-wheeled vehicles as learning models used in the AI ​​refueling judgment process.

[0049] First, suppose that a user (fueler) intending to perform self-service refueling gets into a vehicle, arrives at a gas station, stops the vehicle in a vehicle stopping area adjacent to the weighing machine 135, gets out of the vehicle, and begins to take action to refuel. The surveillance camera 140 captures an image of the vehicle and the fueler to be refueled, and transmits the camera image to the monitoring device 110.

[0050] In the monitoring device 110, the control device 112 provides the camera video received from the monitoring camera 140 to the image analysis device 111. The image analysis device 111 executes the AI ​​refueling determination process using the learning model for four-wheeled vehicles stored in the learning model storage unit 114 based on the provided camera video (step S301). At this time, if the vehicle type is a four-wheeled vehicle, the AI ​​refueling determination process can be executed without any problems. However, if the vehicle type is a two-wheeled vehicle, the AI ​​refueling determination process fails. Alternatively, a refueling permission prohibition state is determined. A case where the AI ​​refueling determination process fails refers to a case where the AI ​​refueling determination is not executed or the accuracy of the AI ​​refueling determination is low. Therefore, if the AI ​​refueling determination process fails (including a case where a refueling permission prohibition state is determined) (step S303; No), the AI ​​refueling determination process is re-executed using the learning model for two-wheeled vehicles stored in the learning model storage unit 114 (step S303). Then, if either of these AI refueling judgment processes determines that refueling is permitted (step S302; Yes, or step S304; Yes), the control device 112 transmits a notification signal indicating that refueling is permitted to the terminal device 120 via the wireless device 113 (step S305).

[0051] Upon receiving this notification signal, the terminal device 120 displays on the display unit 123 that refueling is permitted (see, for example, FIG. 6). After that, when the terminal device 120 receives an operation to permit refueling (step S307), it transmits a refueling permission signal to the refueling management device 130 (step S308). As a result, under the control of the refueling management device 130, the metering machine 135 becomes capable of refueling.

[0052] On the other hand, if both of these AI refueling determination processes determine that the refueling permission is prohibited (step S304; No), a notification signal indicating that a visual determination is required is transmitted to the terminal device 120 via the wireless device 113 (step S306). The terminal device 120, upon receiving this notification signal, displays on the display unit 123 that a visual determination is required (see, for example, FIG. 7). The worker who sees this display visually confirms the vehicle to be refueled and the person refueling, and checks whether the state is such that refueling can be permitted. If it is determined that the state is such that refueling can be permitted, the worker operates the terminal device 120 to permit refueling. When the terminal device 120 receives the operation to permit refueling (step S307), it transmits a refueling permission signal to the refueling management device 130 (step S308). As a result, the metering machine 135 becomes capable of refueling under the control of the refueling management device 130.

[0053] As described above, the monitoring device 110 of the third embodiment is configured to execute the AI ​​refueling judgment process using the learning model for four-wheeled vehicles, and if the AI ​​refueling judgment process fails, execute the AI ​​refueling judgment process again using the learning model for two-wheeled vehicles. In this way, if there is a problem with using the learning model for four-wheeled vehicles, by using another method instead of the AI ​​refueling judgment process using the learning model for four-wheeled vehicles, it is possible to reduce errors in determining whether or not self-refueling is permitted.

[0054] [Fourth Example] Figure 5 shows a refueling judgment flow according to the fourth embodiment. In the fourth embodiment, the learning model storage unit 114 stores a plurality of learning models corresponding to different vehicle types as learning models used in the AI ​​refueling judgment process. Here, five types of learning models are prepared: "four-wheeled vehicle (general)," "four-wheeled vehicle (truck)," "four-wheeled vehicle (minivan)," "two-wheeled vehicle (general)," and "two-wheeled vehicle (covered)." The learning model storage unit 114 may also store another learning model for the vehicle type discrimination process.

[0055] First, suppose that a user (fueler) intending to perform self-service refueling gets into a vehicle, arrives at a gas station, stops the vehicle in a vehicle stopping area adjacent to the weighing machine 135, gets out of the vehicle, and begins to take action to refuel. The surveillance camera 140 captures an image of the vehicle and the fueler to be refueled, and transmits the camera image to the monitoring device 110.

[0056] In the monitoring device 110, the control device 112 provides the camera image received from the monitoring camera 140 to the image analysis device 111. Based on the provided camera image, the image analysis device 111 executes a vehicle type determination process to determine the type of vehicle photographed by the monitoring camera 140, i.e., the type of vehicle to be refueled (step S401). The vehicle type determination is performed at any time while the vehicle is captured in the camera image, and the determination result is confirmed, for example, when a signal indicating that the fuel nozzle has been removed from the metering machine 135 is received from the sensor 150.

[0057] Here, the vehicle type discrimination process determines whether the vehicle type is a "four-wheeled vehicle (general)," "four-wheeled vehicle (truck)," "four-wheeled vehicle (minivan)," "two-wheeled vehicle (general)," or "two-wheeled vehicle (covered)." The vehicle type discrimination process may be performed by image analysis based on multiple discrimination learning models prepared for each vehicle type, or may be performed by image analysis based on a single discrimination learning model that is independent of vehicle type. Vehicle type may also be discriminated by other methods that do not use a learning model. Furthermore, in the case of a supply object other than a vehicle, such as a portable can or a polyethylene container, image analysis based on multiple discrimination learning models prepared may be performed to determine whether refueling is necessary.

[0058] After executing the vehicle type discrimination process, the image analysis device 111 analyzes whether the AI ​​refueling determination process can be executed (step S402). That is, it analyzes whether camera footage suitable for the AI ​​refueling determination process has been obtained. In this example, if any of the following (Requirement 1) to (Requirement 3) is met, it is determined that there is an impediment to the execution of the AI ​​refueling determination process, and if none of the following is met, it is determined that there is no impediment to the execution of the AI ​​refueling determination process. Note that (Requirement 1) to (Requirement 3) are merely examples, and the analysis of whether the AI ​​refueling determination process can be executed may be performed based on other criteria.

[0059] (Requirement 1) is the case where a video is captured in which the head of the person delivering fuel is superimposed on the fuel tank cap of the vehicle. If this requirement is met, the state of the fuel tank cap cannot be confirmed in the camera video, and therefore the device is unsuitable for AI refueling judgment processing. This requirement is intended for cases in which the vehicle type is a "two-wheeled vehicle," i.e., a "two-wheeled vehicle (general)" or a "two-wheeled vehicle (covered)," but it may also be applied to "four-wheeled vehicles." Furthermore, even if a video is captured in which not only the head of the person delivering fuel but also their arms or hands are superimposed on the fuel tank cap of the vehicle, the state of the fuel tank cap cannot be confirmed in the camera video. Therefore, if a video is captured in which the body (head, arms, hands, etc.) of the person delivering fuel is superimposed on the fuel tank cap of the vehicle, the device may not be suitable for AI refueling judgment processing.

[0060] (Requirement 2) is the case where the fuel filler opening of the vehicle is outside the angle of view of the camera image. Examples of cases where the fuel filler opening of the vehicle is outside the angle of view of the camera image include when the fuel filler opening of the vehicle is located on the opposite side of the metering machine 135, or when the vehicle is not properly parked within the vehicle parking area. Even when this requirement is met, the state of the fuel filler opening cannot be confirmed from the camera image, so the system is not suitable for AI refueling judgment processing. This requirement is intended for cases where the type of vehicle is a "four-wheeled vehicle," that is, a "four-wheeled vehicle (general)," a "four-wheeled vehicle (truck)," or a "four-wheeled vehicle (van)," but it may also be applied to "two-wheeled vehicles."

[0061] (Requirement 3) is the case when the image captures a video of the fuel dispenser in a specific posture (for example, leaning forward). If this requirement is met, it is highly likely that the situation around the fuel dispenser cannot be confirmed from the camera image, and therefore the system is not suitable for AI refueling judgment processing. This requirement assumes that the vehicle type is a "four-wheeled vehicle," but it may also be applied to a "two-wheeled vehicle." The posture of the fuel dispenser can be determined, for example, by image analysis based on a learning model for posture discrimination. It is also acceptable to determine the posture of the fuel dispenser using other methods that do not use a learning model.

[0062] If, as a result of the above analysis, it is determined that there is no problem in executing the AI ​​refueling determination process using the learning model (step S403; Yes), the image analysis device 111 executes the AI ​​refueling determination process using the learning model in the learning model storage unit 114 that corresponds to the identified vehicle type (step S404).If, as a result, it is determined that refueling is permitted (step S405; Yes), the control device 112 transmits a notification signal indicating that refueling is permitted to the terminal device 120 via the wireless device 113 (step S406).

[0063] Upon receiving this notification signal, the terminal device 120 displays on the display unit 123 that refueling is permitted (see, for example, FIG. 6). After that, when the terminal device 120 receives an operation to permit refueling (step S408), it transmits a refueling permission signal to the refueling management device 130 (step S409). As a result, under the control of the refueling management device 130, the metering machine 135 becomes capable of refueling.

[0064] On the other hand, if the result of the above analysis determines that there is a problem with executing the AI ​​refueling judgment process using the learning model (step S403; No), the image analysis device 111 does not perform the AI ​​refueling judgment process using the learning model, and transmits a notification signal indicating that a visual judgment is required to the terminal device 120 via the wireless device 113 (step S407). Also, if the result of the AI ​​refueling judgment process (step S404) determines that a refueling permission prohibition state exists (step S405; No), a notification signal indicating that a visual judgment is required is transmitted to the terminal device 120 via the wireless device 113 (step S407).

[0065] Upon receiving these notification signals, the terminal device 120 displays on the display unit 123 that a visual judgment is required (see, for example, FIG. 7). The worker who sees this display visually checks the vehicle to be refueled and the person refueling, and checks whether or not refueling is permitted. If it is determined that refueling is permitted, the worker operates the terminal device 120 to permit refueling. When the terminal device 120 receives the operation to permit refueling (step S408), it transmits a refueling permission signal to the refueling management device 130 (step S409). As a result, the metering machine 135 becomes capable of refueling under the control of the refueling management device 130.

[0066] As described above, the monitoring device 110 of the fourth embodiment has multiple learning models corresponding to different vehicle types, and is configured to determine the type of vehicle included in camera footage and execute the AI ​​refueling judgment process using the learning model corresponding to the determined vehicle type. Furthermore, the monitoring device 110 is configured to analyze the camera footage in advance and execute the AI ​​refueling judgment process if it is determined that the camera footage obtained is suitable for the AI ​​refueling judgment process. This configuration also reduces errors in determining whether self-service refueling is permitted.

[0067] Up to now, the mechanisms for reducing errors in determining whether or not self-service refueling is permitted have been described using the first to fourth embodiments, but the present invention is not limited to these. For example, some of the methods shown in the first to fourth embodiments may be combined and implemented.

[0068] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and modifications thereof are included in the scope and spirit of the invention described in this specification, etc., and are included in the invention described in the claims and their equivalents.

[0069] Furthermore, the present invention can be provided not only as devices such as those described above or as systems composed of these devices, but also as methods executed by these devices, programs for realizing the functions of these devices using a processor, and storage media for storing such programs in a computer-readable manner. [Industrial Applicability]

[0070] The present invention relates to a fuel supply monitoring system that is applied to a gas station where self-service fuel supply is possible. [Explanation of symbols]

[0071] 100: Fuel supply monitoring system, 110: Monitoring device, 111: Image analysis device, 112: Control device, 113: Wireless device, 114: Learning model storage unit, 120: Terminal device, 121: Control unit, 122: Communication unit, 123: Display unit, 124: Operation unit, 130: Fuel supply management device, 135: Meter, 140: Surveillance camera, 150: Sensor

Claims

1. Cameras installed at gas stations, a monitoring device that executes a refueling determination process that determines whether or not refueling of the vehicle photographed by the camera is permitted by analyzing the image of the camera using a learning model prepared in advance; The monitoring device determines whether there is a problem with using the learning model by analyzing the image from the camera, and if the image shows the body of a person filling the fuel tank of the vehicle, it determines that there is a problem with using the learning model.If the analysis shows that there is no problem with using the learning model, it executes the refueling judgment process using the learning model, and if there is a problem with using the learning model, it controls the system so that the judgment of whether refueling of the vehicle is permitted is carried out using a different method.

2. 2. The fuel supply monitoring system according to claim 1, A refueling monitoring system characterized in that the monitoring device determines that there is a problem with using the learning model even if the vehicle's fuel tank opening is out of the field of view of the camera's image.

3. 2. The fuel supply monitoring system according to claim 1, A refueling monitoring system characterized in that the monitoring device determines that the use of the learning model will be hindered even if video of the refueler in a specific posture is captured.

4. 2. The fuel supply monitoring system according to claim 1, the learning model is a first learning model for a vehicle whose type corresponds to a first type, The refueling monitoring system is characterized in that the monitoring device determines the type of vehicle contained in the camera image and, if the determined vehicle type is not the first type, determines that there is a problem with using the learning model.

5. 5. The refueling monitoring system according to claim 1, A refueling monitoring system characterized in that, if the monitoring device determines that there is a problem with the use of the learning model, it notifies a gas station employee to visually determine whether or not refueling of the vehicle is permitted.

6. 5. The fuel supply monitoring system according to claim 4, The learning model further includes a second learning model for a vehicle that falls into a second type different from the first type, A refueling monitoring system characterized in that, when the determined vehicle type is not the first type and therefore the use of the first learning model is hindered, if the determined vehicle type is the second type, the monitoring device executes the refueling judgment process using the second learning model.

7. Cameras installed at gas stations, a monitoring device that executes a refueling determination process that determines whether or not refueling of the vehicle photographed by the camera is permitted by analyzing the image of the camera using a learning model prepared in advance; The learning model includes a plurality of learning models each corresponding to a type of vehicle, A refueling monitoring system characterized in that the monitoring device determines the type of vehicle contained in the camera image and performs the refueling judgment process using a learning model corresponding to the determined vehicle type.

8. Cameras installed at gas stations, a monitoring device that executes a refueling determination process that determines whether or not refueling of the vehicle photographed by the camera is permitted by analyzing the image of the camera using a learning model prepared in advance; The learning model includes a first learning model for a vehicle that corresponds to a first type, and a second learning model for a vehicle that corresponds to a second type different from the first type, A refueling monitoring system characterized in that the monitoring device executes the refueling judgment process using the first learning model, and if the refueling judgment process fails or if the refueling judgment process determines that refueling is not permitted, executes the refueling judgment process again using the second learning model.

9. Cameras installed at gas stations, a monitoring device that executes a refueling determination process that determines whether or not refueling of the vehicle photographed by the camera is permitted by analyzing the image of the camera using a learning model prepared in advance; The learning model includes a first learning model for a vehicle that corresponds to a first type, and a second learning model for a vehicle that corresponds to a second type different from the first type, The monitoring device determines the type of vehicle contained in the camera image and determines whether there is a problem with using the first learning model by analyzing the camera image, and if the analysis shows that there is no problem with using the first learning model, it executes the refueling judgment process using the first learning model, and if there is a problem with using the first learning model because the determined vehicle type is not the first type, and if the determined vehicle type is the second type, it executes the refueling judgment process using the second learning model.

10. 10. The fuel supply monitoring system according to claim 6, A refueling monitoring system characterized in that, when the result of the refueling determination process determines that the vehicle is not in a state in which refueling is permitted, the monitoring device notifies a gas station employee to visually determine whether the vehicle is in a state in which refueling is permitted.

Images