Fueling monitor system

Abstract

To provide a fueling monitor system capable of appropriately dealing with a situation in which a fueling person takes an action that inhibits fueling propriety determination.SOLUTION: A fueling monitor system 100 comprises: an image analysis device 111 that carries out the fueling propriety determination based on a video of a monitoring camera 140 installed in a fueling station; an event detection part 201 that detects various events which can occur during self-fueling; a data storage part 202 that stores notification control data defining a notification to a user that should be carried out when the event occurs, in association with the event related to a reduction in accuracy of the fueling propriety determination; and a notification processing part 203 that carries out notification processing based on the notification control data corresponding to the event, in response to the event being detected.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a refueling monitoring system that determines whether or not refueling is possible based on images captured by cameras installed at gas stations. [Background technology]

[0002] At conventional gas stations, gas station staff (supervisors) inside the building would monitor the behavior of customers (fuel dispensers) filling up their tanks visually or on a surveillance monitor, and then operate a fuel dispenser management device such as a handheld terminal to issue fuel dispense permission to the meter. In recent years, the development of systems that use AI (Artificial Intelligence) to analyze surveillance camera footage has progressed, and these systems are beginning to be applied to the monitoring of self-service gas dispensers.

[0003] Here, the following are examples of prior art in the technical field related to the present invention. For example, Patent Document 1 discloses a self-service refueling management system that can easily determine whether or not a customer is behaving abnormally and reduce the effort required of store staff to check. Furthermore, Patent Document 2 discloses a self-service refueling monitoring system that can efficiently collect learning data for retraining a learning model used to monitor self-service refueling. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-91460 [Patent Document 2] Patent No. 7273109 Summary of the Invention [Problem to be solved by the invention]

[0005] When operating a refueling monitoring system that determines whether or not fueling is permitted based on video footage from cameras installed at gas stations, there is a problem in that the accuracy of the determination decreases if the fuel dispenser takes actions that interfere with the determination. However, the fuel dispenser is not aware of what actions will lead to a decrease in the accuracy of the determination. As a result, there is a possibility that the fuel dispenser may unintentionally take actions that interfere with the determination of whether or not fueling is permitted, resulting in a decrease in the accuracy of the determination.

[0006] The present invention was made in consideration of the above-mentioned conventional circumstances, and aims to provide a refueling monitoring system that can appropriately deal with situations in which a refueler takes actions that hinder the determination of whether or not refueling is possible. [Means for solving the problem]

[0007] In order to achieve the above object, a refueling monitoring system according to one aspect of the present invention is configured as follows: That is, a refueling monitoring system that determines whether refueling is possible based on video footage from a camera installed at a gas station is characterized by including: a storage unit that stores notification control data that is associated with an event related to a decrease in the accuracy of the refueling possibility determination and defines a notification to be given to a refueler when the event occurs; and a notification processing unit that, in response to detection of the event, performs notification processing based on the notification control data corresponding to the event.

[0008] Here, in the above-mentioned refueling monitoring system, events related to a decrease in the accuracy of the determination of whether or not refueling is possible include an event in which the refueler cannot be detected in the camera image during refueling, and the notification to be given to the refueler when such an event occurs may include a notification to guide the refueler to an appropriate posture or position.

[0009] Furthermore, in the above-mentioned refueling monitoring system, events related to a decrease in the accuracy of the determination of whether refueling is possible include an event in which the refueler covers the nozzle in the camera image during refueling, and notifications to be given to the refueler when such an event occurs may include a notification to guide the refueler to an appropriate posture.

[0010] Furthermore, in the above-mentioned refueling monitoring system, events related to a decrease in the accuracy of the determination of whether or not refueling is possible include an event in which the posture of the refueler is inappropriate in the image captured by the camera during refueling, and the notification to be given to the refueler when such an event occurs may include a notification to guide the refueler to an appropriate posture.

[0011] In addition, in the above-mentioned refueling monitoring system, events related to a decrease in the accuracy of the determination of whether refueling is possible include an event in which the refueler's position is inappropriate in the camera image during refueling, and the notification to be given to the refueler when such an event occurs may include a notification to guide the refueler to an appropriate position.

[0012] In addition, in the above-mentioned refueling monitoring system, the notification processing unit can be configured to cancel notifications when a specified exceptional event is detected before refueling begins, such that notifications are canceled when subsequent events related to that refueling are detected.

[0013] In addition, in the above-mentioned refueling monitoring system, multiple events that may occur during self-refueling are classified into levels according to the importance of each event, and events related to a decrease in the accuracy of the refueling possibility determination are events of a predetermined level or above, and the notification processing unit can be configured to perform notification processing based on the notification control data corresponding to the event in response to the detection of an event of the predetermined level or above.

[0014] A control device according to another aspect of the present invention is configured as follows: The control device is characterized by including a storage unit that stores notification control data that is associated with an event related to a decrease in the accuracy of the refueling availability determination and defines a notification to be given to a refueler when the event occurs, and a notification processing unit that, in response to detection of the event, performs notification processing based on the notification control data corresponding to the event.

[0015] A refueling monitoring method according to yet another aspect of the present invention is configured as follows: That is, a refueling monitoring method for determining whether refueling is possible based on video footage from a camera installed at a gas station, the method comprising the steps of: storing notification control data that corresponds to an event related to a decrease in the accuracy of the refueling possibility determination, and that defines a notification to be given to a refueler when the event occurs; and performing notification processing based on the notification control data corresponding to the event in response to detection of the event. [Effects of the Invention]

[0016] According to the present invention, it is possible to provide a refueling monitoring system that can appropriately deal with a situation in which a refueler has behaved in a way that interferes with the determination of whether refueling is possible. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a diagram illustrating an example of the configuration of a refueling monitoring system according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram illustrating an example of level classification of events. [Figure 3] FIG. 10 is a diagram showing an example of notification control data relating to a warning-level event. [Figure 4] FIG. 10 is a diagram showing an outline of a processing flow relating to notification processing. [Figure 5] FIG. 10 is a diagram showing an example of a processing flow before refueling is permitted. [Figure 6] FIG. 10 is a diagram showing an example of a processing flow during refueling after refueling is permitted. DETAILED DESCRIPTION OF THE INVENTION

[0018] An embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows an example of the configuration of a refueling monitoring system 100 according to an embodiment of the present invention. The refueling monitoring system 100 of this example has a function of determining whether refueling is possible based on images from cameras installed at gas stations. The refueling monitoring system 100 includes a monitoring device 110, a terminal device 120, a refueling management device 130, a weighing machine 135, a monitoring camera 140, and a sensor 150.

[0019] The surveillance camera 140 is installed at any position within the gas station, captures images of the self-service refueling, and outputs the captured image data (camera footage) to the monitoring device 110. The surveillance camera 140 is installed for each fueling lane (vehicle stopping area) adjacent to the weighing machine 135 so as to capture images of vehicles and fuel dispensers (users) in the fueling lane. The surveillance camera 140 is installed, for example, in a position and orientation that overlooks from above an area including the area between the weighing machine 135 and the vehicle.

[0020] The sensor 150 senses sound, heat, odor, light, oil leakage, or the like that is generated due to the actions of the fuel dispenser, and outputs the results to the monitoring device 110. The sensor 150 also includes a sensor that detects the removal of the fuel nozzle from the metering machine 135. The sensor 150 is installed, for example, on the metering machine 135 or in its vicinity (such as around the fueling lane).

[0021] The monitoring device 110 is installed in a location (for example, inside a gas station building) away from the weighing machine 135. The monitoring device 110 has an image analysis device 111, a control device 112, a wireless device 113, and a learning model storage unit 114. The learning model storage unit 114 stores a learning model prepared in advance for AI analysis of camera footage. The image analysis device 111 executes a refueling availability determination process based on the learning model in the learning model storage unit 114. That is, the image analysis device 111 analyzes the camera footage using AI to determine whether refueling is permitted for the vehicle in the camera footage (i.e., whether refueling is permitted or not). The refueling availability determination process also takes into account the sensing results of the sensor 150. The wireless device 113 performs wireless communication with the terminal device 120. The control device 112 is interposed between the monitoring camera 140, the sensor 150, the fuel supply management device 130, the image analysis device 111, and the wireless device 113, and comprehensively controls the operations of these devices / equipment. The monitoring device 110 is, for example, a computer equipped with hardware resources such as a processor and memory, and is configured so that the processor executes programs for realizing each function according to the present invention.

[0022] In the refueling permission determination process, it is confirmed whether there are any problems with multiple check items when refueling, based on the camera footage captured by the monitoring camera 140 and the sensing results by the sensor 150. For example, it is confirmed that the fuel nozzle of the metering device 135 is properly inserted (fully inserted) into the fuel filler opening of the vehicle, that there is no heat source such as a cigarette, that there is no portable can, that multiple people are not refueling, etc. If it is confirmed that there are no problems with all of these check items, it is determined that the state is a refueling permission possible state in which refueling is permitted, and if not, it is determined that the state is a refueling permission prohibited state in which refueling is not permitted.

[0023] The terminal device 120 is a device used by a gas station worker (monitor). The terminal device 120 has a control unit 121, a communication unit 122, a display unit 123, and an operation unit 124. The control unit 121 comprehensively controls the operation of each unit of the terminal device 120. The communication unit 122 performs wireless communication with the monitoring device 110. The display unit 123 displays various information including the results of the refueling availability determination process performed by the monitoring device 110. The operation unit 124 accepts various user operations including permission or suspension of self-service refueling. The display unit 123 and the operation unit 124 may be integrated into a touch panel. The terminal device 120 is, for example, a computer equipped with hardware resources such as a processor and memory, and is configured so that the processor executes programs for realizing each function according to the present invention.

[0024] The terminal device 120 receives and displays the results of the refueling availability determination process from the monitoring device 110. The terminal device 120 can also display camera footage along with the results of the refueling availability determination process. The worker considers whether to permit or prohibit (not permit) refueling after referring to the information displayed on the terminal device 120, and if the worker determines that refueling should be permitted, performs an operation corresponding to that on the terminal device 120. Furthermore, if the worker determines that refueling should be stopped (interrupted) during refueling after permitting it, the worker performs an operation corresponding to that on the terminal device 120.

[0025] When the terminal device 120 receives the above-described operation, it transmits a control signal indicating the operation (for example, a refueling permission signal or a refueling stop signal) to the refueling management device 130. The control signal is also accompanied by information identifying the target refueling lane (for example, a lane number). The control signal corresponding to the operation of the terminal device 120 may be transmitted to the refueling management device 130 via the monitoring device 110, or may be transmitted directly to the refueling management device 130 without going through the monitoring device 110. Here, an example will be described in which the terminal device 120 is a portable terminal (i.e., a handheld terminal) such as a tablet that can be carried by an operator, but it may also be a stationary terminal installed (fixed) at a predetermined location.

[0026] The fuel supply management device 130 is also referred to as a self-service controller (SSC), and is installed, for example, in a location remote from the metering device 135 (for example, inside a gas station building). The fuel supply management device 130 controls the operation of the metering device 135 based on a control signal transmitted in response to an operation of the terminal device 120. For example, when the terminal device 120 receives an operation to permit refueling, a fuel supply permission signal is transmitted from the terminal device 120 to the fuel supply management device 130. Upon receiving the fuel supply permission signal, the fuel supply management device 130 drives a pump inside the metering device 135 corresponding to the fuel supply permission signal, thereby enabling the metering device 135 to perform refueling. Furthermore, for example, when the terminal device 120 receives an operation to stop refueling, a fuel supply stop signal is transmitted from the terminal device 120 to the metering device 135. The fuel supply management device 130 that has received the fuel supply stop signal stops the pump inside the metering device 135 that corresponds to the fuel supply stop signal, and as a result, the metering device 135 is in a state where it cannot supply fuel.

[0027] The main feature of this system is that it is equipped with a mechanism for dealing with situations in which the fuel dispenser has engaged in behavior that interferes with the determination of whether or not fueling is possible. This mechanism will be explained in detail below. As shown in Figure 1, the control device 112 in the monitoring device 110 of this system has an event detection unit 201, a data storage unit 202, and a notification processing unit 203.

[0028] Event detection unit 201 performs processing to detect various events that may occur during self-service refueling, based on the video from surveillance camera 140. Event detection may be performed by event unit 201 analyzing the video from surveillance camera 140 on its own, or may be performed using the analysis results of another device (e.g., image analysis device 111) that analyzes the video from surveillance camera 140. Event detection may also be performed using the sensing results from sensor 150, in addition to the video from surveillance camera 140.

[0029] The data storage unit 202 stores notification control data that defines the notification content corresponding to each event that may occur during self-service refueling. The notification control data is realized by a data structure that associates, for example, an event ID that identifies each event that may occur during self-service refueling, data indicating the content of the message to be notified when the occurrence of that event is detected, and a device ID that identifies the message output method (audio output, display output, etc.) and the output destination device.

[0030] In response to an event being detected by the event detection unit 201, the notification processing unit 203 performs notification processing based on the notification control data in the data storage unit 201 that corresponds to the event. As one example, the notification by the notification processing unit 203 is sent to and displayed on the terminal device 120, and confirmed by the monitor. As another example, the notification by the notification processing unit 203 is sent to and displayed on a device that can be seen by the fuel dispenser (for example, a display provided on the weighing machine 135), and confirmed by the fuel dispenser.

[0031] In this example, multiple events that may occur during self-service refueling are managed by dividing them into levels according to the importance of each event. Figure 2 shows an example of dividing events into three levels, "Warning," "Caution," and "Information," in descending order of importance. A warning-level event is an event that requires refueling to be stopped immediately and a warning to be issued to the refueler. A caution-level event is an event that does not require refueling to be stopped, but does require the refueler to be alerted. An information-level event is an event that does not require refueling to be stopped, and does not require notification to the refueler.

[0032] Here, events related to a decrease in the accuracy of the refueling possibility determination are events for which the refueler should be notified, and are therefore classified into one of the two highest levels, i.e., warning level or caution level. Note that events related to a decrease in the accuracy of the refueling possibility determination may include not only detrimental events that cause a decrease in the accuracy of the refueling possibility determination, but also events that indicate a decrease in the accuracy of the refueling possibility determination itself. Whether the accuracy of the refueling possibility determination has decreased can be determined, for example, by comparing the confidence level associated with the determination result when the refueling possibility determination is made with a predetermined threshold. In other words, if the confidence level of the determination result is less than the threshold, it can be recognized as an event that has decreased the accuracy of the refueling possibility determination.

[0033] FIG. 3 shows an example of notification control data for warning-level events stored in the data storage unit 202. The example in FIG. 3 lists four events: "fuel dispenser not detected," "nozzle hidden," "fuel dispenser posture is inappropriate," and "fuel dispenser position is inappropriate." "Fuel dispenser not detected" is an event in which a fuel dispenser cannot be detected in camera footage during refueling, and it is set that a notification should be issued to guide the fuel dispenser to an appropriate posture or position. "nozzle hidden" is an event in which the fuel dispenser hides the nozzle in camera footage during refueling, and it is set that a notification should be issued to guide the fuel dispenser to an appropriate posture. "fuel dispenser posture is inappropriate" is an event in which the fuel dispenser posture is inappropriate in camera footage during refueling, and it is set that a notification should be issued to guide the fuel dispenser to an appropriate posture. "fuel dispenser position is inappropriate" is an event in which the fuel dispenser position is inappropriate in camera footage during refueling, and it is set that a notification should be issued to guide the fuel dispenser to an appropriate position.

[0034] An overview of the operation of the above-mentioned processing units 201 to 203 will be described with reference to the processing flow in Fig. 4. Fig. 4 shows an overview of the processing flow related to the notification processing. The event detection unit 201 acquires and analyzes images taken by the monitoring camera 140 at predetermined intervals (for example, every 3 seconds) (step S101), and detects an event that occurs during self-service refueling.

[0035] Next, the detection result by the event detection unit 201 is determined (step S102). Depending on the result, the notification processing unit 203 performs notification processing based on the notification control data in the data storage unit 201 corresponding to the detected event. That is, if it is determined in step S102 that an information-level event has been detected, an information message transmission process (step S103) is performed, if it is determined that a caution-level event has been detected, an caution message transmission process (step S104) is performed, and if it is determined that a warning-level event has been detected, an alert message transmission process (step S105) is performed.

[0036] Here, in the information message transmission process (step S103), the monitor or fuel supplier is notified by a predetermined method, such as displaying an icon or flashing a lamp, that an information message corresponding to the detected information-level event is ready to be transmitted. Thereafter, the monitor or fuel supplier requests transmission of an information message as needed, and the information message is transmitted, allowing the monitor or fuel supplier to confirm the message content. Meanwhile, in the caution message transmission process (step S104), a caution message corresponding to the detected caution-level event is automatically transmitted, prompting the monitor or fuel supplier to confirm the message content. Similarly, in the warning message transmission process (step S105), a warning message corresponding to the detected warning-level event is automatically transmitted, prompting the monitor or fuel supplier to confirm the message content. At this time, the degree of emphasis of the output for caution messages and warning messages may be changed so that the monitor or fuel supplier can understand the difference in the importance of the messages.

[0037] The notification process will be described in detail with reference to the process flows of FIGS. FIG. 5 shows an example of a processing flow before refueling is permitted. Before refueling is permitted, similar to the description of steps S101 and S102 in FIG. 4, images captured by the monitoring camera 140 are acquired and analyzed (step S201), and a determination is made regarding the event detected as a result (step S202). Here, it is assumed that one of the following events has been detected: "nozzle inserted," which is an example of an information-level event; "nozzle hidden," which is an example of a caution-level event; or "hazardous material detected," which is an example of a warning-level event. For example, if a nozzle is detected in an image and then becomes undetectable, it is determined that a "nozzle hidden" event has been detected. Alternatively, it may be determined that a "nozzle hidden" event has been detected if a nozzle is not detected in an image after refueling is permitted, if an object (such as a person's head) is detected at the position of the nozzle in the image, or if a specific posture of the person refueling (such as leaning forward) is detected in the image.

[0038] If it is determined in step S202 that a "nozzle insertion" event has been detected, it is determined that the state is such that refueling can be permitted (step S203), provided that all other check items (for example, no heat source such as cigarettes is present, no portable can is present, and multiple people are not refueling) are satisfactory, and the process proceeds to the start of refueling (step S204) (details omitted), after which the processing shown in the judgment flow during refueling (FIG. 6) is performed. Also, simultaneously with / before or after the processing of steps S203 and S204, processing to send an information message corresponding to the "nozzle insertion" event (step S205) is also performed, as described in step S103 of FIG. 4.

[0039] If it is determined in step S202 that a "nozzle hidden" event has been detected, it is impossible to determine whether refueling is permitted or not, and a process of transmitting a warning message corresponding to the "nozzle hidden" event (step S206) is performed, as described in step S104 of Fig. 4. If it is determined that a "hazardous material detected" event has been detected, it is determined that a refueling permission prohibition state in which refueling is not permitted (step S207), and a process of transmitting a warning message corresponding to the "hazardous material detected" event (step S208) is performed, as described in step S105 of Fig. 4. Thereafter, the process returns to step S201, and the process is repeated for the next image captured by the monitoring camera 140.

[0040] Fig. 6 shows an example of a processing flow during refueling after refueling is permitted. During refueling after refueling is permitted, similar to the description of steps S101 and S102 in Fig. 4, images captured by monitoring camera 140 are acquired and analyzed (step S301), and a determination is made regarding the event detected as a result (step S302). Here, it is assumed that any of "nozzle inserted," which is an example of an information-level event, "nozzle hidden," which is an example of a caution-level event, or "nozzle not detected," which is an example of a warning-level event, has been detected.

[0041] If it is determined in step S302 that a "nozzle inserted" event has been detected, it is possible to continue permitting refueling, and similar to the description of step S103 in Fig. 4, a process of transmitting an information message corresponding to the "nozzle inserted" event (step S303) is performed. Also, if it is determined that a "nozzle hidden" event has been detected, it is not possible to determine whether to continue permitting refueling or to stop refueling, and similar to the description of step S104 in Fig. 4, a process of transmitting a warning message corresponding to the "nozzle hidden" event (step S304) is performed. Thereafter, the process returns to step S301, and the process is repeated for the next image captured by monitoring camera 140.

[0042] Furthermore, if it is determined that the event of "nozzle not detected" has been detected, it is necessary to stop refueling, and a process to forcibly stop refueling for the vehicle in question is carried out (step S305), and a process to transmit a warning message corresponding to the event of "nozzle not detected" (step S306) is carried out, as explained in step S105 of Fig. 4. Thereafter, the process is repeated until "nozzle insertion" is detected (step S307), and after "nozzle insertion" is detected, the refueling stop is released (i.e., refueling is resumed) (step S308).

[0043] As described above, the refueling monitoring system 100 of this example includes an image analyzer 111 that determines whether refueling is possible based on images from a surveillance camera 140 installed at a gas station, an event detection unit 201 that detects various events that may occur during self-refueling, a data storage unit 202 that stores notification control data that corresponds to an event related to a decrease in the accuracy of the refueling possibility determination and defines the notification to be given to the refueler when the event occurs, and a notification processing unit 203 that performs notification processing based on the notification control data corresponding to the event in response to the detection of an event. With this configuration, the refueling monitoring system 100 of this example can appropriately deal with situations in which the refueler has engaged in behavior that hinders the determination of whether refueling is possible.

[0044] Here, several examples of events related to a decrease in the accuracy of the refueling availability determination have been described, but various events including these will be listed below. Note that the events listed below are merely examples, and the events to which the present invention can be applied are not limited to these. Furthermore, no particular mention is made of the level of each event (information level, caution level, or warning level), but these levels may be set appropriately depending on the importance, urgency, etc. of each event.

[0045] Examples of events that may occur when stopping a vehicle include an event where the vehicle is too far from the stopping frame, an event where the vehicle is too close to the weighing machine, or an event where the vehicle is too far from the weighing machine. These events can be detected, for example, by analyzing video from a camera other than the camera used to determine whether or not refueling is possible. When such an event is detected, control is performed so that a notification to guide the vehicle to an appropriate location is output as audio from the speaker. The reason for outputting audio rather than visual output is that a refueler is present in the vehicle when it is stopped, so a visual notification is less effective.

[0046] Other examples of events that may occur when stopping a vehicle include an event where the vehicle is a specific model, or an event where the vehicle cannot be detected. These events can be detected, for example, by analyzing the video from a camera used to determine whether or not refueling is possible. When such an event is detected, the accuracy of the determination of whether or not refueling is possible is significantly reduced, so control is exercised to cancel (turn off) notifications when subsequent events related to that refueling are detected. This makes it possible to suppress the output of notifications caused by erroneous determinations by the AI. An example of a specific model of vehicle that significantly reduces the accuracy of the determination of whether or not refueling is possible is a special vehicle with a fuel tank opening in a location that is not visible from the camera.

[0047] An example of an event that may occur during refueling is an event in which the refueler cannot be detected in the image of the camera used to determine whether refueling is possible. If this event is detected, it is highly likely that the refueler's position or posture is inappropriate, so control is exercised to display and output a notification to guide the refueler to an appropriate posture or position on a device that can be seen by the refueler (for example, a display provided on the weighing machine 135). Audio output from a speaker may also be used in combination.

[0048] Another example of an event that may occur during refueling is when the refueler covers the nozzle in the image captured by the camera used to determine whether refueling is possible. This event is likely to occur when the refueler is in the correct position but has an incorrect posture. Therefore, when this event is detected, the system controls so that a notification is displayed (and audio output) to the refueler to guide them to the correct posture.

[0049] Another example of an event that may occur during refueling is an event in which the refueler's posture or position is inappropriate in the image captured by the camera used to determine whether refueling is possible. When such an event is detected, control is performed so that a notification is displayed (and audio output) to the refueler to guide the refueler to an appropriate posture or position. Examples of inappropriate refueler postures include a crouching posture or a stooped posture. Skeleton detection technology may be used to detect the refueler's posture.

[0050] Another example of an event that may occur during refueling is an event in which the refueler moves a lot in the image captured by the camera used to determine whether or not refueling is possible. If the refueler is unsteady, there is a concern that the accuracy of the determination of whether or not refueling is possible may decrease. Therefore, when this event is detected, the system controls so that a notification is displayed (and audio output) to the refueler to guide them to an appropriate posture.

[0051] Another example of an event that may occur during refueling is an event in which the accuracy of the determination as to whether or not refueling is possible decreases. A decrease in the accuracy of the determination as to whether or not refueling is possible is likely to occur when the position or posture of the person refueling is inappropriate, so when this event is detected, control is performed so that a notification is displayed (and audio output) to the person refueling to guide them to an appropriate position or posture. The accuracy of the determination as to whether or not refueling is possible can be determined by comparing the confidence level associated with the determination result with a predetermined threshold.

[0052] In addition, the above-mentioned events may occur instantaneously, and in such cases, constantly sending notifications would be wasteful and could be a nuisance to the observer or fuel supplier. Therefore, it is possible to notify the observer or fuel supplier only when the rate at which the event occurs within a certain period of time is equal to or exceeds a predetermined value (for example, when an event is detected in two out of five frames of the camera image).

[0053] Although the present invention has been described with reference to an example in which a warning message is sent in response to an event related to a decrease in the accuracy of the determination of whether or not refueling is possible, this may also be applied to other events. For example, a warning message may be sent in response to an event related to the refueling flow, such as an event of a vehicle stopping, an event of starting to operate the refueling machine, or an event during refueling. In this way, a warning message is sent to the refueler every time the refueler refuels, making the refueler aware of an attitude that will not decrease the accuracy of the determination of whether or not refueling is possible.

[0054] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and modifications thereof are included in the scope and spirit of the invention described in this specification, etc., and are included in the invention described in the claims and their equivalents.

[0055] Furthermore, the present invention can be provided not only as devices such as those described above or as systems composed of these devices, but also as methods executed by these devices, programs for realizing the functions of these devices using a processor, and storage media for storing such programs in a computer-readable manner. [Industrial Applicability]

[0056] The present invention can be used in a refueling monitoring system that determines whether or not refueling is possible based on images from cameras installed at gas stations. [Explanation of symbols]

[0057] 100: Fuel supply monitoring system, 110: Monitoring device, 111: Image analysis device, 112: Control device, 113: Wireless device, 114: Learning model storage unit, 120: Terminal device, 121: Control unit, 122: Communication unit, 123: Display unit, 124: Operation unit, 130: Fuel supply management device, 135: Meter, 140: Surveillance camera, 150: Sensor, 201: Event detection unit, 202: Data storage unit, 203: Notification processing unit

Claims

1. In a fuel supply monitoring system that determines whether or not fuel can be supplied based on images from a camera installed at a gas station, a storage unit that stores notification control data that corresponds to an event related to a decrease in accuracy of the refueling availability determination, and defines a notification to be given to a refueler when the event occurs; a notification processing unit that performs notification processing based on the notification control data corresponding to the event in response to the event being detected.

2. 2. The fuel supply monitoring system according to claim 1, A refueling monitoring system characterized in that events related to a decrease in the accuracy of the refueling possibility determination include an event in which the refueler cannot be detected in the camera image during refueling, and the notification to be given to the refueler when such an event occurs includes a notification to guide the refueler to an appropriate posture or position.

3. 2. The fuel supply monitoring system according to claim 1, A refueling monitoring system characterized in that events related to a decrease in the accuracy of the determination of whether or not refueling is possible include an event in which the refueler covers the nozzle in the camera image during refueling, and the notification to be given to the refueler when such an event occurs includes a notification to guide the refueler to an appropriate posture.

4. 2. The fuel supply monitoring system according to claim 1, A refueling monitoring system characterized in that events related to a decrease in the accuracy of the refueling possibility determination include an event in which the refueler's posture is inappropriate in the image captured by the camera during refueling, and the notification to be given to the refueler when such an event occurs includes a notification to guide the refueler to an appropriate posture.

5. 2. The fuel supply monitoring system according to claim 1, A refueling monitoring system characterized in that events related to a decrease in the accuracy of the refueling possibility determination include an event in which the refueler's position is inappropriate in the image captured by the camera during refueling, and the notification to be given to the refueler when such an event occurs includes a notification to guide the refueler to an appropriate position.

6. 2. The fuel supply monitoring system according to claim 1, A refueling monitoring system characterized in that, when a predetermined exceptional event is detected before the start of refueling, the notification processing unit cancels notifications when subsequent events related to that refueling are detected.

7. 2. The fuel supply monitoring system according to claim 1, The multiple events that can occur during self-service refueling are classified according to the importance of each event. The event related to the deterioration of accuracy of the fuel supply possibility determination is an event of a predetermined level or more, The refueling monitoring system is characterized in that, in response to the detection of an event of a level equal to or higher than the predetermined level, the notification processing unit performs notification processing based on the notification control data corresponding to the event.

8. a storage unit that stores notification control data that defines a notification to be given to a fuel dispenser when an event related to a decrease in accuracy of the fuel dispense possibility determination occurs, in association with the event; a notification processing unit that performs notification processing based on the notification control data corresponding to the event in response to the event being detected.

9. A refueling monitoring method for determining whether or not refueling is possible based on images from a camera installed at a gas station, a step of storing notification control data that defines a notification to be given to a fuel dispenser when an event related to a decrease in accuracy of the fuel dispense possibility determination is generated, in association with the event; a step of performing notification processing based on the notification control data corresponding to the event in response to the event being detected.

Images