Computer program, terminal program, and apparatus and method for processing information

A computer program processes resident reports with road images to identify administrators and transmit tailored advice, addressing the inefficiencies in conventional systems by automating response determination in road maintenance management.

JP2024161811APending Publication Date: 2024-11-20THE JAPAN RES INST
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023076867
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-08
Publication Date
2024-11-20

AI Technical Summary

Technical Problem

Conventional road maintenance management systems lack the ability to automatically determine appropriate responses to resident reports of road damage based on the content and state of the damage, requiring manual decision-making by administrators.

Method used

A computer program that processes resident reports including road images and identifies the relevant administrator, transmitting tailored advice information based on the image content and state of the damage.

Benefits of technology

Enables automated transmission of advice information specific to the report content, improving response efficiency and accuracy in road maintenance management systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024161811000001_ABST
    Figure 2024161811000001_ABST
Patent Text Reader

Abstract

To provide a computer program, terminal program, and apparatus and method for processing information adapted to send advice information matched with a call content.SOLUTION: A computer program allows a computer to execute processing including: receiving call information including a road image including a road as a subject, date and hour of shooting and a site of shooting; identifying an administrator of the road based on the site of shooting; and sending the call information and advice information identified by the road image to an administrator terminal of the identified administrator.SELECTED DRAWING: Figure 11
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a computer program, a terminal program, an information processing device, and an information processing method for transmitting advice information. [Background technology]

[0002] Roads, which are social infrastructure, are expected to be in a state where they can be used 24 hours a day, 365 days a year. To contribute to the realization of this, a road maintenance management system for road managers has been proposed (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7023690 Summary of the Invention [Problem to be solved by the invention]

[0004] Road users vary, but most of the users of roads that pass through suburban and residential areas are local residents. Therefore, it is expected that residents will report damage to roads. The road level management system described in Patent Document 1 also has a function for receiving reports from residents.

[0005] The content of reports from residents varies, and the response that road administrators should take varies depending on the content. Even reports about damage require different responses, such as immediate action or waiting for a while, depending on the state of the damage and the repair plan for the road in question. However, in conventional road maintenance systems, the decision on the response to be taken is left to the road administrator.

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a computer program, a terminal program, an information processing device, and an information processing method for transmitting advice information according to the contents of a report. [Means for solving the problem]

[0007] A computer program according to one embodiment of the present application causes a computer to execute a process of receiving notification information including a road image in which a road is included as a subject, the date and time of shooting, and the location of the shooting, identifying an administrator of the road based on the location of the shooting, and transmitting the notification information and advice information identified based on the road image to an administrator terminal of the identified administrator. Effect of the Invention

[0008] In one aspect of the present application, it is possible to transmit advice information according to the contents of the report. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing an example of the configuration of a communication system. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a server. [Diagram 3] 3 is a block diagram showing an example of the hardware configuration of a resident terminal; FIG. [Figure 4] 4 is a block diagram showing an example of a hardware configuration of an administrator terminal; [Diagram 5] FIG. 2 is an explanatory diagram showing an example of a road administrator DB. [Figure 6] FIG. 11 is an explanatory diagram showing an example of a report DB. [Figure 7] FIG. 13 is an explanatory diagram illustrating an example of a case DB. [Figure 8] FIG. 13 is an explanatory diagram showing an example of an advice DB. [Figure 9] FIG. 13 is an explanatory diagram illustrating an example of a message DB. [Figure 10] 13 is a flowchart showing an example of a procedure for a report reception process. [Figure 11] 13 is a flowchart illustrating an example of a procedure for case registration processing. [Figure 12] FIG. 11 is an explanatory diagram showing an example of a notification setting DB. [Figure 13]13 is a flowchart illustrating an example of a procedure for a push notification request process. [Figure 14] FIG. 4 is an explanatory diagram showing an example of a message DB stored in a resident terminal. [Figure 15] FIG. 11 is an explanatory diagram showing an example of a notification DB. [Figure 16] 13 is a flowchart illustrating an example of a procedure for a local notification process. [Figure 17] FIG. 2 is an explanatory diagram showing an example of the configuration of a learning model. [Figure 18] 13 is a flowchart illustrating an example of a procedure for a learning model generation process. [Figure 19] 13 is a flowchart showing an example of a procedure for a re-reporting process. [Figure 20] FIG. 2 is an explanatory diagram showing an example of a shooting screen. [Figure 21] FIG. 13 is an explanatory diagram relating to a progress prediction model. [Figure 22] FIG. 13 is an explanatory diagram showing progress prediction using a progress prediction model. [Diagram 23] FIG. 13 is an explanatory diagram showing an example of a notification screen. [Figure 24] 13 is a flowchart illustrating an example of a procedure for a status display process. [Diagram 25] FIG. 13 is an explanatory diagram showing an example of a case status screen. [Figure 26] FIG. 13 is an explanatory diagram showing an example of a notification screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (Basic functions) An embodiment will be described below with reference to the drawings. FIG. 1 is an explanatory diagram showing an example of the configuration of a communication system. The communication system 100 includes a server 1, a resident terminal 2, and a manager terminal 3. The server 1, the resident terminal 2, and the manager terminal 3 are connected to each other via a network N so that they can communicate with each other. By using a push notification service 4, the server 1 is able to send push notifications to the resident terminal 2 without collecting personal information. In the following description, "road" includes "roads" and "road structures" unless otherwise specified. Road structures include "bridges," "tunnels," "sheds," "large culverts," "pedestrian crossings," and "gate-type signs, etc."

[0011] The server 1 is composed of a server computer, a workstation, a PC (Personal Computer), etc. The server 1 may be composed of a multi-computer consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer. The functions of the server 1 may be realized by a cloud service.

[0012] The resident terminal 2 is a terminal used by road users, particularly local residents. The resident terminal 2 is composed of a tablet computer, a smartphone, a laptop computer, etc. Although only one resident terminal 2 is shown in FIG. 1, it is assumed that there are two or more resident terminals.

[0013] The administrator terminal 3 is a terminal used by the road administrator. The resident terminal 2 is composed of a notebook computer, a tablet computer, a smartphone, etc. Although only one administrator terminal 3 is shown in FIG. 1, it is assumed that there are two or more administrator terminals.

[0014] 2 is a block diagram showing an example of the hardware configuration of the server 1. The server 1 includes a control unit 11, a main memory unit 12, an auxiliary memory unit 13, a communication unit 15, and a reading unit 16. Each component is connected to a bus B.

[0015] The control unit 11 has one or more arithmetic processing devices such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), etc. The control unit 11 reads out and executes a control program 1P (program, program product) stored in the auxiliary storage unit 13, thereby performing various information processing, control processing, etc. related to the communication system 100, and realizes functional units such as a receiving unit, a specifying unit, and a transmitting unit.

[0016] The main memory unit 12 is a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, etc. The main memory unit 12 mainly temporarily stores data necessary for the control unit 11 to execute arithmetic processing.

[0017] The auxiliary memory unit 13 is a hard disk or a solid state drive (SSD), etc., and stores the control program 1P and various DBs (databases) required for the control unit 11 to execute processing. The auxiliary memory unit 13 stores a road administrator DB 131, a report DB 132, a case DB 133, an advice DB 134, a message DB 135, and a notification setting DB 136. The auxiliary memory unit 13 also stores a learning model 141. The auxiliary memory unit 13 may be an external storage device separate from the server 1 and externally connected. The various DBs, etc. stored in the auxiliary memory unit 13 may be stored in a database server or cloud storage different from the server 1.

[0018] The communication unit 15 communicates with the resident terminal 2 and the manager terminal 3 via the network N. The control unit 11 may use the communication unit 15 to download the control program 1P from another computer via the network N or the like, and store it in the auxiliary storage unit 13.

[0019] The reading unit 16 reads the portable storage medium 1a including a CD (Compact Disc)-ROM and a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 16 and store it in the auxiliary storage unit 13. The control unit 11 may read the control program 1P from the semiconductor memory 1b.

[0020] 3 is a block diagram showing an example of the hardware configuration of the resident terminal 2. The resident terminal 2 includes a control unit 21, a main memory unit 22, an auxiliary memory unit 23, a communication unit 24, a display panel 25, an operation unit 26, a position acquisition unit 27, and an imaging unit .

[0021] The control unit 21 has one or more arithmetic processing devices such as a CPU, an MPU, a GPU, etc. The control unit 21 provides various functions by reading and executing a control program 2P (program, program product) stored in the auxiliary storage unit 23.

[0022] The main memory unit 22 is an SRAM, a DRAM, a flash memory, etc. The main memory unit 22 mainly temporarily stores data necessary for the control unit 21 to execute arithmetic processing.

[0023] The auxiliary memory unit 23 is a hard disk or SSD, etc., and stores various data necessary for the control unit 21 to execute processing. The auxiliary memory unit 23 stores a message DB 232 and a notification DB 233. The auxiliary memory unit 23 stores a device token 231 acquired from the push notification service 4. The auxiliary memory unit 23 may be separate from the resident terminal 2 and may be an external storage device externally connected. The various DBs, etc. stored in the auxiliary memory unit 23 may be stored in a database server or cloud storage.

[0024] The communication unit 24 communicates with the server 1 via the network N. In addition, the control unit 21 may use the communication unit 24 to download the control program 2P from another computer via the network N or the like, and store it in the auxiliary storage unit 23.

[0025] The display panel 25 can be configured with a liquid crystal panel or an organic EL (Electro Luminescence) display, etc. The operation unit 26 can be configured with, for example, a touch panel incorporated in the display panel 25, and can perform predetermined operations performed by the resident on the display panel 25. The operation unit 26 can also perform operations on a software keyboard displayed on the display panel 25. The operation unit 26 may also be a hardware keyboard, a mouse, etc.

[0026] The position acquisition unit 27 is composed of a GPS (Global Positioning System) receiver, etc. The position acquisition unit 27 receives radio waves from GPS satellites. The position acquisition unit 27 determines its own position based on the received satellite radio waves.

[0027] The imaging unit 28 is, for example, a CCD camera or a CMOS camera, and obtains image data by photoelectrically converting an optical signal input via a CCD or a CMOS.

[0028] 4 is a block diagram showing an example of the hardware configuration of the administrator terminal 3. The administrator terminal 3 includes a control unit 31, a main memory unit 32, an auxiliary memory unit 33, a communication unit , an input unit 35, and a display unit .

[0029] The control unit 31 has one or more arithmetic processing devices such as a CPU, an MPU, a GPU, etc. The control unit 31 provides various functions by reading and executing a control program 3P (program, program product) stored in the auxiliary storage unit 33.

[0030] The main memory unit 32 is an SRAM, a DRAM, a flash memory, etc. The main memory unit 32 mainly temporarily stores data necessary for the control unit 31 to execute arithmetic processing.

[0031] The auxiliary storage unit 33 is a hard disk or SSD, etc., and stores various data necessary for the control unit 31 to execute processing. The auxiliary storage unit 33 may be an external storage device that is separate from the administrator terminal 3 and externally connected. The various DBs, etc. stored in the auxiliary storage unit 33 may be stored in a database server or cloud storage.

[0032] The communication unit 34 communicates with the server 1 via the network N. In addition, the control unit 31 may use the communication unit 34 to download the control program 3P from another computer via the network N or the like, and store it in the auxiliary storage unit 33.

[0033] The input unit 35 is a keyboard, a mouse, etc. The display unit 36 ​​includes a liquid crystal display panel, an organic EL (electroluminescence) display panel, etc. The input unit 35 and the display unit 36 ​​may be integrated to configure a touch panel display. The manager terminal 3 may display on an external display device.

[0034] The database used by the communication system 100 will be described. FIG. 5 is an explanatory diagram showing an example of the road administrator DB. The road administrator DB 131 stores master data related to the road administrator. The road administrator DB 131 includes an ID column, an administrator name column, an object column, a managed section column, a connection information column, and an email address column. The ID column stores an ID that can uniquely identify the road administrator. The administrator name column stores the name of the road administrator. The object column stores the object of the road managed by the road administrator. Roads can generally be classified into four types: expressways, general national roads, prefectural roads, and municipal roads. The object column stores one of the four classifications. The managed section column stores the road sections managed by each road administrator. The road section includes the name of the road and the start and end points of the section. The connection information column stores information for connecting to the business system of the road administrator. The connection information column stores, for example, a uniform resource identifier (URI) for exchanging data with the business system of the road administrator by an open application programming interface (API) or the like. If the road administrator does not provide access via an open API, no value is set in the connection information column, or a character string indicating this is stored. The email address column stores the email address of the road administrator.

[0035] FIG. 6 is an explanatory diagram showing an example of the report DB. The report DB 132 stores the contents of reports submitted by residents. The report DB 132 includes a report ID column, a shooting date and time column, a shooting location column, an image column, a text column, a report date and time column, a name column, an address column, a location supplement column, a type column, a judgment category column, a status column, a case ID column, and a device token column. The report ID column stores a report ID that can uniquely identify a report. The shooting date and time column stores the date and time when an image included in the report was shot (shooting date and time). The shooting date and time is acquired, for example, from Exif information attached to the image. The shooting location column stores the location when the image was shot (shooting location), for example, latitude and longitude. The latitude and longitude of the shooting location are acquired from Exif information. The image column stores image data. The text column stores text entered by residents. The report date and time column stores the date and time when the report was sent. The name column stores the name or common name of the road that is the subject of the report. The address column inputs the address of the location of the report. The location supplement column stores information that supplements the address of the location of the report. The values ​​stored in the name column, address column, and location supplement column are input by the reporter, but they are not required to be input items, so there are cases where no value is set. The type column stores the type of report. If the report is about road damage, the type column stores "damage". If the report is about road safety measures, the type column stores "safety measures". The judgment category column stores the category of the report determined by the server 1 based on the transmitted image. The status column stores the status of the report. The status is, for example, "received", "sent", "linked", etc. "Received" indicates a state immediately after the report is received and the report content has not yet been sent to the road administrator. "Sent" indicates that the report has been sent to the road administrator. "Linked" indicates that the report is associated with a case. A case indicates an incident that requires the road administrator to take some kind of action. Each case is associated with at least one report. The case ID column stores the case ID of the case associated with the report. If the report is not associated with a case, the case ID column does not have a value set, or stores a character string or the like indicating this. The device token column stores a device token that can identify the resident terminal 2 that sent the report.The device token is obtained from the push notification service 4 when the resident terminal 2 installs the control program 2P in order to use the communication system 100. By using the device token, it is possible to send a push notification to the resident terminal 2 without collecting personal information. Note that in cases where the shooting date and time and shooting location cannot be obtained from the Exif information, the shooting date and time and shooting location may be required items and may be input by the resident who reported the incident.

[0036] FIG. 7 is an explanatory diagram showing an example of the case DB. The case DB 133 stores case information. The case DB 133 includes a case ID column, an administrator ID column, a reception date and time column, a road name column, a nickname column, an address column, a location supplement column, a location coordinate column, a report ID column, a confirmed category column, a response status column, and a next scheduled column. The case ID column stores a case ID that can uniquely identify a case. The administrator ID column stores an administrator ID that indicates the administrator of the road related to the case. The reception date and time column stores the date and time when the case is received (reception date and time). For example, the reception date and time is the date and time when the road administrator set the case based on the first report. The road name column stores the name of the road associated with the case. The nickname column stores the nickname if the road associated with the case has a nickname. The address column stores an address that indicates the location of the case. The location supplement column stores content that supplements the address that indicates the location of the case. The location coordinate column stores geographic coordinate values ​​that indicate the location of the case. For example, the geographic coordinate values ​​are latitude and longitude. The report ID column stores the report ID of the report associated with the case. The confirmed category column stores the category of the case that has been approved and confirmed by the road administrator. The response status column stores the status of the case. Examples of the status are "Response under consideration", "Response in progress", "Waiting", and "Response completed". "Response under consideration" indicates that the road administrator believes that some kind of response is necessary immediately and is considering the content of the response. "Response in progress" indicates that the response content has been decided and preparations and work are being carried out. "Waiting" indicates that the response content has been decided, but it is too early to start, so the response is on hold. For example, if a pothole appears on a road, it is necessary to take action to repair it. However, if the pothole is small, repairs are not made immediately, but are made when it becomes larger to a certain extent. If the pothole is waiting to become larger, the status becomes "Waiting". Potholes are holes, depressions, and cracks that occur in part of the surface of an asphalt pavement, mainly due to repeated driving of vehicles. "Response completed" indicates that the response has been completed. The next scheduled column stores the date and time of the next scheduled process. For example, when a request is made to the reporter to collect information on the status of a pothole waiting to be repaired, the next scheduled date and time of the process to set up a push notification of the request is stored.

[0037] FIG. 8 is an explanatory diagram showing an example of an advice DB. The advice DB 134 stores advice regarding a report to be attached when the report is sent to the road administrator. The advice DB 134 includes a type column, a category column, and an advice column. The type column stores the type of the report. The category column stores the category of the report content. The advice column stores the content of the advice. As shown in FIG. 8, advice is determined based on the type and category.

[0038] FIG. 9 is an explanatory diagram showing an example of a message DB. The message DB 135 stores messages to be sent from the server 1 to the resident terminal 2. The message DB 135 includes an ID column, a response type column, a case type column, and a message column. The ID column stores an ID that can uniquely identify a message. The response type column stores the type of message. "Reply" indicates that the message is in response to a report. "Encourage" indicates that the message is encouraging an additional report. "Guide" indicates that the message is advice regarding the creation of a report. The case type column stores the type of case. "New" indicates that the message is sent when a new case is started as a result of a report. "Already reported" indicates that the message is sent when a report is made about an existing case. The message column stores the message to be sent.

[0039] The information processing performed by the communication system 100 will be described. FIG. 10 is a flow chart showing an example of the procedure of the report reception processing. A resident who wishes to report (hereinafter, referred to as the "reporter") takes a picture of the road condition with the imaging unit 28 of the resident terminal 2. The control unit 21 of the resident terminal 2 acquires an image (road image) from the imaging unit 28 (step S1). The reporter inputs a description of the road condition that triggered the report. If possible, the reporter also inputs the road name, road number, the mark of the photographing position, and nearby landmarks. The control unit 21 accepts the input (step S2). The control unit 21 transmits report information including the information input by the reporter and the road image, and the device token 231 to the server 1 (step S3). The control unit 11 of the server 1 receives the report information and the device token 231 (step S4). The control unit 11 analyzes the report information and determines whether the report is a new one that is not related to an existing case (step S5). For example, the control unit 11 accesses the Exif information attached to the road image included in the report information, and if the geographic coordinate value of the shooting position is recorded, acquires the coordinate value. The control unit 11 compares the acquired coordinate value with the position coordinates of each case stored in the case DB 133, and judges whether or not a case corresponding to the report has already been stored. For example, if the distance between the shooting position and the case position is equal to or less than a predetermined threshold, the control unit 11 judges that the case corresponds to the report information. If the control unit 11 judges that the report is new (YES in step S5), it performs a case registration process (step S13) and moves the process to step S8. If the control unit 11 judges that the report is not new (NO in step S5), it acquires the case status from the handling status column of the case DB 133 (step S6). The control unit 11 creates a response (step S7). The control unit 11 acquires a message according to the case status from the message DB 135. For example, if the case status is "in progress", it acquires a message saying "This appears to be the same as your report / request. This is being handled." Also, if the case status is "Waiting" (waiting for a change in the pothole status), a message will be obtained saying "Please report again in one month." In this case, a command may be created in the resident terminal 2 to activate the reminder function in one month.The control unit 11 sends a response including a message and setting information to the resident terminal 2 (step S8). The control unit 21 of the resident terminal 2 receives the response (step S9). The control unit 21 determines whether the response includes a command (step S10). If the control unit 21 determines that the response includes a command (YES in step S10), it executes the command (step S11). For example, if the command is to set a reminder, it stores the date and time of execution and the settings for the message to be displayed. If the control unit 21 determines that the response does not include a command (NO in step S10), it moves the process to step S12. The control unit 21 displays the message included in the response on the display panel 25 (step S12) and ends the process.

[0040] FIG. 11 is a flowchart showing an example of the procedure of the case registration process. The case registration process corresponds to step S13 in FIG. 10. The control unit 11 of the server 1 identifies the administrator of the road that is the subject of the report (step S21). For example, the control unit 11 refers to an electronic map based on the geographic coordinate value of the shooting position obtained from the Exif information attached to the road image, and acquires the road number, road name, address, etc. This information is compared with the contents stored in the management section column of the road administrator DB 131, and the road administrator is identified. The control unit 11 creates a notice addressed to the road administrator (step S22). The control unit 11 identifies the category of the report from the contents of the road image identified by the image recognition technology and the text entered by the reporter. The control unit 11 acquires advice according to the identified category from the advice DB 134. For example, if the category is identified as "road surface>pothole>small" from the road image, etc., the control unit 11 acquires the advice "This is not the size that needs repair. Recheck in (months)." A number corresponding to the size of the recognized pothole is substituted for "()". The control unit 11 creates a notification including report information including a road image and advice. The control unit 11 transmits the created notification to the manager terminal 3 (step S23). The control unit 31 of the manager terminal 3 receives the notification (step S24). The control unit 31 displays the notification on the display unit 36 ​​(step S25). The road manager checks whether the location that is the subject of the report is within the road section that the road manager manages. If it is within the location, the road manager checks whether the assigned category is appropriate, whether immediate action is required, and so on. The road manager inputs the confirmation result to the manager terminal 3. The confirmation result also includes a response as to whether or not to accept the case that is started by the report as its own case. The control unit 31 accepts the confirmation result (step S26). The control unit 31 transmits the confirmation result to the server 1 (step S27). The control unit 11 of the server 1 receives the confirmation result (step S28). The control unit 11 refers to the confirmation result and judges whether the road administrator has accepted the case (step S29). If the control unit 11 judges that the case has not been accepted (NO in step S29), the process returns to step S21 and starts again from identifying the road administrator.When the control unit 11 determines that the call has been accepted (YES in step S29), it creates a response to the caller (step S30) and returns the process to the caller.

[0041] As described above, the communication system 100 can attach advice to a report about a road and transmit it to the road administrator.

[0042] (Reminder function) The remind function is a function that requests the reporter to report again by a push notification. FIG. 12 is an explanatory diagram showing an example of a notification setting DB. The notification setting DB 136 includes a case ID column, an administrator ID column, a sending date and time column, a message column, a device token column, and a report ID column. The case ID column stores the case ID. The administrator ID column stores the ID of the road administrator. The sending date and time column stores the date and time when the push notification is sent. The message column stores the message to be sent. The device token column stores the device token 231 that identifies the destination of the push notification. The report ID column stores the report ID. For example, if the report category is "road surface>pothole>small" and it is determined that the condition should be checked again in one month, a new record is created in the notification setting DB 136 to send a push notification in one month.

[0043] FIG. 13 is a flowchart showing an example of the procedure of the push notification request process. The push notification request process is a process of requesting the push notification service 4 to send a push notification. The push notification request process is periodically and repeatedly executed by a daily batch or the like. The control unit 11 of the server 1 acquires a notification setting from the notification setting DB 136 (step S41). The control unit 11 selects a notification setting to be processed (step S42). The control unit 11 judges whether the sending date and time is today or not (step S43). If the control unit 11 judges that the sending date and time is not today (NO in step S43), the process proceeds to step S45. If the control unit 11 judges that the sending date and time is today (YES in step S43), the control unit 11 transmits a request including the device token 231, a message (report promotion information), and the sending date and time to the push notification service 4 (step S44). The control unit 11 judges whether there is an unprocessed notification setting (step S45). When the control unit 11 determines that there is an unprocessed notification setting (YES in step S45), the control unit 11 returns the process to step S42 and processes the unprocessed notification setting. When the control unit 11 determines that there is no unprocessed notification setting (NO in step S45), the control unit 11 ends the process.

[0044] By using the remind function to request the reporter of a case under progress monitoring to report again by push notification, the road manager can grasp the current situation from the reporter's re-report. Push notifications can be sent if the device token 231 is obtained from the resident terminal 2, so it is possible to send reminder notifications to reporters without collecting personal information such as email addresses or mobile phone numbers.

[0045] (Local notification function) The local notification function is a function that uses local push notification to notify the resident terminal 2. Figure 14 is an explanatory diagram showing an example of a message DB stored in the resident terminal. The message DB 232 is a database that is referenced when the local notification function operates on the resident terminal 2. The message DB 232 includes an ID column, a type column, a condition column, and a message column. The ID column stores an ID that can uniquely identify a message. The type column stores the type of message. The condition column stores the conditions for displaying a message. The message column stores the message to be displayed.

[0046] FIG. 15 is an explanatory diagram showing an example of a notification DB. The notification DB 233 stores the settings of a local push notification. The notification DB 233 includes a case ID column, a report ID column, a notification date and time column, a target position column, and an MID column. The case ID column stores a case ID. The report ID column stores a report ID (report number). The notification date and time column stores the date and time when the local push notification is performed. The target position column stores the geographic coordinates of a point linked to the case. The MID column stores the ID of a message to be displayed. A value may not be set in either the notification date and time column or the target position column. When values ​​are set in both the notification date and time column and the target position column, if it is determined that the resident terminal 2 is at or near the target position on the notification date and time, a local push notification is performed. When a value is set in the notification date and time column but not in the target position column, a local push notification is performed on the notification date and time regardless of the location of the resident terminal 2. Regardless of the date and time, if it is determined that the resident terminal 2 is at or near the target location, a local push notification is performed.

[0047] FIG. 16 is a flowchart showing an example of the procedure of the local notification process. The local notification process is a process executed by the resident terminal 2. The control unit 21 refers to the notification DB 233 and determines whether or not there is a notification (step S61). If the control unit 21 determines that there is no notification (NO in step S61), it ends the process. If the control unit 21 determines that there is a notification (YES in step S61), it acquires the notification (step S62). The control unit 21 selects the notification to be processed (step S63). The control unit 21 determines whether or not a date and time is specified in the notification (step S64). If the control unit 21 determines that a date and time is specified in the notification (YES in step S64), it determines whether or not the current date and time corresponds to the notification date and time (step S65). Since the current date and time may not completely match the notification date and time at the timing of the process, for example, if the current time is two minutes before or after the notification date and time, it may be determined that the current time corresponds to the notification date and time. If the control unit 21 determines that the current date and time does not correspond to the notification date and time (NO in step S65), it moves the process to step S70. When the control unit 21 determines that the notification date and time is met (YES in step S65), it determines whether or not the notification specifies a location (step S66). When the control unit 21 determines that the notification does not specify a location (NO in step S66), it advances the process to step S68. When the control unit 21 determines that the notification specifies a location (YES in step S66), it determines whether or not the own location is within the target location range (step S67). The target location range is, for example, the target location or within a radius of 200 meters centered on the target location. The control unit 21 acquires the own location from the location acquisition unit 27. The control unit 21 determines whether or not the own location is within the target location range based on the acquired coordinate values ​​of the own location and the coordinate values ​​of the target location. When the control unit 21 determines that the own location is not within the target location range (NO in step S67), it moves the process to step S70. When the control unit 21 determines that the own location is within the target location range (YES in step S67), it acquires a message to be displayed (step S68). The control unit 21 performs the notification using the local push notification mechanism (step S69). The control unit 21 determines whether or not there is an unprocessed notification (step S70).If the control unit 21 determines that there is an unprocessed notification (YES in step S70), the process returns to step S63, and processes the unprocessed notification. If the control unit 21 determines that there is no unprocessed notification (NO in step S70), the process ends. If the control unit 21 determines that the notification does not specify a date and time (NO in step S64), the control unit 21 determines whether or not the notification specifies a location (step S71). If the control unit 21 determines that the notification does not specify a location (NO in step S71), the process proceeds to step S70. If the control unit 21 determines that the notification specifies a location (YES in step S71), the control unit 21 determines whether or not the control unit's own location is within the target location range (step S72). The determination method is the same as described above. If the control unit 21 determines that the control unit's own location is not within the target location range (NO in step S72), the process proceeds to step S70. When the control unit 21 determines that the local position is within the target position range (YES in step S72), the process proceeds to step S68 and executes the subsequent steps. Note that it is desirable to delete the notification that has been completed from the notification DB 233. The local notification process is repeatedly executed by an interval timer interrupt or the like.

[0048] The local notification function makes it possible to display promotion information such as reacquisition of road images even when the resident terminal 2 is offline and cannot receive a push notification from the push notification service 4.

[0049] (Category determination by learning model) As described above, the server 1 attaches advice when sending a report to a road administrator. In order for the server 1 to select appropriate advice, it is important to determine the category of the report, which is mainly performed using road images. Below, we will explain category determination using a learning model.

[0050] FIG. 17 is an explanatory diagram showing an example of the configuration of a learning model. The learning model 141 is a neural network generated by deep learning using a data set including road images, as well as categories, urgency, and priority as training data. The learning model 141 is configured with a CNN (Convolutional Neural Network), TensorFlow, Vision Transformer, and the like. The learning model 141 is trained to output categories, urgency, and priority when a road image is input. The learning model 141 is not limited to a neural network, and may be a model based on other learning algorithms, such as a decision tree, a random forest, or a support vector machine (SVM).

[0051] FIG. 18 is a flowchart showing an example of a procedure of the learning model generation process. The control unit 11 of the server 1 acquires training data (step S81). The control unit 11 selects one record to be processed from the acquired training data (step S82). The control unit 11 performs learning (step S83). The control unit 11 inputs a road image to the learning model 141, and optimizes parameters such as weights between neurons constituting the learning model 141 so that the category, urgency, and priority output from the learning model 141 become the category, urgency, and priority included in the training data, respectively. The control unit 11 determines whether or not there is unprocessed data in the acquired training data (step S84). If the control unit 11 determines that there is unprocessed data (YES in step S84), the process returns to step S82 and continues. If the control unit 11 determines that there is no unprocessed data (NO in step S84), the control unit 11 stores the optimized parameters of the learning model 141 (step S85) and ends the process.

[0052] The learning model 141 is used when determining the category of a report in step S22 of Fig. 11. By using the learning model 141, the accuracy of determining the category of a report is improved. When the learning model 141 is used, the category, urgency, and priority are obtained, so the advice DB 134 shown in Fig. 8 is changed to a configuration in which the category, urgency, and priority are associated with advice and stored.

[0053] (Guide function) When the reporter captures a road image, it is useful to display information that guides the photographer. In particular, when capturing an image of the same location as a location already reported, if the angle, zoom magnification, etc. are the same as the photo already captured, comparison is easy. FIG. 19 is a flowchart showing an example of the procedure of re-report processing. The reporter who receives a request for re-reporting by push notification performs an input operation to execute the request on the resident terminal 2. The control unit 21 of the resident terminal 2 receives the case ID and report ID of the case associated with the request by the input operation (step S91). The control unit 21 transmits the case ID and report ID to the server 1 (step S92). The control unit 11 of the server 1 receives the case ID and report ID (step S93). The control unit 11 acquires an image associated with the report ID from the report DB 132 (step S94). The control unit 11 transmits the acquired image to the resident terminal 2 (step S95). The control unit 21 of the resident terminal 2 receives the image (step S96). The control unit 21 displays a shooting screen in which the received image and the image captured by the imaging unit 28 in real time are arranged side by side (step S97). The reporter operates the shutter icon to shoot while referring to the displayed image and the real-time image so that both images are similar. The control unit 21 judges whether or not the shot image has been acquired (step S98). If the control unit 21 judges that the shot image has not been acquired (NO in step S98), it executes step S98 repeatedly. If the control unit 21 judges that the shot image has been acquired (YES in step S98), it displays the acquired still image (step S99). The reporter checks the shot image and performs a transmission operation or a retake operation. The control unit 21 judges whether or not to transmit the image (step S100). If the control unit 21 judges that the image should not be transmitted (NO in step S100), it returns the process to step S98. If the control unit 21 judges that the image should be transmitted (YES in step S100), it transmits the shot image to the server 1 (step S101). The control unit 11 of the server 1 receives the image (step S102). The control unit 11 stores the received image (step S103). The control unit 11 transmits a completion message to the resident terminal 2 (step S104). The control unit 21 of the resident terminal 2 receives the completion message (step S105).The control unit 21 displays a completion message (step S106) and ends the process.

[0054] FIG. 20 is an explanatory diagram showing an example of a shooting screen. The shooting screen d01 includes a reference image area d011, a captured image area d012, and a shutter icon d013. The reference image area d011 displays an image captured at the time of the previous report. The captured image area d012 displays an image captured in real time by the imaging unit 28. After a shooting operation, the captured image area d012 displays a captured still image. When the shutter icon d013 is selected, the image displayed in the captured image area d012 is acquired as a still image. The shooting screen d01 is an example of guide information.

[0055] (Time Series Forecasting) As mentioned above, when a report is received that a pothole has been found but repairs are not made immediately, the pothole is monitored. In this case, it is useful to predict future progress. Below, pothole progress prediction using a time series prediction model is explained. FIG. 21 is an explanatory diagram of the progress prediction model. The progress prediction model is a neural network related to LSTM (Long-Short Term Memory). LSTM is a type of RNN (Recurrent Neural Network), and is a neural network that inputs time series data prior to the prediction target time point and outputs a predicted value for the target time point.

[0056] LSTM has an input layer, an intermediate layer, and an output layer. The input layer has a plurality of neurons that respectively receive inputs of images at each time point along the time series. The output layer has neurons that output images. The intermediate layer has neurons for calculating predicted values ​​from input values ​​to each neuron in the input layer. The neurons in the intermediate layer are called LSTM Blocks, and calculate values ​​at the next time point from time series data up to the most recent time point by performing calculations on input values ​​at the next time point using calculation results in the intermediate layer on input values ​​at past times. In FIG. 21, EOS stands for End Of Sequence, and indicates the end of time series data. Note that FIG. 21 shows a Sequence-to-Sequence model, which is an example of a learning model using LSTM, and the present embodiment is not limited to this. For example, the intermediate layer is not limited to one layer, and may be two or more layers. Also, the number of neurons in the input layer and the output layer is not limited to three. Also, the progress prediction model is not limited to LSTM, and may be a neural network other than LSTM.

[0057] The progress prediction model is trained as follows: Multiple sets of time-series images of potholes are prepared. For example, each set of images is taken once a month to show the condition of the pothole. It is desirable for each set of images to include images of the size of the pothole that requires repair. Here, each set is assumed to contain six images.

[0058] The following learning is performed using each set as training data. The control unit 11 of the server 1 selects one data set. The control unit 11 creates learning data with one input layer and five output layers. The control unit 11 performs learning using the created learning data. Next, the control unit 11 creates learning data with two input layers and four output layers, and performs learning. The control unit 11 repeats similar learning. Then, when learning of the five input layers and one output layer is completed, the control unit 11 ends learning using the selected data set, and performs learning using other data sets in the same manner. When learning with all data sets is completed, the control unit 11 ends learning of the progress prediction model.

[0059] 22 is an explanatory diagram showing progress prediction using a progress prediction model. When the control unit 11 of the server 1 receives a report about a pothole, the control unit 11 inputs the received road image into the progress prediction model. The control unit 11 acquires the image output by the progress prediction model. The control unit 11 transmits advice information including the acquired image to the administrator terminal 3.

[0060] FIG. 23 is an explanatory diagram showing an example of a notification screen. The notification screen d02 is a screen displayed on the manager terminal 3 when a new report about a pothole is received. The notification screen d02 includes a photographing position d021, a road image d022, a message d023, a predicted image d024, and advice d025. The photographing position d021 is information about the position where the road image was photographed. The road image d022 is the photographed road image. The message d023 is a message including the priority and urgency. The predicted image d024 is a predicted image generated by a progress prediction model. The advice d025 is advice based on a progress prediction (progress prediction).

[0061] A progress prediction screen created using a progress prediction model is displayed on the notification screen, enabling road managers to create highly accurate work plans.

[0062] FIG. 24 is a flow chart showing an example of the procedure of the status display process. The reporter selects the case display on the resident terminal 2. The control unit 21 of the resident terminal 2 transmits the device token 231 to the server 1 (step S121). The control unit 11 of the server 1 receives the device token 231 (step S122). The control unit 11 creates a case list (step S123). The control unit 11 searches the report DB 132 using the device token 231 as a key, and acquires a report ID. The control unit 11 searches the case DB 133 using the acquired report ID as a key. The control unit 11 creates a list from the search results. The control unit 11 transmits the case list to the resident terminal 2 (step S124). The control unit 21 of the resident terminal 2 receives the case list, and displays it on the display panel 25 (step S125). The reporter selects the case he or she wishes to refer to from the case list. The control unit 21 accepts the selection (step S126). The control unit 21 transmits the case ID of the selected case to the server 1 (step S127). The control unit 11 of the server 1 receives the case ID (step S128). The control unit 11 creates a case status screen showing the status (progress) of the case (step S129). The control unit 11 transmits the case status screen to the resident terminal 2 (step S130). The control unit 21 of the resident terminal 2 receives the case status screen (step S131). The control unit 21 displays the case status screen (step S132) and ends the process.

[0063] Fig. 25 is an explanatory diagram showing an example of a case status screen. The case status screen d03 includes a report type d031, a road image d032, a case position d033, a status d034, and a schedule d035. The report type d031 displays the type. The road image d032 is the latest road image. The case position d033 is the location of the case. The status d034 is the status of the case. The schedule d035 is a future schedule.

[0064] The case status screen allows the reporter to check the progress of the case he or she reported. In addition, by displaying future schedules, it is possible to notify the reporter of requests in advance, if necessary.

[0065] Up to this point, the report on road damage has been described, but the report on road safety may also be made. FIG. 26 is an explanatory diagram showing an example of a report screen. The report screen d04 is a screen for inputting the report contents, and is displayed by the resident terminal 2. The report screen d04 includes a report type d041, a road image d042, a road name field d043, an address field d044, a landmark field d045, a report field d046, a send button d047, and a cancel button d048. The report type d041 selects the report type. The road image d042 displays a road image specified by the reporter. The road name field d043 is a field for inputting the name of the road. The address field d044 is a field for inputting the address of the report position. The landmark field d045 is a field for inputting landmarks that will be near the report position. The report field d046 is a field for inputting the report contents. When the send button d047 is selected, the report information is transmitted to the server 1. When the cancel button d048 is selected, the report screen d04 is closed and the user returns to the previous screen.

[0066] The processing after sending the report information is the same as that of a report about road damage. In this way, reports about road safety can also be made, and it becomes possible to request the implementation of safety measures.

[0067] The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims. In addition, the claims are written in a format in which a claim cites two or more other claims (multiple claim format), but this is not limited to this format. A multiple claim that cites at least one other claim (multi-multi claim) may also be written.

[0068] The following supplementary notes are further disclosed regarding the above-described embodiment.

[0069] (Appendix 1) receiving report information including a road image including a road as a subject, a shooting date and time, and a shooting location; Identifying an administrator of the road based on the photographing position; The notification information and the advice information specified by the road image are transmitted to a manager terminal of the specified manager. A computer program that causes a computer to carry out processing.

[0070] (Appendix 2) If the report information is related to damage, the advice information includes a progress status of the damage or a prediction of the progress of the damage. 2. The computer program of claim 1.

[0071] (Appendix 3) A storage unit that stores the notification information is referred to, and past notification information at a position that is the same as or near the photographing position is searched for; Generate a reply message based on the search results; The generated reply message is transmitted to the reporter terminal that transmitted the report information. 10. A computer program according to claim 1 or 2.

[0072] (Appendix 4) issue a report number associated with the report information, and transmit the report number to the reporter terminal; When the report number is received from the reporter terminal, case information of the case specified by the received report number is transmitted to the reporter terminal. 10. A computer program according to claim 1 or 2.

[0073] (Appendix 5) After a predetermined period of time has elapsed since the photographing date and time, prompting information for prompting the reporter terminal to photograph and transmit the road image again is transmitted to the reporter terminal. 10. A computer program according to claim 1 or 2.

[0074] (Appendix 6) The notification information includes request information regarding safe use of roads. 10. A computer program according to claim 1 or 2.

[0075] (Appendix 7) A road image included in the report information is input to a trained model that outputs a case type when a road image is input; Identifying the advice information based on the type output by the trained model 10. A computer program according to claim 1 or 2.

[0076] (Appendix 8) Transmitting report information including a road image including a road as a subject, a shooting date and time, and a shooting position; Receive reporting promotion information including guide information about shooting, Display reporting promotion information Terminal program.

[0077] (Appendix 9) The guide information includes a shooting direction, a shooting angle, a time, or weather information. 9. The terminal program according to claim 8.

[0078] (Appendix 10) The notification prompting information includes a date and time of a request for photography, and when the date and time of the request for photography is approaching, a request message is displayed. 10. The terminal program according to claim 9.

[0079] (Appendix 11) Get location information, If the distance calculated based on the acquired location information and the location information included in the notification prompting information is equal to or less than a predetermined threshold, a request message is displayed. 11. The terminal program according to any one of claims 8 to 10.

[0080] (Appendix 12) a receiving unit that receives notification information including a road image including a road as a subject, a shooting date and time, and a shooting position; An identification unit that identifies an administrator of the road based on the photographing position; a transmission unit that transmits the notification information and the advice information identified by the road image to an administrator terminal of the identified administrator; An information processing device comprising:

[0081] (Appendix 13) receiving notification information including a road image including a road as a subject, a shooting date and time, and a shooting position; Identifying an administrator of the road based on the photographing position; The notification information and the advice information specified by the road image are transmitted to a manager terminal of the specified manager. An information processing method in which processing is performed by a computer. [Explanation of symbols]

[0082] 100: Contact system 1: Server 11: Control section 12: Main memory 13: Auxiliary storage section 131: Road administrator DB 132:Report DB 133:Project DB 134: Advice DB 135: Message DB 136: Notification settings DB 141: Learning model 15: Communications Department 16: Reading section 1P: Control program 1a: Portable storage medium 1b: Semiconductor memory 2: Resident terminal 21: Control section 22: Main memory 23: Auxiliary storage section 231: Device Token 232: Message DB 233: Notification DB 24: Communications Department 25: Display panel 26:Operation unit 27: Position acquisition section 28: Imaging unit 2P: Control program 3: Administrator terminal 31: Control section 32: Main memory 33:Auxiliary storage section 34: Communications Department 35: Input section 36:Display section 3P: Control program 4: Push notification service B: Bus N: Network

Claims

1. receiving notification information including a road image including a road as a subject, a shooting date and time, and a shooting position; Identifying an administrator of the road based on the photographing position; The notification information and the advice information specified by the road image are transmitted to a manager terminal of the specified manager. A computer program that causes a computer to carry out processing.

2. If the report information is related to damage, the advice information includes a progress status of the damage or a prediction of the progress of the damage.

2. The computer program product of claim 1.

3. A storage unit that stores the notification information is referred to, and past notification information at a position that is the same as or near the photographing position is searched for; Generate a reply message based on the search results; The generated reply message is transmitted to the reporter terminal that transmitted the report information.

3. A computer program according to claim 1 or 2.

4. Issue a report number associated with the report information; Transmitting the report number to the reporter terminal; When the report number is received from the reporter terminal, case information of the case specified by the received report number is transmitted to the reporter terminal.

3. A computer program according to claim 1 or 2.

5. After a predetermined period of time has elapsed since the photographing date and time, prompting information for prompting the reporter to photograph and transmit the road image again is transmitted to the reporter terminal.

3. A computer program according to claim 1 or 2.

6. Transmitting report information including a road image including a road as a subject, a shooting date and time, and a shooting position; receiving notification prompting information including guide information for shooting in relation to the notification information; Display the received notification prompting information Terminal program.

7. a receiving unit that receives notification information including a road image including a road as a subject, a shooting date and time, and a shooting position; An identification unit that identifies an administrator of the road based on the photographing position; a transmission unit that transmits the notification information and the advice information identified by the road image to an administrator terminal of the identified administrator; An information processing device comprising:

8. receiving notification information including a road image including a road as a subject, a shooting date and time, and a shooting position; Identifying an administrator of the road based on the photographing position; The notification information and the advice information specified by the road image are transmitted to a manager terminal of the specified manager. An information processing method in which processing is performed by a computer.

Citation Information

Patent Citations

  • Road maintenance management system, road maintenance management method and computer program

    JP7023690B2