Method and electronic equipment for detecting potential traffic safety hazards

Abstract

The invention discloses a method for detecting potential traffic safety hazards and electronic equipment, and the method comprises the steps: determining a troubleshooting task, and determining a target troubleshooting scene according to the troubleshooting task; wherein the troubleshooting task indicates to troubleshoot the hidden danger condition of the set road section; determining N troubleshooting items corresponding to the target troubleshooting scene according to a corresponding relationship between the task scene and the troubleshooting items; wherein each troubleshooting item corresponds to one hidden danger item, and each troubleshooting item comprises at least one troubleshooting element; obtaining answers, corresponding to the questions associated with the N investigation items, of the user; m hidden danger items are determined according to the answers; wherein M is smaller than or equal to N; and for each hidden danger item, acquiring acquisition data associated with the hidden danger item, and verifying the validity of the hidden danger item based on the acquisition data. And the troubleshooting efficiency and the troubleshooting comprehensiveness are improved.

Description

Technical Field

[0001] This application relates to the technical field of large language models, and particularly to a method and an electronic device for detecting traffic safety hazards. Background Art

[0002] With the rapid construction of road infrastructure, the incidence of road traffic safety accidents shows an increasing trend year by year. Therefore, the investigation of road traffic safety hazards has received more and more attention. Usually, road safety hazards involve multiple aspects such as signs and markings, road network structure, facilities and equipment, etc., with a large quantity, wide range, and strong professionalism.

[0003] In the related art, the investigation of road traffic safety hazards is mainly achieved through on-site manual investigation or video analysis technology. During the on-site investigation process, due to the large amount of content involved and complex standards and specifications, high professionalism and experience are required for the investigators, and the efficiency and effect of the investigation are generally not ideal. In addition, when identifying through video analysis technology, obvious hazards such as sign occlusion and unclear markings can be identified, but there are still a large number of hidden hazards (such as poor sight distance, inconsistent sign and marking information, etc.), and at this time, on-site manual investigation is still required to discover them.

[0004] Therefore, the existing investigation methods have poor comprehensiveness and low efficiency. Summary of the Invention

[0005] An exemplary embodiment of this application provides a method and an electronic device for detecting traffic safety hazards to improve the investigation efficiency and comprehensiveness.

[0006] According to the first aspect of the exemplary embodiment, a method for detecting traffic safety hazards is provided, including:

[0007] Determine an investigation task, and determine a target investigation scenario according to the investigation task; wherein, the investigation task indicates the hazard situation of a set section to be investigated;

[0008] According to the corresponding relationship between the task scenario and the investigation items, determine N investigation items corresponding to the target investigation scenario; wherein, each investigation item corresponds to a hazard item, and each investigation item includes at least one investigation element;

[0009] Obtain the answers corresponding to each of the various questions associated with the N investigation items from the user;

[0010] Determine M hazard items according to each answer; wherein, M is less than or equal to N;

[0011] For each hazard item, obtain the collected data associated with the hazard item, and verify the effectiveness of the hazard item based on the collected data.

[0012] In an embodiment of the present application, after determining the troubleshooting task, first determine the target troubleshooting scenario to which it belongs, and then determine N troubleshooting items corresponding to the target troubleshooting scenario according to the pre-set correspondence between the task scenario and the troubleshooting items. Among them, each troubleshooting item corresponds to a potential hazard item, and each troubleshooting item includes at least one troubleshooting element. Then, obtain the answers corresponding to each of the various questions associated with the N troubleshooting items from the user, and determine M potential hazard items based on each answer. In order to further determine the potential hazard items, for each potential hazard item, the collected data associated with the potential hazard item can be obtained, and the validity of the potential hazard item can be verified based on the collected data. Such a design, compared with manual troubleshooting or fixed-process troubleshooting in the prior art, takes into account the troubleshooting scenario, and then uses the Q&A information associated with the troubleshooting items in this scenario to confirm the potential hazard items, making the troubleshooting more comprehensive and more efficient.

[0013] In an alternative embodiment, determining the troubleshooting task includes:

[0014] Receiving the troubleshooting task sent by the task distribution system; wherein, the troubleshooting task includes the task location; or

[0015] Receiving the troubleshooting task from the user's mobile terminal; wherein, the troubleshooting task includes the positioning information.

[0016] In the above embodiment, the troubleshooting task can be sent by the system or generated by the user (troubleshooter) triggering a corresponding operation. The sources of the troubleshooters are different, their contents are different, and thus the methods for determining the target troubleshooting scenario to which they belong are different.

[0017] In an alternative embodiment, determining N troubleshooting items corresponding to the target troubleshooting scenario according to the correspondence between the task scenario and the troubleshooting items includes:

[0018] Determining the knowledge graph representing the correspondence between the task scenario and the troubleshooting items;

[0019] According to the task location or positioning information in the troubleshooting task, determine the target troubleshooting scenario of the troubleshooting task;

[0020] Traverse the knowledge graph according to the target troubleshooting scenario to determine N troubleshooting items corresponding to the target troubleshooting scenario.

[0021] In the above embodiment, the correspondence between the task scenario and the troubleshooting items is represented by a structured knowledge graph. In this way, after determining the target troubleshooting scenario of the troubleshooting task, based on the target troubleshooting scenario, traverse the knowledge graph, and then N troubleshooting items matching the target troubleshooting scenario can be determined. Such a method can further ensure the comprehensiveness of the troubleshooting and not miss any troubleshooting items.

[0022] In an alternative embodiment, determining the knowledge graph representing the correspondence between the task scenario and the troubleshooting items includes:

[0023] Obtain reference data; wherein, the reference data includes hidden danger judgment specification data, hidden danger judgment guidance data, and historical hidden danger investigation cases;

[0024] Analyze the reference data to determine a hidden danger item knowledge table; wherein, the hidden danger item knowledge table includes the investigation elements, judgment conditions, and corresponding hidden danger item types of each investigation item; the investigation elements include some or all of the scenario, dependent data, and prerequisite conditions;

[0025] Convert the hidden danger item knowledge table into a knowledge graph.

[0026] In the above embodiment, since the knowledge graph is more conducive to traversal, after obtaining the reference data, the reference data is analyzed, and the analyzed data constitutes a hidden danger item knowledge table. Then, according to the knowledge graph technology, the hidden danger item knowledge table is converted into a knowledge graph to facilitate the determination of the investigation items that match the target investigation scenario.

[0027] In an alternative embodiment, the method further includes:

[0028] Input N investigation items into a set large language model to determine an investigation plan;

[0029] Wherein, the investigation plan includes multiple pieces of investigation information, and each piece of investigation information includes whether the problem condition is met, the problem content, and some or all of the investigation elements involved in the problem.

[0030] In the above embodiment, when determining the hidden danger, it depends on the answer of the knowledge-based question answering system. For the questions about the target investigation scenario in the knowledge-based question answering system, the N investigation plans of the target investigation scenario can be output to the set large language model, and the output of the set large language model is the investigation plan. The investigation plan not only includes the problem content but also whether the problem condition is met and the investigation elements involved in the problem. These questions can be output through an electronic device, and then the answers of the user to these questions can be obtained to further determine the hidden danger items according to these answers.

[0031] In an alternative embodiment, obtaining the answers corresponding to each of the various questions associated with the N investigation items from the user includes:

[0032] Obtain the feedback information of the user for the first question; wherein, the first question is any one of the questions in the investigation plan;

[0033] If the feedback information is the first answer to the first question, determine the first answer as the answer to the first question;

[0034] If the feedback information is a second question derived from the first question, input the second question into the set database and obtain the answer to the second question from the set database;

[0035] Obtain the answer to the first question determined by the user based on the answer to the second question.

[0036] In the above embodiments, in one case, the user's feedback information is the answer to the corresponding question. In another case, the user's feedback information is a new question derived from the corresponding question. In the second case, in order to ensure the accuracy of the obtained answer, the answer to the new question can be provided first to facilitate the user to further determine the answer to the original question based on the answer to the new question, so that the obtained answer to the original question is more accurate.

[0037] In an alternative embodiment, the method further includes:

[0038] If the second question is a knowledge quiz question, determine the form of the answer to the second question according to the characteristics of the second question.

[0039] In the above embodiments, when the second question is a knowledge quiz question, the form of the answer to the second question can be determined according to the characteristics of the second question, such as a table or an icon, etc. This can facilitate the user to more intuitively understand the answer to the second question, and thus better assist in answering the first question.

[0040] In an alternative embodiment, after obtaining the user's feedback information on the first question, the method further includes:

[0041] If the feedback information is the first answer to the third question, set the status of the third question in the troubleshooting plan to completed; where the question is any question other than the first question in the troubleshooting plan.

[0042] In the above embodiments, for the case where the answer is not the answer to the current question but the answer to other questions in the troubleshooting plan, the status of the question can be updated in a timely manner to avoid asking the user to answer duplicate questions.

[0043] According to the second aspect of the exemplary embodiment, there is provided an electronic device including a processor and a memory;

[0044] The memory is configured to execute:

[0045] Store the correspondence between the task scenario and the troubleshooting items;

[0046] The processor is configured to execute:

[0047] Determine the troubleshooting task and determine the target troubleshooting scenario according to the troubleshooting task; where the troubleshooting task indicates the hidden danger situation of the set section to be troubleshot;

[0048] According to the correspondence between the task scenario and the troubleshooting items, determine N troubleshooting items corresponding to the target troubleshooting scenario; where each troubleshooting item corresponds to a hidden danger item, and each troubleshooting item includes at least one troubleshooting element.

[0049] Obtain the answers corresponding to each of the various problems associated with N investigation items for the user;

[0050] Determine M potential hazard items based on each answer; where M is less than or equal to N;

[0051] For each potential hazard item, obtain the collection data associated with the potential hazard item, and verify the effectiveness of the potential hazard item based on the collection data.

[0052] According to the third aspect in the exemplary embodiment, a device for detecting traffic safety hazards is provided, including:

[0053] A processing unit, configured to: determine an investigation task, and determine a target investigation scenario according to the investigation task; where the investigation task indicates the hazard situation of a set section to be investigated;

[0054] The processing unit is further configured to: determine N investigation items corresponding to the target investigation scenario according to the correspondence between the task scenario and the investigation items; where each investigation item corresponds to a potential hazard item, and each investigation item includes at least one investigation element;

[0055] A transmission unit, configured to: obtain the answers corresponding to each of the various problems associated with N investigation items for the user;

[0056] The processing unit is further configured to: determine M potential hazard items based on each answer; where M is less than or equal to N;

[0057] The transmission unit is further configured to: for each potential hazard item, obtain the collection data associated with the potential hazard item, and verify the effectiveness of the potential hazard item based on the collection data.

[0058] According to the fourth aspect in the exemplary embodiment, a computer storage medium is provided, in which computer program instructions are stored, and when the instructions run on a computer, the computer is caused to execute the method for detecting traffic safety hazards as in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0060] Figure 1 An exemplary schematic diagram of a fork provided by an embodiment of the present application is shown;

[0061] Figure 2The flowchart of a method for detecting potential traffic safety hazards provided by an embodiment of the present application is exemplarily shown;

[0062] Figure 3 The flowchart of a method for determining N investigation items corresponding to a target investigation scenario provided by an embodiment of the present application is exemplarily shown;

[0063] Figure 4 The flowchart of a method for constructing a knowledge graph provided by an embodiment of the present application is exemplarily shown;

[0064] Figure 5 The schematic diagram of a knowledge graph provided by an embodiment of the present application is exemplarily shown;

[0065] Figure 6 The flowchart of a method for determining the answer to a question provided by an embodiment of the present application is exemplarily shown;

[0066] Figure 7 The schematic diagram of a multi-agent collaborative investigation guidance dialogue system provided by an embodiment of the present application is exemplarily shown;

[0067] Figure 8 The flowchart of the SOP for task investigation in a single intersection scenario provided by an embodiment of the present application is exemplarily shown;

[0068] Figure 9 The structural schematic diagram of a device for detecting potential traffic safety hazards provided by an embodiment of the present application is exemplarily shown;

[0069] Figure 10 The structural schematic diagram of an electronic device provided by an embodiment of the present application is exemplarily shown. Detailed implementation manners

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

[0071] For the convenience of understanding, the nouns involved in the embodiments of the present application are explained below:

[0072] (1) Large Language Model (LLM), which is mainly used for natural language processing tasks, such as chatbots, automatic writing, machine translation, etc. It can also be used in fields such as content generation, question-and-answer systems, text classification, sentiment analysis, and machine translation.

[0073] (2) An agent is an intelligent system that can autonomously sense the environment, make decisions, and execute actions. It can not only perform text processing but also execute tasks in physical or virtual environments. The core capabilities of an agent include perception, decision-making, action, long-term planning, and learning, usually combining multiple technologies such as deep learning and reinforcement learning. An agent has characteristics such as autonomy, reactivity, initiative, and sociality, and can operate and make decisions without direct human intervention.

[0074] Agents can also be applied to a wider range of scenarios, including automated systems (such as smart home control systems, autonomous vehicles), game AI, customer service (such as intelligent customer service systems), industrial automation (such as robotic arms in factories), etc. Agents can not only answer questions but also assist in solving practical problems and execute complex manufacturing tasks.

[0075] In the embodiments of this application, both large language models and agents are applied to the field of traffic road safety.

[0076] Road safety hazards involve multiple aspects such as signs and markings, road network structure, facilities and equipment, etc., with a large quantity, wide coverage, and strong professionalism.

[0077] In related technologies, there is a relative lack of intelligent means for on-site inspection of road traffic safety hazards, mainly achieved through manual on-site inspection or video analysis technology. During the on-site inspection process, due to the large amount of content involved and complex standards and specifications, high professionalism and experience are required for the inspection personnel, and the efficiency and effect of the inspection are generally not ideal. In addition, when identifying through video analysis technology, obvious hazards such as sign occlusion and unclear markings can be identified, but there are still a large number of hidden hazards (such as poor visibility, inconsistent sign and marking information, etc.), and at this time, manual on-site inspection is still required to discover them. In addition, in related technologies, the standardized inspection methods with fixed processes lack sufficient autonomy and flexibility, have poor adaptability in complex combined scenarios, and low inspection efficiency.

[0078] Therefore, the embodiments of this application provide a scenario-based method for inspecting road traffic safety hazards based on large language models and agents, constructing a knowledge system for inspecting road traffic safety hazards, and forming a standardized inspection guidance process for road traffic safety hazards accordingly. Based on the large language model, targeted inspection steps are dynamically planned and generated according to specific scenarios to form a hazard inspection dialogue guidance system, and the multi-agent collaboration technology is used to flexibly handle process deviations, overall ensuring the controllability and autonomy of the inspection process and facilitating practical applications. Automatically conduct a comprehensive inspection of traffic safety hazards. By combining large language model and agent technologies, in the form of natural dialogue, guide the inspection personnel to complete the inspection work step by step, improving the comprehensiveness and efficiency of the inspection.

[0079] After introducing the design concept of the embodiments of the present application, the following briefly introduces the application scenarios applicable to the technical solutions of the embodiments of the present application. It should be noted that the application scenarios introduced below are only for illustrating the embodiments of the present application rather than limiting them. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0080] Reference Figure 1 , a schematic diagram of a fork is shown. Referring to Figure 1 , if there is no signpost, the driver may not be able to notice in time that this is a fork.

[0081] To further illustrate the technical solutions provided by the embodiments of the present application, the following will describe them in detail in combination with the accompanying drawings and specific implementation manners. Although the embodiments of the present application provide method operation steps as shown in the following embodiments or drawings, in the method, more or fewer operation steps may be included based on routine or non-creative labor. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of the present application.

[0082] The following will combine Figure 1 the application scenario shown, referring to Figure 2 the flowchart of a method for detecting traffic safety hazards shown, to illustrate the technical solutions provided by the embodiments of the present application.

[0083] S201: Determine the investigation task, and determine the target investigation scenario according to the investigation task.

[0084] S202: Determine N investigation items corresponding to the target investigation scenario according to the corresponding relationship between the task scenario and the investigation items.

[0085] Among them, each investigation item corresponds to a potential hazard item, and each investigation item includes at least one investigation element.

[0086] S203: Obtain the answers corresponding to each of the various questions associated with the N investigation items from the user.

[0087] S204: Determine M potential hazard items according to each answer.

[0088] Among them, M is less than or equal to N.

[0089] S205: For each potential hazard item, obtain the collected data associated with the potential hazard item, and verify the effectiveness of the potential hazard item based on the collected data.

[0090] In the embodiment of the present application, after determining the investigation task, first determine the target investigation scenario to which it belongs, and then determine N investigation items corresponding to the target investigation scenario according to the pre-set correspondence between the task scenario and the investigation items. Among them, each investigation item corresponds to a potential hazard item, and each investigation item includes at least one investigation element. Then, obtain the answers corresponding to each of the various problems associated with the N investigation items from the user, and determine M potential hazard items based on each answer. In order to further determine the potential hazard items, for each potential hazard item, the collected data associated with the potential hazard item can be obtained, and the validity of the potential hazard item can be verified based on the collected data. Such a design, compared with manual investigation or fixed process investigation in the prior art, takes into account the investigation scenario, and then uses the Q&A information associated with the investigation items in this scenario to confirm the potential hazard items, making the investigation more comprehensive and the investigation efficiency higher.

[0091] Regarding S201, the investigation task is used to indicate the potential hazard situation of the set investigation section. The investigation task usually can also include road network basic data, such as information on the number of lanes and road level. There are two ways for the electronic device to determine the investigation task. One is that the background system issues the investigation task to the electronic device, and the electronic device receives the investigation task. The task content carried in the investigation task can include the task executor, task location, etc. Another way is that the investigator carries a mobile terminal to the section to be investigated, generates an investigation task by operating the mobile terminal, and sends the investigation task to the electronic device. In this case, the investigation task can carry positioning information. Therefore, the electronic device can determine the target investigation scenario corresponding to the investigation task according to the task location or the positioning information. Optionally, the target investigation scenario can be any one of a plane intersection, a steep slope section, a continuous downhill section, a sharp bend section, a combination of bend and slope, a section around a school, a section passing through a village or town, a section adjacent to water, a section adjacent to a cliff, a section with frequent accidents, a tunnel entrance and exit. In actual applications, an investigation task can be a single scenario or a combination of multiple scenarios. Here, an example is used to illustrate.

[0092] Regarding S202, since there are multiple task scenarios in the actual situation, therefore, the correspondence between the task scenario and the investigation items can be established in advance. Among them, each investigation item corresponds to a potential hazard item, and each investigation item includes at least one investigation element. Potential hazard items are, for example, unchannelized large plane intersections, lack of a central isolation guardrail, etc. Investigation elements are, for example, highway grade, number of lanes, speed limit value, etc. Therefore, after determining the target investigation scenario, N investigation items corresponding to the target investigation scenario can be determined according to this correspondence. Optionally, the process of determining N investigation items corresponding to the target investigation scenario can be achieved through Figure 3 the steps S201-1 to S201-3 in

[0093] S201-1: Determine the knowledge graph for representing the correspondence between the task scenario and the investigation items.

[0094] Optionally, to determine the knowledge graph, first determine the hidden danger item knowledge table, and then convert the hidden danger item knowledge table into a knowledge graph. This process can be implemented through Figure 4 steps S201-1-1 to S201-1-3 in

[0095] S201-1-1: Obtain reference data.

[0096] Optionally, the reference data includes hidden danger judgment specification data, hidden danger judgment guidance data, and historical hidden danger investigation cases. Among them, the hidden danger judgment specification data can be the specifications formulated by relevant departments for judging hidden danger items, the hidden danger judgment guidance data is the data suggesting what kind of situations may have hidden danger situations, and the historical hidden danger investigation cases are the historical hidden danger situation cases.

[0097] S201-1-2: Analyze the reference data to determine the hidden danger item knowledge table.

[0098] Optionally, analyze the reference data, extract the involved scenarios, dependent data, preconditions, judgment conditions, hidden danger item types, etc., and organize these data to obtain the hidden danger item knowledge table. Among them, the hidden danger item knowledge table includes the investigation elements, judgment conditions, and corresponding hidden danger item types of each investigation item; the investigation elements include some or all of the scenarios, dependent data, and preconditions.

[0099] Table 1 shows a hidden danger item knowledge table for traffic safety provided by an embodiment of the present application. Table 1 shows two scenarios: plane intersections and passing through villages and towns. Among them, the plane intersection scenario includes three investigation items (channelization investigation, sight distance investigation, and angle investigation), and the passing through villages and towns scenario includes two investigation items (installation of a central isolation guardrail and opening of a central isolation guardrail).

[0100] Optionally, each investigation item includes at least one investigation element. For example, in the investigation item for channelization, the dependent data in the investigation element is the highway grade and the number of lanes; in the investigation item for sight distance, the dependent data in the investigation element is the highway grade and the speed limit value; in the investigation item for angle, the dependent data in the investigation element is the road network data of the intersecting sections.

[0101] Table 1 A hidden danger item knowledge table for traffic safety

[0102]

[0103]

[0104] S201-1-3: Convert the hidden danger item knowledge table into a knowledge graph.

[0105] Optionally, for semi-structured tabular data, graph knowledge extraction techniques can be used to extract and form a structured hidden danger item knowledge graph, including several categories of entities such as dependent data, preconditions, judgment rules, collected content, and recommended measures. Specifically, the scenarios, dependent data, Yin Huangxiang, etc. in Table 1 are extracted and converted into corresponding graph entities. For the two columns of judgment rules and preconditions that are more textually described, key information is extracted using NLP techniques and converted into computable expressions to facilitate the subsequent generation of the knowledge graph. Among them, the expression can be: Rule A = (highway grade ≥ 2 & number of lanes > 4). Figure 5 This is a schematic diagram of a knowledge graph provided by an embodiment of the present application.

[0106] S201-2: According to the task location or positioning information in the inspection task, determine the target inspection scenario of the inspection task.

[0107] In one case, the system issues an inspection task to the electronic device. In this case, the inspection information of the inspection task includes the task location. In this way, the target inspection scenario can be directly determined by comparing the location with the set inspection scenarios. In another case, the user (inspection personnel) arrives at the location to be inspected and generates an inspection task by operating the mobile terminal. In this case, the inspection information of the inspection task includes the positioning information of the mobile terminal. In this way, the target inspection scenario can be directly determined by comparing the positioning information with the set inspection scenarios.

[0108] S201-3: Traverse the knowledge graph according to the target inspection scenario to determine N inspection items corresponding to the target inspection scenario.

[0109] Optionally, since the knowledge graph is a structured representation of multiple inspection scenarios and their corresponding inspection items, therefore, according to the target inspection scenario, the knowledge graph can be traversed to retrieve subgraphs that meet the target inspection scenario in the knowledge graph. The inspection items of these subgraphs constitute N inspection items corresponding to the target inspection scenario. Combining Table 1, if the target inspection scenario is the plane intersection scenario, then N is 3, and the 3 inspection items are the channelization inspection, sight distance inspection, and angle inspection corresponding to serial numbers 1, 2, and 3 in Table 1 respectively.

[0110] Regarding S203, obtain the answers corresponding to each question associated with the N inspection items for the user.

[0111] Optionally, when the N inspection items are input to a set large language model, an inspection plan can be determined. The inspection plan includes multiple inspection information, and each piece of inspection information includes whether the problem conditions are met, the problem content, and some or all of the inspection elements involved in the problem.

[0112] In this process, since N troubleshooting items correspond to a subgraph, the subgraph can be converted into a natural language description. By involving the guiding process to construct a prompt, the planning and content generation capabilities of the large language model are utilized to generate a troubleshooting plan.

[0113] An example of constructing a prompt for the guiding process is as follows:

[0114] ##Goal##

[0115] Given the following list of hazard item troubleshooting tasks that are interrelated, extract the dependencies before and after, and independently plan the troubleshooting steps.

[0116] ##Steps##

[0117] 1. Identify the dependencies before and after the tasks in the list

[0118] 2. Extract all the status information to be confirmed based on the dependencies to form a state space

[0119] 3. Independently plan the troubleshooting steps, and each step is formatted as: ("stepN": [pre-condition], <question>,[update-states])

[0120] ##Example##

[0121] Task list:

[0122] - Large plane intersections are not channelized: The judgment rule is that channelization design is required for intersections at or above secondary level (highway grade ≥ 2 & channelization status = not channelized). The dependent status data is the highway grade, and the content to be collected is on-site photos

[0123] - Poor sight distance at intersections: The prerequisite is that there are no signal lights installed at the intersection (whether there are signal lights = false). The judgment rule is that within the through vision triangle formed by the respective stopping sight distances between intersecting highways, the driver's line of sight is blocked by buildings such as houses, mountains, green plants such as trees, signboards or other obstacles, and the intersection point and the traffic conditions on the intersecting roads cannot be seen. The dependent status data is whether there are signal lights, and the content to be collected is intersection photos and obstacles

[0124] -......

[0125] Output: Set the output of the large language model

[0126] "step1": [Number of lanes ≥ 4], "What is the technical grade of this highway?", [Highway grade]

[0127] "step2": [Highway grade ≥ 2], "Is there channelization design at the intersection?", [Whether channelized]

[0128] "step3": [There are speed limit requirements], "Is there a signal light installed at this intersection?", [Whether there are signal lights]

[0129] "step4": [Whether there are signal lights = false] "Is the sight distance at the intersection good?", [Guardrail opening status]

[0130] "step5": [Install a central isolation guardrail], "Are there too many openings in the central isolation guardrail?", [Guardrail opening status] ......

[0132] From this, a troubleshooting plan can be obtained according to the output of the set large language model, that is, part or all of the problem conditions, problem content, and troubleshooting elements involved in the problem. In this example, it is the troubleshooting plan composed of step1-step5

[0133] Taking one of the problems in the troubleshooting plan as an example, through Figure 6 Steps S203-1 to S203-4 in it are used to illustrate the process of determining the answer to a problem:

[0134] S203-1: Obtain the feedback information of the user on the first question.

[0135] Among them, an agent is set in the electronic device. The agent includes a user agent (responsible for receiving user requests, using the LLM, and centrally coordinating each agent for task planning and response generation), a real-time task planning agent (responsible for planning the next task action according to the current state space), and a knowledge Q&A agent (responsible for providing professional knowledge during the troubleshooting process, sorting out relevant standards and regulations, and converting relevant hidden danger professional knowledge into vector data through knowledge embedding). These agents are respectively responsible for different functions and constitute a troubleshooting guidance dialogue system.

[0136] In a specific example, the prompt example of the real-time task planning agent is as follows:

[0137] ##Goal##

[0138] As an AI chatbot, your task is to confirm step by step according to the process and ensure that each step is accurate.

[0139] ##Steps##

[0140] The hidden danger problem recognition process includes:

[0141] - Obtain the output of the list of hidden danger items S02 that need to be confirmed according to the road section type

[0142] - Traverse and inquire about the list of hidden danger items in order, and ask for confirmation for each item whether there is such a problem until the reply is completed

[0143] - Judge whether there is a hidden danger according to the answer. If there is a hidden danger, suspend the confirmation of the hidden danger item, perform the data collection step, and continue the above hidden danger item confirmation after the collection is completed.

[0144] - The hidden danger item data collection step is: query the data collection method list of the hidden danger item, and upload information according to the requirements for each item

[0145] - End process: Store the information of the confirmed hidden danger problem list. Note that the reply style only shows the unconfirmed items. The style of the hidden danger item confirmation reply is as follows:

[0146] ```

[0147] Hidden danger problem list

[0148] 1. xx (unconfirmed)

[0149] 2. xx (unconfirmed)

[0150] Is there such a problem: **1. xx** Please answer whether there is such a hidden danger? (**Yes** or **No**)```

[0151] The reply style for collecting data on hidden danger items is as follows:

[0152] Please collect the data of xxxx.

[0153] The electronic device can play the first question through voice and obtain the feedback information of the user on the first question. For example, the first question is "What is the technical grade of this road?" and the feedback information can be "Grade Four"; another example, the first question is "Is there a channelization design at the intersection?" and the feedback information can be "I don't know what channelization is. I see a stop sign".

[0154] S203-2: If the feedback information is the first answer to the first question, determine that the first answer is the answer to the first question.

[0155] Among them, in the above first example, the feedback information is the answer to the first question.

[0156] S203-3: If the feedback information is the second question derived from the first question, input the second question into the set database and obtain the answer to the second question from the set database.

[0157] Among them, in the above second example, the feedback information is the second question derived from the first question. At this time, input the second question ("I don't know what channelization is. I see a stop sign") into the knowledge Q&A agent, and obtain the answer to the second question through the database in the knowledge Q&A agent. The answer to the second question is, for example, "Okay, I have recorded that there is a stop sign at this intersection. Canalization is a traffic management measure that uses traffic islands...".

[0158] In the actual application process, when the second question is a knowledge Q&A question, the characteristics of the second question are different, and the corresponding answer forms are different. For each output scenario corresponding to each characteristic, the format of the prompt customized output template can be adopted. Next, three scenarios are taken as examples for illustration.

[0159] The first case, the characteristic of the second question represents a brief noun explanation scenario.

[0160] For the brief noun explanation scenario, generally, the content reply is required to be simple and clear, without additional analysis, and directly answer according to the original knowledge base content. Therefore, the output format is defined as:

[0161]

[0162] In the first case, the feature representation list of the second question elaborates the scenario

[0163] For the task planning and process summary scenario, it is generally required to be provided in the form of a list, with not too many items. The content exceeding the limit number of items should be summarized. Therefore, the output format is defined as:

[0164]

[0165] In the first case, the feature representation chart of the second question describes the scenario

[0166] For the chart description scenario, it is generally required to clearly recall and display a specific chart, and the summary of the chart can be brief. Therefore, the output format is defined as:

[0167]

[0168] S203-4: Obtain the answer to the first question determined by the user according to the answer to the second question.

[0169] After obtaining the answer to the second question, obtain the answer to the first question determined by the user according to the answer to the second question. For example, "unchannelized" and "channelized". In this way, the answers to each question in the troubleshooting plan can be obtained.

[0170] In addition, in the actual application process, there is also such a situation that for one question, the user answers the answer to another question. At this time, the troubleshooting plan can be corrected in time. For example, if the feedback information is the first answer to the third question, the status of the third question in the troubleshooting plan is set to completed. The question is any question other than the first question in the troubleshooting plan. In this way, there is no need to repeat the questions that have been asked.

[0171] Figure 7 It is a schematic diagram of a troubleshooting guidance dialogue system with multi-agent collaboration provided by an embodiment of the present application. In this example, it includes a knowledge Q&A question, so that the answer can be obtained through the knowledge Q&A agent and provided to the user. Based on the multi-agent collaboration technology, on the basis of the generated SOP, the actions are planned in real time according to the scenario dialogue to achieve accurate intent understanding and action response.

[0172] Regarding S204, determine M potential hazard items according to each answer.

[0173] After obtaining the answers to each question, M potential hazard items can be determined according to the judgment rules. That is to say, there are M potential hazard items in the target troubleshooting scenario, where M is less than or equal to N. For example, in the scenario of a plane intersection, the potential hazard items are that large plane intersections are unchannelized and the sight distance at intersections is poor.

[0174] Regarding S205, for each potential hazard item, collect the associated data and verify the effectiveness of the potential hazard item based on the collected data.

[0175] To further verify the effectiveness of the potential hazard item, for each potential hazard item, the checklist for potential hazard item inspection shown in Table 1 can be combined to obtain the associated collected data, and the effectiveness of the potential hazard item can be further verified based on the collected data. For example, the collected data corresponding to the lack of channelization at large plane intersections is the intersection photos, and the collected data corresponding to poor intersection sight distance is the pictures of poor intersection sight distance, sight distance obstruction, actual sight distance, etc.

[0176] In the embodiments of the present application, first, a construction standard for the knowledge graph of all potential road traffic safety hazard items is proposed, and a structured representation model of potential hazard items is constructed from aspects such as the affiliated scenario, dependent data, prerequisite conditions, judgment rules, collection content, and recommended measures; second, using the planning and content generation capabilities of the large language model, according to the given scenario, a standard operating procedure (SOP) for scenario-based inspection tasks is dynamically generated as the main line of the inspection process; finally, a real-time inspection guidance dialogue system based on multi-agent collaboration is proposed. Through the collaboration of user agent intelligent agents, real-time task planning intelligent agents, and knowledge Q&A intelligent agents, controllability and autonomy are balanced to guide the inspectors to complete the given inspection tasks.

[0177] To more clearly illustrate the execution process of the inspection task, Figure 8 is a flowchart of the SOP for the task inspection of a single intersection scenario provided by the embodiments of the present application. According to Figure 8 It can be seen that when the answers to these questions are all obtained, the potential hazard item can be determined.

[0178] In the embodiments of the present application, a standardized representation model of road safety hazard items is proposed, and a knowledge graph of all traffic safety hazard items is constructed. Using the technology of dynamically generating the safety hazard inspection process based on the large language model technology, according to the given inspection task, the included scenarios are autonomously identified, and the inspection process is targeted planned, with strong flexibility, solving the problem of poor adaptability of traditional fixed processes. Solve the problem of insufficient autonomy and flexibility of traditional fixed SOP processes, poor adaptability and low inspection efficiency in complex combined scenarios. Based on multi-agent collaboration technology, on the basis of the generated SOP, actions are planned in real time based on scenario conversations to achieve accurate intention understanding and action response.

[0179] As Figure 9 shown, based on the same inventive concept, the embodiments of the present application provide a device for detecting traffic safety hazards, including a processing unit 91 and a transmission unit 92.

[0180] A processing unit 91, configured to: determine a troubleshooting task, and determine a target troubleshooting scenario according to the troubleshooting task; wherein, the troubleshooting task indicates the hidden danger situation of a set section to be troubleshot;

[0181] The processing unit 91 is further configured to: determine N troubleshooting items corresponding to the target troubleshooting scenario according to the corresponding relationship between the task scenario and the troubleshooting items; wherein, each troubleshooting item corresponds to a hidden danger item, and each troubleshooting item includes at least one troubleshooting element;

[0182] A transmission unit 92, configured to: obtain the answers corresponding to each problem associated with the N troubleshooting items by the user;

[0183] The processing unit 91 is further configured to: determine M hidden danger items according to each answer; wherein, M is less than or equal to N;

[0184] The transmission unit 92 is further configured to: for each hidden danger item, obtain the collected data associated with the hidden danger item, and verify the effectiveness of the hidden danger item based on the collected data.

[0185] In an alternative embodiment, the processing unit 91 is specifically configured to:

[0186] Receive a troubleshooting task sent by a task distribution system; wherein, the troubleshooting task includes a task location; or

[0187] Receive a troubleshooting task from the user's mobile terminal; wherein, the troubleshooting task includes location information.

[0188] In an alternative embodiment, the processing unit 91 is specifically configured to:

[0189] Determine a knowledge graph for representing the corresponding relationship between the task scenario and the troubleshooting items;

[0190] According to the task location or location information in the troubleshooting task, determine the target troubleshooting scenario of the troubleshooting task;

[0191] Traverse the knowledge graph according to the target troubleshooting scenario to determine N troubleshooting items corresponding to the target troubleshooting scenario.

[0192] In an alternative embodiment, the processing unit 91 is specifically configured to:

[0193] Obtain reference data; wherein, the reference data includes hidden danger judgment specification data, hidden danger judgment guidance data, and historical hidden danger troubleshooting cases;

[0194] Analyze the reference data to determine a hidden danger item knowledge table; wherein, the hidden danger item knowledge table includes the troubleshooting elements, judgment conditions, and corresponding hidden danger item types of each troubleshooting item; the troubleshooting elements include some or all of the scenario, dependent data, and preconditions;

[0195] Convert the hidden danger item knowledge table into a knowledge graph.

[0196] In an alternative embodiment, the processing unit 91 is further configured to:

[0197] Input N troubleshooting items into a set large language model to determine a troubleshooting plan;

[0198] Wherein, the troubleshooting plan includes multiple pieces of troubleshooting information, and each piece of troubleshooting information includes some or all of whether the problem conditions are met, the problem content, and the troubleshooting elements involved in the problem.

[0199] In an alternative embodiment, the transmission unit 92 is specifically configured to:

[0200] Obtain the feedback information of the user on the first problem; wherein, the first problem is any problem in the troubleshooting plan;

[0201] If the feedback information is the first answer to the first problem, determine that the first answer is the answer to the first problem;

[0202] If the feedback information is the second problem derived from the first problem, input the second problem into the set database, and obtain the answer to the second problem from the set database;

[0203] Obtain the answer to the first problem determined by the user according to the answer to the second problem.

[0204] In an alternative embodiment, the processing unit 91 is further configured to:

[0205] If the second problem is a knowledge Q&A type problem, determine the form of the answer to the second problem according to the characteristics of the second problem.

[0206] In an alternative embodiment, after obtaining the feedback information of the user on the first problem, the processing unit 91 is further configured to:

[0207] If the feedback information is the first answer to the third problem, set the status of the third problem in the troubleshooting plan to completed; wherein, the problem is any problem other than the first problem in the troubleshooting plan.

[0208] Since this device is the device in the method in the embodiment of the present application, and the principle of this device to solve problems is similar to that of this method, the implementation of this device can refer to the implementation of the method, and the repeated parts will not be elaborated.

[0209] As Figure 10 shown, based on the same inventive concept, the embodiment of the present application provides an electronic device, including a processor 101 and a memory 102.

[0210] The memory 102 is configured to execute:

[0211] Store the corresponding relationship between the task scenario and the troubleshooting items;

[0212] The processor 101 is configured to execute:

[0213] Determine a troubleshooting task and determine a target troubleshooting scenario according to the troubleshooting task; wherein, the troubleshooting task indicates the hidden danger situation of a set section to be troubleshot;

[0214] According to the corresponding relationship between the task scenario and the troubleshooting items, determine N troubleshooting items corresponding to the target troubleshooting scenario; wherein, each troubleshooting item corresponds to a hidden danger item, and each troubleshooting item includes at least one troubleshooting element;

[0215] Obtain the answers corresponding to each problem associated with the N troubleshooting items by the user;

[0216] Determine M hidden danger items according to each answer; wherein, M is less than or equal to N;

[0217] For each hidden danger item, obtain the collected data associated with the hidden danger item and verify the effectiveness of the hidden danger item based on the collected data.

[0218] In an optional implementation manner, the processor 101 is specifically configured to execute:

[0219] Receive a troubleshooting task sent by a task distribution system; wherein, the troubleshooting task includes a task location; or

[0220] Receive a troubleshooting task from the user's mobile terminal; wherein, the troubleshooting task includes location information.

[0221] In an optional implementation manner, the processor 101 is specifically used for:

[0222] Determine a knowledge graph for representing the corresponding relationship between the task scenario and the troubleshooting items;

[0223] According to the task location or location information in the troubleshooting task, determine the target troubleshooting scenario of the troubleshooting task;

[0224] Traverse the knowledge graph according to the target troubleshooting scenario to determine N troubleshooting items corresponding to the target troubleshooting scenario.

[0225] In an optional implementation manner, the processor 101 is specifically used for:

[0226] Obtain reference data; wherein, the reference data includes hidden danger judgment specification data, hidden danger judgment guidance data, and historical hidden danger troubleshooting cases;

[0227] Analyze the reference data to determine a hidden danger item knowledge table; wherein, the hidden danger item knowledge table includes the troubleshooting elements, judgment conditions, and corresponding hidden danger item types of each troubleshooting item; the troubleshooting elements include some or all of the scenario, dependent data, and prerequisite conditions;

[0228] Convert the hidden danger item knowledge table into a knowledge graph.

[0229] In an alternative embodiment, the processor 101 is further configured to:

[0230] Input N investigation items into a set large language model to determine an investigation plan;

[0231] Wherein, the investigation plan includes multiple investigation information, and each piece of investigation information includes whether the problem conditions are met, the problem content, and some or all of the investigation elements involved in the problem.

[0232] In an alternative embodiment, the processor 101 is specifically configured to:

[0233] Obtain the feedback information of the user on the first problem; wherein, the first problem is any problem in the investigation plan;

[0234] If the feedback information is the first answer to the first problem, determine that the first answer is the answer to the first problem;

[0235] If the feedback information is a second problem derived from the first problem, input the second problem into a set database and obtain the answer to the second problem from the set database;

[0236] Obtain the answer to the first problem determined by the user according to the answer to the second problem.

[0237] In an alternative embodiment, the processor 101 is further configured to:

[0238] If the second problem is a knowledge Q&A type problem, determine the form of the answer to the second problem according to the characteristics of the second problem.

[0239] In an alternative embodiment, after obtaining the feedback information of the user on the first problem, the processor 101 is further configured to:

[0240] If the feedback information is the first answer to the third problem, set the status of the third problem in the investigation plan to completed; wherein, the problem is any problem in the investigation plan other than the first problem.

[0241] The embodiment of the present application further provides a computer storage medium, in which computer program instructions are stored. When the instructions run on a computer, the computer is caused to execute the steps of the method for detecting traffic safety hidden dangers described above.

[0242] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0243] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0244] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0245] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0246] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.< / question>

Claims

1. A method for detecting traffic safety hazards, characterized in that: include: Determine an inspection task, and determine a target inspection scenario according to the inspection task; wherein the inspection task indicates an inspection of hidden dangers on a set road section; According to the correspondence between the task scenario and the troubleshooting items, determine N troubleshooting items corresponding to the target troubleshooting scenario; wherein each troubleshooting item corresponds to a hidden danger item, and each troubleshooting item includes at least one troubleshooting element; Obtaining answers corresponding to each question associated with the N troubleshooting items from the user; Determine M potential risk items based on the answers; where M is less than or equal to N; For each potential risk item, acquired data associated with the potential risk item is acquired, and the validity of the potential risk item is verified based on the acquired data.

2. The method according to claim 1, characterized in that The determining of the troubleshooting task includes: Receiving a troubleshooting task issued by a task issuing system; wherein the troubleshooting task includes a task location; or Receive a troubleshooting task from the user's mobile terminal; wherein the troubleshooting task includes positioning information.

3. The method according to claim 2, characterized in that Determining N troubleshooting items corresponding to the target troubleshooting scenario according to the correspondence between the task scenario and the troubleshooting items includes: Determine a knowledge graph used to represent the correspondence between task scenarios and troubleshooting items; Determine the target investigation scene of the investigation task according to the task location or positioning information in the investigation task; The knowledge graph is traversed according to the target investigation scenario to determine N investigation items corresponding to the target investigation scenario.

4. The method according to claim 3, characterized in that The step of determining a knowledge graph for representing the corresponding relationship between task scenarios and troubleshooting items includes: Obtain reference data; wherein the reference data includes hidden danger judgment specification data, hidden danger judgment guidance data and historical hidden danger investigation cases; Parse the reference data to determine a potential risk item knowledge table; wherein the potential risk item knowledge table includes the screening elements, judgment conditions and corresponding potential risk item types of each screening item; the screening elements include part or all of the scenarios, dependent data and prerequisites; The hidden danger item knowledge table is converted into a knowledge graph.

5. The method according to claim 1, characterized in that: The method further comprises: Input the N investigation items into a set large language model to determine an investigation plan; The troubleshooting plan includes multiple troubleshooting information, and each troubleshooting information includes whether the problem condition is met, the problem content, and part or all of the troubleshooting elements involved in the problem.

6. The method according to claim 5, characterized in that The obtaining of the user's answers to the respective questions associated with the N troubleshooting items includes: Obtaining user feedback information on a first question; wherein the first question is any question in the troubleshooting plan; If the feedback information is a first answer to the first question, determining that the first answer is an answer to the first question; If the feedback information is a second question derived from the first question, inputting the second question into a setting database, and obtaining an answer to the second question from the setting database; The answer to the first question determined by the user according to the answer to the second question is obtained.

7. The method according to claim 6, characterized in that The method further comprises: If the second question is a knowledge question, the form of the answer to the second question is determined according to the characteristics of the second question.

8. The method according to claim 6, characterized in that After obtaining the user's feedback information on the first question, the method further includes: If the feedback information is a first answer to a third question, the status of the third question in the troubleshooting plan is set to completed; wherein the question is any question in the troubleshooting plan except the first question.

9. An electronic device, characterized in that: including a processor and a memory; The memory is configured to execute: Store the correspondence between task scenarios and troubleshooting items; The processor is configured to perform: Determine an inspection task, and determine a target inspection scenario according to the inspection task; wherein the inspection task indicates an inspection of hidden dangers on a set road section; According to the correspondence between the task scenario and the troubleshooting items, determine N troubleshooting items corresponding to the target troubleshooting scenario; wherein each troubleshooting item corresponds to a hidden danger item, and each troubleshooting item includes at least one troubleshooting element; Obtaining answers corresponding to each question associated with the N troubleshooting items from the user; Determine M potential risk items based on the answers; where M is less than or equal to N; For each potential risk item, acquired data associated with the potential risk item is acquired, and the validity of the potential risk item is verified based on the acquired data.

10. The electronic device according to claim 9, characterized in that: The processor is specifically configured to execute: Receiving a troubleshooting task issued by a task issuing system; wherein the troubleshooting task includes a task location; or Receive a troubleshooting task from the user's mobile terminal; wherein the troubleshooting task includes positioning information.

Images