Road potential safety hazard troubleshooting terminal and server

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a road potential safety hazard troubleshooting terminal and a server. If an investigation starting request is received, the obtained list of sites to be investigated is displayed, if an investigation instruction for investigating any target site to be investigated in the list of sites to be investigated is received, an on-site investigation record is obtained, then the service analyzes content included in the on-site investigation record based on each problem to be investigated, and the analyzed content is sent to the server; the troubleshooting result of each to-be-troubleshot problem is determined and displayed on the display, so that troubleshooting personnel can check conveniently, only the field troubleshooting records need to be obtained in the whole process, the content included in the field troubleshooting records is analyzed on the basis of each to-be-troubleshot problem subsequently, the troubleshooting results are determined, the troubleshooting personnel do not need to deeply analyze the field troubleshooting records on site, and the troubleshooting efficiency is improved. And the technical requirements on troubleshooting personnel are reduced, and the hidden danger troubleshooting efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a road safety hazard inspection terminal and server. Background Art

[0002] Safety hazard inspection in social life is the focus of work of various departments. For example, the inspection of road safety hazards is difficult to find, collect and report. In recent years, the state has proposed a nationwide road traffic safety hazard inspection and rectification action to reduce the stock of hazards and control the number of accidents.

[0003] At present, the inspection of road safety hazards is a job that requires high professional skills. Inspectors need to have knowledge and experience of road safety hazards, and the professional skills required of inspectors are extremely high. This means that only a small number of people with professional knowledge can do this work, resulting in low efficiency in safety hazard inspection and failure to effectively reduce the total amount of safety hazards. In addition, inspectors are required to consult relevant documents such as laws and standards during the inspection process, which means that each inspector needs to spend a long time to consult information at each hazard point, and the inspection time for each hazard point is long, resulting in low overall inspection efficiency.

[0004] Therefore, how to improve the efficiency of potential safety hazards detection has become an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present application provide a road hazard inspection terminal and a server to solve the problem of low efficiency of safety hazard inspection in the prior art.

[0006] In a first aspect, the present application provides a road hazard inspection terminal, the terminal comprising a display and a processor:

[0007] The processor is configured to:

[0008] If a request to start the investigation is received, the obtained list of places to be investigated is displayed on the display;

[0009] If an inspection instruction is received to inspect any target location to be inspected in the list of locations to be inspected, an on-site inspection record is obtained and sent to the server, wherein the on-site inspection record includes at least one of an image, an audio, and a text;

[0010] If each problem to be checked and the corresponding troubleshooting result are received, they are displayed on the display, wherein the troubleshooting result is determined by the server analyzing the content included in the on-site troubleshooting record based on each problem to be checked.

[0011] In a second aspect, the present application provides a server configured to execute:

[0012] Receive on-site inspection records sent by the road hidden danger inspection terminal;

[0013] Based on each problem to be checked, the content included in the on-site inspection record is parsed to obtain the inspection result corresponding to each problem to be checked, and each problem to be checked and the corresponding inspection result are sent to the road hazard inspection terminal.

[0014] In a third aspect, the present application provides a method for checking for hidden dangers on roads, the method comprising:

[0015] If a request to start the investigation is received, the list of locations to be investigated is displayed;

[0016] If an inspection instruction is received to inspect any target location to be inspected in the list of locations to be inspected, obtaining an on-site inspection record;

[0017] Based on each problem to be investigated, the content included in the on-site investigation record is parsed to obtain and display the investigation results corresponding to each problem to be answered.

[0018] In a fourth aspect, the present application provides a road hidden danger detection device, the device comprising:

[0019] A display module, for displaying the obtained list of locations to be investigated if an investigation initiation request is received;

[0020] An acquisition module, configured to acquire an on-site inspection record upon receiving an inspection instruction to inspect any target location to be inspected in the list of locations to be inspected;

[0021] The troubleshooting module is used to parse the content included in the on-site troubleshooting record based on each problem to be troubleshooted, obtain the troubleshooting results corresponding to each problem to be answered and display them.

[0022] In a fifth aspect, the present application also provides an electronic device, comprising a processor, wherein the processor is configured to implement the steps of any of the above-mentioned methods for detecting road hazards when executing a computer program stored in a memory.

[0023] In a sixth aspect, the present application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for detecting road hazards.

[0024] Because in the embodiment of the present application, if a request to start an investigation is received, the list of locations to be investigated is displayed; if an inspection instruction to investigate any target location to be investigated in the list of locations to be investigated is received, the on-site inspection record is obtained, and then the service is enabled to parse the contents included in the on-site inspection record based on each problem to be investigated, determine the inspection result of each problem to be investigated and display it on the display for the convenience of the inspector to review. The entire process only requires obtaining the on-site inspection record, and then parsing the contents included in the on-site inspection record based on each problem to be investigated to determine the inspection result. There is no need for the inspector to conduct an in-depth analysis of the on-site inspection record on site, which reduces the technical requirements for inspectors and improves the efficiency of hidden danger investigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A schematic diagram of a road hazard inspection terminal provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of the structure of a road hidden danger inspection terminal provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram showing a list of locations to be checked provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of displaying the results of a troubleshooting provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the back side of a terminal provided in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of on-site inspection record input provided in an embodiment of the present application;

[0032] Figure 7 A schematic diagram of an image upload page provided in an embodiment of the present application;

[0033] Figure 8 A schematic diagram of on-site inspection record input provided in an embodiment of the present application;

[0034] Fig. 9 A schematic diagram of on-site inspection record input provided in an embodiment of the present application;

[0035] Fig.10A schematic diagram of determining a target supplementary location to be investigated provided in an embodiment of the present application;

[0036] Fig.11 A schematic diagram of displaying the results of a troubleshooting provided in an embodiment of the present application;

[0037] Fig.12 A schematic diagram of a road hazard inspection process provided in an embodiment of the present application;

[0038] Fig.13 A schematic diagram of the structure of a road hidden danger detection device provided in an embodiment of the present application;

[0039] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the embodiment of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0041] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0042] It should be noted that the terms "including" and "having" and their variations involved in the documents of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] The terms "first" and "second" in this document are used for descriptive purposes only and should not be understood as explicitly or implicitly indicating relative importance or implicitly indicating the number of technical features indicated. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0044] The word “exemplary” is used hereinafter to mean “serving as an example, example, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0045] At present, for the inspection of road hazards, generally, the inspector receives the inspection task and arrives at the site according to the location information of the site to be inspected. The inspector determines the operations to be performed and the inspection focus based on the description in the task, his own understanding and process requirements. For example, for the scene of a long downhill section, it is generally necessary to judge the downhill slope, whether there is a small access intersection, whether the deceleration setting, speed limit signs and monitoring facilities are complete, whether there is poor sight distance, whether there is a combination of curves and ramps, and supplement the observed problems according to the specific situation on site. After the inspector completes the inspection according to the above steps, he can fill in the situation at the road hazard inspection terminal. If you encounter unclear problems during the inspection, you need to manually search for electronic or paper materials, or consult other inspectors. After the inspection is completed, the inspection report is submitted and sent to relevant personnel for review and process flow.

[0046] The above-mentioned process of road hidden danger inspection is relatively complicated, and the requirements for the professional ability of the inspectors are relatively high, resulting in low efficiency of road hidden danger inspection. Based on this, the embodiment of the present application provides a road hidden danger inspection terminal and server, the terminal includes a display and a processor, wherein the processor is configured to: if an inspection start request is received, the obtained list of places to be inspected is displayed on the display; if an inspection instruction to inspect any target place to be inspected in the list of places to be inspected is received, the on-site inspection record is obtained, and based on each problem to be inspected, the content included in the on-site inspection record is parsed, the inspection result of each problem to be inspected is determined, and each problem to be inspected and the corresponding inspection result are displayed on the display.

[0047] For example, the road hazard inspection terminal used in the road hazard inspection process can be a mobile phone, on which an application is installed, and the application can be used for road hazard inspection. When performing corresponding operations, the functions of sound collection, image collection, sound playback, and location information collection can be realized through the support of the road hazard inspection terminal hardware. Figure 1 A schematic diagram of a road hazard inspection terminal provided in an embodiment of the present application, such as Figure 1 As shown, the terminal can be configured with a microphone, a display, and a speaker. The microphone is used for sound collection, the display is used for information display, and the speaker is used for sound playback. At the same time, the terminal can also have a global positioning system (GPS) positioning function to realize location information collection.

[0048] It should be noted that the above examples are only for the sake of understanding. The road hazard inspection terminal is not limited to mobile phones, but can also be other devices, such as servers, recorders, etc. Of course, even if the road hazard inspection terminal is a mobile phone, the functions of the mobile phone are not limited to the above examples. The following will describe the process of road hazard inspection based on the road hazard inspection terminal in combination with various embodiments. Figure 2 A schematic diagram of the structure of a road hazard inspection terminal provided in an embodiment of the present application, the terminal 10 includes a display 101 and a processor 102, wherein the processor 102 is configured to: if an inspection start request is received, display the acquired list of locations to be inspected on the display 101; if an inspection instruction to inspect any target location to be inspected in the list of locations to be inspected is received, obtain an on-site inspection record, and send it to a server, wherein the on-site inspection record includes at least one of an image, audio, and text; if each problem to be inspected and the corresponding inspection result are received, they are displayed on the display 101, wherein the inspection result is determined by the server parsing the content included in the on-site inspection record based on each problem to be inspected.

[0049] In order to check for hidden dangers on roads, the inspector may use the road hidden danger inspection terminal 10 (hereinafter referred to as the terminal 10) to perform the road hidden danger inspection task. When it is necessary to inspect for hidden dangers on roads, the inspector may send an inspection start request to the terminal 10. For example, when an application is started on the display page of the terminal, it can be considered that the inspector has sent the inspection start request. Of course, the inspector may also send the inspection start request by clicking a start button in an application.

[0050] In the embodiment of the present application, if the processor 102 receives a request to start the inspection, it means that the inspector wants to perform the hidden danger inspection task, and the processor 102 can control the display 101 to display the obtained list of places to be inspected. The list of places to be inspected can be obtained by the terminal 10 from a preset storage location, or sent by other electronic devices connected to the terminal 10.

[0051] For example, the terminal 10 can establish a connection with the server via WebSocket and periodically pull the list of locations to be checked from the server in a heartbeat manner. After receiving the request from the terminal 10, the server can read all the locations to be checked from the database and return all the locations to be checked to the terminal 10 in a set format.

[0052] Figure 3 A schematic diagram of a list of locations to be checked provided in an embodiment of the present application is shown in FIG. Figure 3As shown, the display 101 of the terminal 10 displays the acquired list of places to be checked, and each place to be checked has a corresponding "check" button. In order to facilitate the checker to go to the corresponding place for checking, the specific location and name corresponding to each place to be checked can be displayed. After the checker checks the list of places to be checked at the terminal 10, he can arrive at the inspection site according to the description of the specific location corresponding to each place to be checked. After the checker arrives at the site, he can click the corresponding "check" button on the display page to enter the inspection page of the corresponding place to be checked and start to perform the inspection task. The list of places to be checked may include the places to be checked that have been checked, and may also include the places to be checked that have not been checked. The checker can click the "to be checked" or "checked" button to filter the place. Of course, the checker can also view the inspection tasks he needs to perform by clicking the "my" button.

[0053] After checking the list of places to be checked, the inspector can select any place to be checked for inspection. For the convenience of description, the selected place to be checked is referred to as the target place to be checked. If the processor 102 receives an inspection instruction to inspect the target place to be checked, it can be considered that the inspector is about to inspect the hidden dangers of the target place to be checked. The on-site inspection record can be obtained so that the server can determine whether the corresponding target object has a safety hazard based on the on-site inspection record. The on-site inspection record can be an image, an audio, or a text, and can also include both an image and a text. The on-site inspection record can be directly input by the inspector. For example, after the inspector arrives at the scene, he shoots the target object and inputs the shot image into the terminal 10 carried by him. The on-site inspection record can also be sent by other electronic devices connected to the terminal 10. The on-site inspection record can also be obtained from a preset storage location. In other words, the obtained on-site inspection record can be real-time data or historical data. This also shows that the road hidden danger inspection terminal provided in the embodiment of the present application can perform real-time online inspection or offline inspection. The embodiment of the present application does not limit the method for obtaining on-site inspection records, and technicians in this field can configure it as needed.

[0054] After obtaining the on-site inspection record, the on-site inspection record can be sent to the server, and the server determines the inspection result corresponding to each problem to be inspected based on the on-site inspection record. The server can obtain each problem to be inspected when determining the problem to be inspected. In an embodiment of the present application, the problem to be inspected can be one or more. For example, multiple problems to be inspected are saved in a list of problems to be inspected. That is to say, each location to be inspected stores each problem that needs to be inspected, and these problems to be inspected constitute a list of problems to be inspected. The number of problems to be inspected corresponding to the same location to be inspected can be one or more. The problems to be inspected corresponding to different locations to be inspected can be the same or different. The problem to be inspected can be pre-stored in a preset storage location, such as pre-stored in a database; it can also be obtained from the outside together with the on-site inspection record and sent by the road hidden danger inspection terminal, such as when the inspector enters the on-site inspection record, the corresponding list of problems to be inspected is entered into the road hidden danger inspection terminal.

[0055] After obtaining the on-site inspection record and the list of problems to be inspected, the content included in the on-site inspection record can be parsed based on each problem to be inspected in the list of problems to be inspected to determine the inspection result of each problem to be inspected. For example, based on the pre-trained image recognition model, the image is recognized to determine the inspection result corresponding to each problem to be inspected. Exemplarily, any problem to be inspected can be "Is there a warning sign on the side of the road?", and the determined inspection result can be "Yes".

[0056] After determining the inspection results corresponding to each problem to be inspected, in order for the inspector to view the inspection results, the server can send each problem to be inspected and the corresponding inspection results to the road hazard inspection terminal, and the road hazard inspection terminal will display each problem to be inspected and the corresponding inspection results.

[0057] Figure 4 A schematic diagram of displaying the results of a troubleshooting provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the target location to be checked is "Location to be checked 1", the check result corresponding to the problem to be checked "Is there any road surface damage?" is "Yes", the check result corresponding to the problem to be checked "Is there any roadside guardrail missing or damaged?" is "Yes", the check result corresponding to the problem to be checked "Is there any traffic sign damaged or unclear?" is "Yes", the check result corresponding to the problem to be checked "Is there any signal light missing or damaged?" is "Yes", and the check result corresponding to the problem to be checked "Is there any road lighting facility damaged or missing?" is "Yes". After checking the check result corresponding to each problem to be checked, the inspector can check the check result and click the "Save" button after confirming that it is correct.

[0058] For example, the result of the inspection may be that an abnormality exists, that an abnormality does not exist, or that an abnormality is not detected. If the inspection result is that the problem to be inspected is not detected, it means that the problem to be inspected was not inspected in this inspection, and other inspectors can wait for the subsequent inspection. In other words, when inspectors conduct road hazard inspections, they can inspect all the problems to be inspected corresponding to a certain location to be inspected, or they can inspect some of the problems to be inspected.

[0059] Because in the embodiment of the present application, if a request to start an investigation is received, the list of locations to be investigated is displayed; if an inspection instruction to investigate any target location to be investigated in the list of locations to be investigated is received, the on-site inspection record is obtained, and then the service is enabled to parse the contents included in the on-site inspection record based on each problem to be investigated, determine the inspection result of each problem to be investigated and display it on the display for the convenience of the inspector to review. The entire process only requires obtaining the on-site inspection record, and then parsing the contents included in the on-site inspection record based on each problem to be investigated to determine the inspection result. There is no need for the inspector to conduct an in-depth analysis of the on-site inspection record on site, which reduces the technical requirements for inspectors and improves the efficiency of hidden danger investigation.

[0060] In order to improve the accuracy of road hazard inspections, based on the above embodiments, in an embodiment of the present application, the server can analyze the on-site inspection records and the list of problems to be inspected based on a large model to determine the inspection results corresponding to each problem to be inspected in the list of problems to be inspected.

[0061] In order to improve the accuracy of road hidden danger inspection, in an embodiment of the present application, when the server parses the content included in the on-site inspection record based on each problem to be inspected and determines the inspection result of each problem to be inspected, it can analyze the on-site inspection record and the list of problems to be inspected based on the big model to determine the inspection result corresponding to each problem to be inspected. In an embodiment of the present application, the big model can be a big language model or a multimodal big model. Those skilled in the art can select a suitable big model according to the type of on-site inspection record, and the embodiment of the present application does not limit this. After receiving the on-site inspection record and each problem to be inspected, the big model can analyze each problem to be inspected in turn and determine the corresponding inspection result. For example, any problem to be inspected can be "Is there a warning sign on the side of the road?", and the inspection result determined by the big model can be "existence".

[0062] It should be noted that how the big model analyzes the on-site inspection records and each problem to be inspected is the internal processing mechanism of the big model, and the embodiments of the present application will not go into details about this.

[0063] In order to further improve the accuracy of road hidden danger detection, based on the above embodiments, the server is specifically configured to execute:

[0064] If the on-site inspection record is an image, according to the objects to be inspected corresponding to each problem to be inspected, identify the target object corresponding to each object to be inspected in the image, and whether the target object has an abnormality;

[0065] Based on the big model, analyze whether the target object has an abnormality and each problem to be checked, and determine the troubleshooting result corresponding to each problem to be checked.

[0066] In the embodiment of the present application, the terminal 10 may also include a camera, such as Figure 5 As shown, Figure 5 This is a schematic diagram of the back of the terminal provided in the embodiment of the present application. Figure 5 The upper left corner (left and right in the figure) of the terminal shown in is the rear camera of the terminal 10.

[0067] When entering on-site inspection records, inspectors can click the "Add Image" button to upload at least one image. Figure 6 As shown, Figure 6 A schematic diagram of a field investigation record input provided in an embodiment of the present application, where the investigator can click Figure 6 After receiving the image adding instruction, the terminal 10 can jump to display the following Figure 7 The image shown in the Add page, Figure 7 A schematic diagram of an image upload page provided for an embodiment of the present application, wherein the inspector can select an existing image in the terminal 10 or take a photo of the image immediately. After the image selection is completed, the inspector can click the "Finish" button to complete the upload of the on-site inspection record.

[0068] If the processor 102 receives an image capture instruction, it can control the image acquisition unit to capture an image of the target location to be inspected. The image capture instruction can be triggered after the inspector clicks the "shoot" button. In other words, the inspector can use the handheld terminal 10 to capture an image of the target location to be inspected through the image acquisition unit.

[0069] After acquiring the image of the target location to be inspected, the image can be sent to the server as a site inspection record.

[0070] If the on-site inspection record received by the server is an image, the server can first determine the object to be inspected corresponding to each problem to be inspected. For example, the object to be checked for the question to be checked, “Is the zebra crossing on the road clear?”, is “zebra crossing”; the object to be checked for the question to be checked, “Is the house tilted?”, is “house”; the object to be checked for the question to be checked, “Is there a missing or damaged warning sign?”, is “warning sign”; the object to be checked for the question to be checked, “Is there a missing or damaged roadside guardrail?”, is “roadside guardrail”; the object to be checked for the question to be checked, “Is there a missing or damaged central isolation belt?”, is “central isolation belt”; the object to be checked for the question to be checked, “Is there a missing or damaged speed limit sign?”, is “speed limit sign”; the object to be checked for the question to be checked, “Is there a roadside obstacle?”, is “roadside obstacle”; the object to be checked for the question to be checked, “Is there a problem of superelevation on the curve?”, is “road surface”; the object to be checked for the question to be checked, “Is there an unreasonable road surface anti-skid coefficient?”, is “road surface”. The object to be checked corresponding to each problem to be checked may be determined after analyzing the problem to be checked based on a pre-trained model; or may be saved by relevant personnel when writing the problem to be checked.

[0071] After determining the object to be checked corresponding to each problem to be checked, the target object corresponding to each object to be checked in the image can be identified based on the object to be checked corresponding to each problem to be checked. In other words, identify whether there is a target object in the image and the position of the target object in the image, so as to facilitate the subsequent determination of whether there is an abnormality in the target. While identifying each target object, it is also possible to determine whether there is an abnormality in the target object, for example, determine whether the target object is damaged. It should be noted that when identifying and detecting the target object, both a small model and a large model can be used for processing, and technicians in this field can configure it as needed.

[0072] After determining the target object and whether the target object is abnormal, the description text of whether the target object is abnormal and each problem to be checked can be input into the big model, so that the big model analyzes the description text and each problem to be checked and determines the troubleshooting result corresponding to each problem to be checked. Exemplarily, the data format output by the big model can be in the form shown in Table 1 below:

[0073] Table 1

[0074]

[0075] In a possible implementation, when identifying a target object in an image, identification can also be performed based on a large model. In an embodiment of the present application, an image and a corresponding prompt word can be input into a large model, so that the large model can identify the area where the target object in the image is located, and determine whether the target object has an abnormality, thereby determining the troubleshooting result corresponding to each problem to be troubleshooted.

[0076] In a possible implementation, after the image is acquired, the image may be preprocessed, such as noise reduction, to further ensure that subsequent road hazard inspection is accurately performed based on the image.

[0077] In an embodiment of the present application, a training set may be pre-constructed, and the original model may be trained based on the sample data included in the training set to obtain a model for identifying a target object and detecting whether the target object has an abnormality.

[0078] When constructing a training set, a preset number of sample images can be collected in advance, and the area where the target object is located can be marked in the sample image, for example, using a labeling box to mark the target object, and a corresponding label can be saved for each sample image, and the label is used to describe the target object in the corresponding image and whether the target object is abnormal. The above sample images and corresponding labels constitute sample data.

[0079] After obtaining the training set, each sample data in the training set can be input into the original model for training until the model converges to obtain a trained model. The trained model can be used to identify whether there is a target object in the image and whether there is an abnormality in the target object. The following Table 2 is an example of a recognition result provided in an embodiment of the present application:

[0080] Table 2

[0081] Target object Does it exist? Whether to save the exception again Warning signs yes yes Roadside guardrail no - Central isolation zone yes no Speed ​​limit sign yes no Roadside obstacles yes -

[0082] The "-" in the above Table 1 indicates that the corresponding result is not recognized. According to the above Table 1, the trained model can recognize the warning sign, and the warning sign has an abnormality, cannot recognize the roadside guardrail, can recognize the central isolation belt, and the central isolation belt has no abnormality, can recognize the speed limit sign, and the speed limit sign has no abnormality, can recognize the existence of roadside obstacles, but there is no problem of whether there is an abnormality for the obstacle, and the roadside obstacles need to be cleared in time. Therefore, the recognition result of whether there is an abnormality corresponding to the roadside obstacle in the above Table 1 is "-".

[0083] In order to further improve the accuracy of road hidden danger inspection, based on the above embodiments, in the embodiment of the present application, if the on-site inspection record is audio, the server is further configured to execute:

[0084] Performing speech recognition on the audio to obtain a recognition text corresponding to the audio and sending the recognition text to the road hazard inspection terminal;

[0085] If a text confirmation instruction is received, the on-site inspection record is updated using the identified text.

[0086] In order to facilitate the investigator to enter the on-site investigation record, the terminal 10 may also include a microphone. The investigator may enter the on-site investigation record by voice input. Figure 8 A schematic diagram of a field investigation record input provided in an embodiment of the present application is shown as follows: Figure 8 As shown, the investigator can click the "voice input" button to speak the information to be input in natural language, which will be collected by the audio collection unit to obtain audio.

[0087] If the processor 102 receives the audio input instruction, it can control the audio collection unit to collect audio through the microphone. The audio input instruction can be triggered when the investigator clicks the "voice input" button.

[0088] After receiving the collected audio, the processor 102 may send the audio to the server. If the server receives the on-site inspection record as audio, it may perform voice recognition on the audio to obtain the recognition text corresponding to the audio. Since the voice recognition process may cause incorrect recognition or typos, after obtaining the recognition text, the recognition text may be sent to the road hazard inspection terminal. If the road hazard inspection terminal receives the recognition text sent by the server, it may display the recognition text on the display 101 to facilitate the inspector to check the recognition text.

[0089] Fig. 9 A schematic diagram of a field investigation record input provided in an embodiment of the present application is shown as follows: Fig. 9 As shown, the terminal 10 will recognize the text "Description of hidden dangers entered by the user: This road has many problems. The road surface is bumpy and the car is easy to lose control. The roadside guardrail is also broken and cannot stop the car from rushing out. The road signs and traffic lights are not clear, which is easy to cause accidents. The lights are not enough at night, and the driver cannot see the road clearly, which increases the risk. It is recommended to repair the road, repair the guardrails, clarify the road signs and traffic lights, and strengthen the night lighting." in the dialog box. After confirming that everything is correct, the inspector can click the "Upload" button to complete the entry of the on-site inspection record.

[0090] If the processor 102 receives the text confirmation instruction, it can send the recognition text to the server. After receiving the text confirmation instruction, the server can use the recognition text to update the on-site inspection record. The text confirmation instruction can be triggered after the inspector clicks the "upload" button on the road hidden danger inspection terminal.

[0091] In an embodiment of the present application, if the on-site inspection record is audio, the processor of the road hazard inspection terminal can also recognize the audio, obtain the recognition text corresponding to the audio, and display the recognition text on the display; if a text confirmation instruction is received, the recognition text can be used to update the on-site inspection record. That is to say, in an embodiment of the present application, when performing speech recognition on the audio, the terminal 10 can recognize it based on the built-in speech-to-text software development kit (SDK), that is, the audio is processed locally. If the terminal 10 does not have a built-in speech-to-text SDK, the audio can be uploaded to the server, and after the speech recognition is completed, the server will send the recognition text to the terminal 10.

[0092] In order to further improve the accuracy of road hidden danger detection, based on the above embodiments, in the embodiment of the present application, the server is specifically configured to execute:

[0093] If the on-site inspection record is in the form of text, the on-site inspection record, each problem to be inspected and the prompt text are input into the big model, so that the big model analyzes the on-site inspection record and each problem to be inspected based on the prompt of the prompt text, and determines the inspection result corresponding to each problem to be inspected.

[0094] In an embodiment of the present application, if the on-site investigation record obtained by the server is text, the text classification, text extraction, and text generation technology of the large model can be called to match the on-site investigation record with the list of problems to be investigated. If there is a part in the on-site investigation record related to the problems to be investigated, an answer is given according to the content of the on-site investigation record. Questions not covered by the on-site investigation record do not need to be answered. The text can be input by the investigator in audio form, or the investigator can manually enter the corresponding content in the input box. In an embodiment of the present application, the on-site investigation record, the list of problems to be investigated, and the prompt text can be input into the large model, so that the large model analyzes the on-site investigation record and the list of problems to be investigated based on the prompt of the prompt text, and determines the investigation results corresponding to each problem to be investigated in the list of problems to be investigated.

[0095] Specifically, the on-site inspection records and issues to be inspected may be as shown in Table 3 below:

[0096] Table 3

[0097]

[0098] Based on the above on-site investigation records and problems to be investigated, the investigation results can be shown in Table 4 below:

[0099] Table 4

[0100]

[0101] In order to further improve the accuracy of road hidden danger detection, based on the above embodiments, in the embodiment of the present application, the server is specifically configured to execute:

[0102] If the on-site inspection record is a plurality of images of the same target object, determining the overlapping area in all the images according to the content included in each image;

[0103] Generating a complete target object image of the target object according to the target objects included in all the overlapping areas;

[0104] Based on the large model, it is determined whether there is any abnormality in the target object in the complete target object image, and the troubleshooting result of the problem to be troubleshooted corresponding to the target object is determined.

[0105] In an embodiment of the present application, the inspector can upload or take multiple images of the same target object, such as multiple images at different shooting positions and shooting angles. Due to different shooting positions and shooting angles, a certain image may only include a partial image of the target object. Therefore, in order to further improve the accuracy of road hidden danger inspection, in an embodiment of the present application, if the acquired on-site inspection record is multiple images of the same target object, the overlapping area in all images is determined according to the content included in each image, and the overlapping area is the image area containing the target object. After all the overlapping areas are determined, a complete target object image of the target object can be generated based on the target objects contained in all the overlapping areas. Exemplarily, in an embodiment of the present application, the images of these overlapping areas can be contrast enhanced, grayscale enhanced, stretched, and recombined to obtain a complete target object image of the target object. How to generate a complete image corresponding to the object based on multiple images of the same object is a prior art, and the embodiment of the present application will not be repeated here.

[0106] In the embodiment of the present application, in order to further improve the accuracy of road hazard detection, the neighborhood averaging method and the median filtering method can also be used to eliminate noise from the complete target object image.

[0107] After obtaining the complete target object image, it is possible to determine whether the target object in the complete target object image has an abnormality based on the large model, and determine the troubleshooting results of the problem to be troubleshooted corresponding to the target object. The above embodiment has already described how the large model determines whether the target object has an abnormality and the corresponding troubleshooting results, and the embodiments of this application will not be repeated here.

[0108] In order to further improve the accuracy of road hidden danger detection, based on the above embodiments, in the embodiment of the present application, the server is specifically configured to execute:

[0109] If the on-site inspection record is a plurality of images of the same target object, for each image, the image and each problem to be inspected are analyzed based on the large model to determine a candidate inspection result for each problem to be inspected corresponding to the target object in the image;

[0110] For each problem to be checked, the candidate checking results of each image corresponding to the problem to be checked are fused to obtain the checking result corresponding to the problem to be checked.

[0111] Since for the same object, the observed state is different when the observation angle is different, for example, the top of a cube object is damaged, but it cannot be found that it is damaged when observed from the front view. Therefore, in order to further improve the accuracy of road hidden danger inspection, in an embodiment of the present application, if the acquired on-site inspection record is multiple images of the same target object, the target object in each image can be analyzed to determine the candidate inspection result corresponding to each image, and then based on the candidate inspection result corresponding to each image, the final inspection result is comprehensively determined. In an embodiment of the present application, for each image, the image and the list of problems to be inspected can be analyzed based on the large model to determine the candidate inspection result corresponding to each problem to be inspected for the target object in the image. After obtaining the candidate inspection result corresponding to each problem to be inspected for each image, the candidate inspection result corresponding to the problem to be inspected can be fused for each problem to be inspected, and the inspection result corresponding to the problem to be inspected can be obtained. When the candidate inspection results are fused, the candidate inspection result with the most occurrences can be determined as the inspection result corresponding to the corresponding problem to be inspected. Alternatively, each candidate screening result may be input into a large model so that the large model can integrate all candidate screening results and summarize the screening results corresponding to the problem to be screened.

[0112] In order to further improve the efficiency of road hidden danger inspection, based on the above embodiments, in the embodiment of the present application, the server is further configured as follows:

[0113] Obtaining the current first location information of the road hidden danger inspection terminal and the second location information of the to-be-inspected location corresponding to the to-be-inspected problem for which no inspection result has been obtained;

[0114] Based on the first location information, the second location information and the distance threshold, the target supplementary location to be checked that is within the distance threshold is determined and sent to the road hazard inspection terminal, so that the road hazard inspection terminal displays the target supplementary location to be checked, so that the user of the road hazard inspection terminal can go to the target supplementary location to be checked to check the problems to be checked for which no inspection results have been obtained.

[0115] In order to further improve the efficiency of road hazard inspection, in the embodiment of the present application, the server can also obtain the current first location information of the terminal. The first location information can be obtained by the terminal through the GPS function and sent to the server. After obtaining the first location information, the terminal 10 can upload the first location information to the server at a fixed frequency, so that the server can obtain the location to be inspected near the terminal 10, so that the inspector can conduct road hazard inspection nearby.

[0116] In an embodiment of the present application, the server may also determine all the problems to be checked for which no troubleshooting results have been obtained. And determine the second location information of the place to be checked corresponding to the list of problems to be checked where each problem to be checked is located. The first location information and the second location information may be a street name, a house number, or latitude and longitude information, which is not limited in the embodiment of the present application.

[0117] After obtaining the first location information and the second location information, the target supplementary locations to be checked that are within the distance threshold of the terminal can be determined based on the first location information, the second location information, and the distance threshold. That is, based on the first location information and the second location information, determine which locations to be checked that have not been checked within the distance threshold of the terminal 10, and determine these locations to be checked as target supplementary locations to be checked. The distance threshold can be any value, for example, 100 meters, 1000 meters, 58.5 meters, etc., and can be configured by a person skilled in the art as needed.

[0118] After determining the target location to be supplemented for investigation, the server may send the target location to be supplemented for investigation to the terminal 10. After receiving the target location to be supplemented for investigation, the terminal 10 may display the second location information corresponding to the target location to be supplemented for investigation on the display 101, so that the user can go to the target location to be supplemented for investigation to investigate the problem to be investigated for which the investigation result has not been obtained.

[0119] Fig.10 A schematic diagram of determining a target supplementary location to be investigated provided in an embodiment of the present application, such as Fig.10As shown, the locations to be checked corresponding to the problems to be checked for which the check results have not been obtained include: location to be checked 1, location to be checked 2, location to be checked 3, location to be checked 4, and location to be checked 5. According to the first location information and each second location information and the distance threshold r, it can be determined that the target supplementary locations to be checked within the distance threshold r of the terminal 10 are location to be checked 3, location to be checked 4, and location to be checked 5. The inspector can go to any target supplementary location to be checked to check for road hazards.

[0120] In order to further improve the efficiency of road hidden danger inspection, based on the above embodiments, in the embodiment of the present application, the server is further configured to execute:

[0121] Determine the current moving speed of the road hidden danger inspection terminal;

[0122] Determine the target delay value corresponding to the moving rate according to the correspondence between different rates and delay values ​​stored in advance;

[0123] According to the current time and the target delay value, a supplementary location to be checked that is being removed is determined, and the supplementary location to be checked that is being removed is deleted from the target supplementary location to be checked.

[0124] Since the inspector may be standing still, walking, or in a moving vehicle when carrying the terminal 10. In order to further improve the efficiency of road hidden danger inspection, it can be determined whether the terminal 10 is close to or away from the target supplementary location to be inspected. In an embodiment of the present application, the moving speed of the current terminal 10 can be determined. And according to the correspondence between the different speeds and the delay values ​​saved in advance, the target delay value corresponding to the moving speed is determined. Since the moving speed of the vehicle is relatively fast and the walking speed is relatively slow, therefore, in an embodiment of the present application, the delay value corresponding to high-speed movement can be determined as 0.2 seconds, the delay value corresponding to low-speed movement can be determined as 10 seconds, and the delay value corresponding to relative stillness can be determined as 45 seconds. In an embodiment of the present application, after determining the current moving speed, it can be determined whether the moving speed is high-speed movement, low-speed movement, or relative stillness.

[0125] After the target delay value is determined, the additional location to be checked that is being moved away can be determined based on the current time and the target delay value. That is, the location of the terminal at the moment of the current time + target delay value is determined, and then the distance between the location and the location corresponding to the first location information relative to each target additional location to be checked is compared, thereby determining whether the terminal 10 is moving away from the target additional location to be checked, or is approaching the target additional location to be checked.

[0126] In order to prevent the inspectors from “going back on their ways” or “taking long detours”, after the additional locations to be inspected that are being moved away are determined, the additional locations to be inspected that are being moved away can be deleted from the determined target additional locations to be inspected.

[0127] For one or more target supplementary locations to be checked that the terminal 10 is approaching, the nearest target supplementary location to be checked can be determined as a recommended location to be checked. After the checker arrives at the target supplementary location to be checked, a check instruction to check the target supplementary location to be checked can be sent. At this time, the problems to be checked in the list of problems to be checked obtained by the processor 102 can be unanswered questions, that is, problems to be checked for which no check results have been obtained.

[0128] In a possible implementation, for some relatively simple questions, the investigator can be guided to answer. In the embodiment of the present application, the investigator can be guided to open the application deployed on the investigation terminal by means of notification reminders, and the text information of the questions to be answered can be presented on the interface of the display, and the user can be guided to answer each question. The speaker of the terminal can also be used to broadcast the questions to be answered in order by voice broadcast. After the broadcast is completed, the microphone is turned on to collect the investigator's reply, and the investigator's voice reply is converted into text, and then fuzzy matched with the standard answer, and the relevant answer is selected according to the matching result. For example, the terminal can broadcast the question to be investigated "whether there are roadside obstacles", and the investigator can answer by inputting "no" in text, or by inputting "no" by voice input. The terminal 10 can determine the investigation result of "whether there are roadside obstacles: no" based on the user's answer.

[0129] To facilitate subsequent review, after obtaining the troubleshooting results, the terminal 10 can send the corresponding questions to be answered and the troubleshooting results to the server so that the server can save them.

[0130] It should be noted that the present application may display the inspection results of road hazards in any form and is not limited to the above examples. Those skilled in the art may configure the display method of the inspection results as needed. Fig.11 A schematic diagram of displaying the results of a troubleshooting provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, when displaying each problem to be checked and the corresponding troubleshooting result, all the optional options corresponding to each problem to be checked can be displayed, and the troubleshooting result finally obtained from the server can be displayed emphatically in the display, for example, the border of the option box corresponding to the determined troubleshooting result can be displayed emphatically. Fig.11As shown in the figure, if the result of the check for question 1 "Is there any road damage?" is yes, the border of the "Yes" option box is highlighted when displayed. If the result of the check for question 2 "Is there any roadside guardrail missing or damaged?" is no, the border of the "No" option box is highlighted when displayed.

[0131] In an embodiment of the present application, after obtaining data or requests, the terminal can send the corresponding data or requests to the server, such as sending a troubleshooting start request, troubleshooting instructions, on-site troubleshooting records, etc. to the server. The server obtains relevant data from the database according to the request and returns it to the terminal. For example, after the server obtains the problem to be troubleshooted from the database, it sends each problem to be troubleshooted to the terminal. The server can also store the data obtained from the terminal, such as the troubleshooting results corresponding to each problem to be troubleshooted, on-site troubleshooting records, etc., in the database.

[0132] Based on the same inventive concept, the present application also provides a method for checking road hazards. Fig.12 A schematic diagram of a road hidden danger inspection process provided in an embodiment of the present application, the process includes the following steps:

[0133] S1201: If a request to start an investigation is received, the obtained list of locations to be investigated is displayed.

[0134] The road hazard detection method provided in the present application is applied to electronic devices, which may be servers, PCs, mobile terminals, etc.

[0135] S1202: If an inspection instruction is received to inspect any target location to be inspected in the list of locations to be inspected, an on-site inspection record is obtained.

[0136] S1203: Based on each problem to be investigated, the content included in the on-site investigation record is parsed to obtain and display the investigation results corresponding to each problem to be answered.

[0137] In a possible implementation, based on each problem to be investigated, parsing the content included in the on-site investigation record to obtain the investigation result corresponding to each problem to be answered includes:

[0138] The on-site inspection records and the list of problems to be inspected are analyzed based on the big model to determine the inspection results corresponding to each problem to be inspected in the list of problems to be inspected.

[0139] In a possible implementation manner, obtaining the on-site investigation record includes:

[0140] If an image shooting instruction is received, an image of the target location to be checked is shot;

[0141] Based on each problem to be investigated, the content included in the on-site investigation record is parsed to obtain the investigation result corresponding to each problem to be answered, including:

[0142] According to the objects to be checked corresponding to each problem to be checked, identifying the target object corresponding to each object to be checked in the image, and whether the target object has an abnormality;

[0143] Based on the big model, analyze whether the target object has an abnormality and each problem to be checked, and determine the troubleshooting result corresponding to each problem to be checked.

[0144] In a possible implementation manner, obtaining the on-site investigation record includes:

[0145] If an audio input command is received, the audio is collected;

[0146] Performing speech recognition on the audio, obtaining and displaying the recognized text corresponding to the audio;

[0147] If a text confirmation instruction is received, the identified text is determined as the on-site inspection record.

[0148] In a possible implementation, based on each problem to be investigated, parsing the content included in the on-site investigation record to obtain the investigation result corresponding to each problem to be answered includes:

[0149] If the on-site inspection record is in the form of text, the on-site inspection record, each problem to be inspected and the prompt text are input into the big model, so that the big model analyzes the on-site inspection record and each problem to be inspected based on the prompt of the prompt text, and determines the inspection result corresponding to each problem to be inspected.

[0150] In a possible implementation, based on each problem to be investigated, parsing the content included in the on-site investigation record to obtain the investigation result corresponding to each problem to be answered includes:

[0151] If the on-site inspection record is a plurality of images of the same target object, determining the overlapping area in all the images according to the content included in each image;

[0152] Generating a complete target object image of the target object according to the target objects included in all the overlapping areas;

[0153] Based on the large model, it is determined whether there is any abnormality in the target object in the complete target object image, and the troubleshooting result of the problem to be troubleshooted corresponding to the target object is determined.

[0154] In a possible implementation, based on each problem to be investigated, parsing the content included in the on-site investigation record to obtain the investigation result corresponding to each problem to be answered includes:

[0155] If the on-site inspection record is a plurality of images of the same target object, for each image, the image and each problem to be inspected are analyzed based on the large model to determine a candidate inspection result for each problem to be inspected corresponding to the target object in the image;

[0156] For each problem to be checked, the candidate checking results of each image corresponding to the problem to be checked are fused to obtain the checking result corresponding to the problem to be checked.

[0157] In a possible implementation, the method further includes:

[0158] Obtaining the current first location information of the road hidden danger inspection terminal and the second location information of the to-be-inspected location corresponding to the to-be-inspected problem for which no inspection result has been obtained;

[0159] Based on the first location information, the second location information and the distance threshold, the target supplementary location to be checked that is within the distance threshold is determined and sent to the road hazard inspection terminal, so that the road hazard inspection terminal displays the target supplementary location to be checked, so that the user of the road hazard inspection terminal can go to the target supplementary location to be checked to check the problems to be checked for which no inspection results have been obtained.

[0160] In a possible implementation manner, after determining that the target additional locations to be checked are within the distance threshold, the method further includes:

[0161] Determine the current moving speed of the road hidden danger inspection terminal;

[0162] Determine the target delay value corresponding to the moving rate according to the correspondence between different rates and delay values ​​stored in advance;

[0163] According to the current time and the target delay value, a supplementary location to be checked that is being removed is determined, and the supplementary location to be checked that is being removed is deleted from the target supplementary location to be checked.

[0164] Since the principle of the above-mentioned road hidden danger inspection method is similar to the principle of road hidden danger inspection performed by the road hidden danger inspection terminal and the server, the implementation of the above-mentioned road hidden danger inspection method can refer to the above-mentioned embodiment, and the repeated parts will not be repeated.

[0165] Based on the same inventive concept, the embodiment of the present application also provides a road hidden danger detection device, Fig.13A schematic diagram of a road hidden danger detection device provided in an embodiment of the present application, the device comprising:

[0166] Display module 1301, for displaying the acquired list of locations to be inspected if an inspection start request is received;

[0167] The acquisition module 1302 is used to acquire the on-site inspection record if an inspection instruction to inspect any target location to be inspected in the list of locations to be inspected is received;

[0168] The troubleshooting module 1303 is used to parse the content included in the on-site troubleshooting record based on each problem to be troubleshooted, obtain the troubleshooting result corresponding to each problem to be answered, and display it.

[0169] Based on the same inventive concept, an embodiment of the present application further provides an electronic device, Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig.14 As shown, it includes: a processor 1401, a communication interface 1402, a memory 1403 and a communication bus 1404, wherein the processor 1401, the communication interface 1402, and the memory 1403 communicate with each other through the communication bus 1404;

[0170] The memory 1403 stores a computer program, and when the program is executed by the processor 1401, the processor 1401 executes the steps of the road hazard detection method.

[0171] The communication bus mentioned in the above display device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface 1402 is used for communication between the above display device and other devices. The memory may include a random access memory (RAM) and may also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0172] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0173] Based on the above embodiments, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program executable by a processor. When the program runs on the processor, the processor implements the steps of the road hazard inspection method when executing.

[0174] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0175] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0178] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A road hidden danger inspection terminal, characterized in that: The terminal includes a display and a processor: The processor is configured to: If a request to start the investigation is received, the obtained list of places to be investigated is displayed on the display; If an inspection instruction is received to inspect any target location to be inspected in the list of locations to be inspected, an on-site inspection record is obtained and sent to the server, wherein the on-site inspection record includes at least one of an image, an audio, and a text; If each problem to be checked and the corresponding troubleshooting result are received, they are displayed on the display, wherein the troubleshooting result is determined by the server analyzing the content included in the on-site troubleshooting record based on each problem to be checked.

2. The terminal according to claim 1, characterized in that If the on-site investigation record is audio, the processor is further configured to: Performing speech recognition on the audio to obtain a recognition text corresponding to the audio, and displaying the recognition text on the display; if a text confirmation instruction is received, using the recognition text to update the on-site investigation record; or, Sending the audio to the server, allowing the server to perform speech recognition on the audio, obtaining a recognition text corresponding to the audio and feeding it back; if the recognition text sent by the server is received, displaying the recognition text on the display; If a text confirmation instruction is received, the text confirmation instruction is forwarded to the server, so that the server determines the recognized text as the on-site inspection record.

3. A server, characterized in that: is configured to execute: Receive on-site inspection records sent by the road hidden danger inspection terminal; Based on each problem to be checked, the content included in the on-site inspection record is parsed to obtain the inspection result corresponding to each problem to be checked, and each problem to be checked and the corresponding inspection result are sent to the road hazard inspection terminal.

4. The server according to claim 3, characterized in that: Specifically configured to execute: If the on-site inspection record is an image, according to the objects to be inspected corresponding to each problem to be inspected, identify the target object corresponding to each object to be inspected in the image, and whether the target object has any abnormality; Based on the big model, analyze whether there is an abnormality in the target object and each problem to be checked, and determine the troubleshooting result corresponding to each problem to be checked.

5. The server according to claim 3, characterized in that: If the on-site investigation record is audio, it is also configured to execute: Performing speech recognition on the audio to obtain a recognition text corresponding to the audio and sending the recognition text to the road hazard inspection terminal; If a text confirmation instruction is received, the on-site inspection record is updated using the identified text.

6. The server according to claim 3 or 5, characterized in that: Specifically configured to execute: If the on-site inspection record is in the form of text, the on-site inspection record, each problem to be inspected and the prompt text are input into the big model, so that the big model analyzes the on-site inspection record and each problem to be inspected based on the prompt of the prompt text, and determines the inspection result corresponding to each problem to be inspected.

7. The server according to claim 3, characterized in that: Specifically configured to execute: If the on-site inspection record is a plurality of images of the same target object, determining the overlapping area in all the images according to the content included in each image; Generating a complete target object image of the target object according to the target objects included in all the overlapping areas; Based on the large model, it is determined whether there is any abnormality in the target object in the complete target object image, and the troubleshooting result of the problem to be troubleshooted corresponding to the target object is determined.

8. The server according to claim 3, characterized in that: Specifically configured to execute: If the on-site inspection record is a plurality of images of the same target object, for each image, the image and each problem to be inspected are analyzed based on the large model to determine a candidate inspection result for each problem to be inspected corresponding to the target object in the image; For each problem to be checked, the candidate checking results of each image corresponding to the problem to be checked are fused to obtain the checking result corresponding to the problem to be checked.

9. The server according to claim 3, characterized in that: Also configured as: Acquire the current first location information of the road hidden danger inspection terminal and the second location information of the to-be-inspected location corresponding to the to-be-inspected problem for which no inspection result has been obtained; Based on the first location information, the second location information and the distance threshold, the target supplementary location to be checked that is within the distance threshold is determined and sent to the road hazard inspection terminal, so that the road hazard inspection terminal displays the target supplementary location to be checked, so that the user of the road hazard inspection terminal can go to the target supplementary location to be checked to check the problems to be checked for which no inspection results have been obtained.

10. The server according to claim 9, characterized in that Also configured to execute: Determine the current moving speed of the road hidden danger inspection terminal; Determine the target delay value corresponding to the moving rate according to the correspondence between different rates and delay values ​​stored in advance; According to the current time and the target delay value, a supplementary location to be checked that is being removed is determined, and the supplementary location to be checked that is being removed is deleted from the target supplementary location to be checked.

Images