Road inspection method and device, storage medium and electronic equipment

Abstract

The invention provides a road inspection method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a plurality of inspection points on a target road, determining a vehicle inspection position point according to each inspection point, the vehicle inspection position point being a position where an image collection device of an automatic driving vehicle can shoot the inspection point, the inspection points are in one-to-one correspondence with the vehicle inspection position points; adopting a heuristic search algorithm to calculate and obtain an inspection route at least according to all the vehicle inspection position points; and controlling the automatic driving vehicle to inspect the inspection points in sequence according to the inspection route to obtain corresponding inspection images, uploading all the inspection images to a back-end server to determine an inspection result, and inspecting the target road. The method has higher stability, can adapt to more extreme working environments and collect more stable detection data, and meanwhile, the automatic driving vehicle is provided with various abundant high-precision sensors, so that target defects and quality can be monitored more comprehensively.

Description

Technical Field

[0001] The present application relates to the field of inspection, and in particular, to a road inspection method, a road inspection device, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the continuous development of science and technology, high-precision scanning equipment such as high-resolution visible light cameras, lidar, millimeter-wave radar, etc. have greatly enriched the inspection methods of urban municipal pipelines.

[0003] In addition to the manual inspection mode, existing technical solutions generally use drones equipped with special sensors to inspect power transmission and distribution lines or bridges, and monitor and manage defects or quality through data collection, processing and analysis. Some existing solutions focus on intelligent defect analysis of distribution lines, or mainly focus on bridge construction quality supervision. These solutions use drones to obtain real-time data, generate three-dimensional models, and conduct deep learning analysis to monitor and manage target objects. Drone-based inspection technology is susceptible to extreme weather and environmental influences, and may encounter problems such as wind and electromagnetic interference during flight, increasing flight risks. At the same time, it is limited by the drone's aerial photography perspective and payload limitations, and it is impossible to obtain detailed and comprehensive target information. Summary of the invention

[0004] The main purpose of the present application is to provide a road inspection method, a road inspection device, a computer-readable storage medium and an electronic device, so as to at least solve the problem in the prior art that the use of drones for road inspection is easily affected by extreme weather, environment and electromagnetic interference, and cannot obtain detailed and comprehensive target information.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a road inspection method is provided, comprising: obtaining multiple inspection points on a target road, and confirming a vehicle inspection position point of an autonomous driving vehicle based on each of the inspection points, wherein the vehicle inspection position point is a position where an image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection point corresponds one-to-one to the vehicle inspection position point; using a heuristic search algorithm to calculate an inspection route based on at least all of the vehicle inspection position points; controlling the autonomous driving vehicle to inspect each of the inspection points in turn along the inspection route, obtaining an inspection image corresponding to each of the inspection points, and uploading all of the inspection images to a back-end server so that the back-end server determines the inspection result of each of the inspection points based on each of the inspection images, so as to inspect the target road.

[0006] Optionally, the inspection point is located in a non-motorized vehicle lane on the target road, multiple inspection points on the target road are obtained, and the vehicle inspection position point of the autonomous driving vehicle is confirmed based on each of the inspection points, including: using on-board OTA technology to receive the position coordinates of all the inspection points on the target road; based on the position coordinates of the inspection points, determining the position point in the motor vehicle lane on the target road where the image acquisition device can capture the inspection point, and determining the position point as the vehicle inspection position point corresponding to the inspection point.

[0007] Optionally, there are at least three inspection points, and a heuristic search algorithm is used to calculate an inspection route based on at least all the vehicle inspection position points, including: obtaining a pipeline line connecting two adjacent inspection points to obtain a plurality of pipeline lines; using an A* algorithm to determine the inspection sub-route of the autonomous driving vehicle between two adjacent inspection points based on two adjacent vehicle inspection position points and the line direction of the pipeline line between the corresponding two inspection points, the line direction of the pipeline line being the tangent direction of the pipeline line; and determining the inspection route of the autonomous driving vehicle based on all the inspection sub-routes.

[0008] Optionally, two adjacent inspection points are respectively a first inspection point and a second inspection point, the first inspection point is an inspection point inspected before the second inspection point, the first inspection point and the second inspection point correspond to a first vehicle inspection position point and a second vehicle inspection position point respectively, and an A* algorithm is used to determine an inspection sub-route of the autonomous driving vehicle between two adjacent inspection points according to the two adjacent vehicle inspection position points and the line direction of the pipeline line between the corresponding two inspection points, including: determining a current position point of the autonomous driving vehicle, the current position point being a position point passed by the autonomous driving vehicle when it moves from the first vehicle inspection position point to the second vehicle inspection position point at the current moment; determining a prepared driving route of the autonomous driving vehicle according to the current position point of the autonomous driving vehicle and a prepared position point of the autonomous driving vehicle at the next moment, the prepared driving route There are multiple backup position points, and the backup position points correspond to the backup driving routes one by one; the angle between the backup driving route of the autonomous driving vehicle and the route direction of the pipeline line is determined as the target angle; according to the size of the target angle, the path score of the backup driving route is determined, and the path score of the backup driving route is negatively correlated with the size of the target angle, and the target angle is 0°~180°; the heuristic function h*(x)=h(x)+Score(x,y) of the A* algorithm is obtained, wherein h*(x) is the heuristic function of the A* algorithm, h(x) is the estimated cost from the current position point x to the second inspection point, and Score(x,y) is the path score from the current position point x to the backup position point y; based on the heuristic function, the A* algorithm is used to determine the inspection sub-route of the autonomous driving vehicle between two adjacent inspection points.

[0009] Optionally, the autonomous driving vehicle is controlled to inspect each of the inspection points in turn along the inspection route to obtain an inspection image corresponding to each of the inspection points, including: positioning the autonomous driving vehicle in real time to obtain real-time position data of the autonomous driving vehicle; when the real-time position data of the autonomous driving vehicle is within a preset range, reducing the driving speed of the autonomous driving vehicle to a target driving speed, the preset range being a range within which the image acquisition device of the autonomous driving vehicle can capture the inspection points, and the target driving speed being a driving speed at which the clarity of the image captured by the image acquisition device of the autonomous driving vehicle is greater than a preset clarity; when the driving speed of the autonomous driving vehicle is the target driving speed, controlling the image acquisition device of the autonomous driving vehicle to capture an inspection image including the inspection points.

[0010] Optionally, the autonomous driving vehicle is positioned in real time to obtain real-time position data of the autonomous driving vehicle, including: using the computer vision technology of the autonomous driving vehicle to perform feature matching on feature objects in the surrounding area of ​​the autonomous driving vehicle to obtain a feature matching result, wherein the surrounding area is an area centered on the autonomous driving vehicle and with a preset length as a radius; based on the feature matching result, combined with a high-precision positioning algorithm, the autonomous driving vehicle is positioned in real time to obtain real-time position data of the autonomous driving vehicle, and the unit of the real-time position data of the autonomous driving vehicle is decimeter or centimeter.

[0011] Optionally, all of the inspection images are uploaded to a back-end server so that the back-end server determines the inspection results of each inspection point based on each inspection image to inspect the target road, including: uploading all of the inspection images to the back-end server so that the back-end server determines the inspection results of the corresponding inspection points based on a YOLOv8 deep learning model and the inspection images, the inspection results indicating that the inspection point is normal or abnormal, wherein the YOLOv8 deep learning model is trained using multiple sets of training data, each set of the multiple sets of training data includes: historical inspection images and historical inspection results of the inspection points corresponding to the historical inspection images, acquired within a historical time period.

[0012] According to another aspect of the present application, a road inspection device is provided, comprising: an acquisition unit, used to acquire multiple inspection points on a target road, and confirm a vehicle inspection position point of an autonomous driving vehicle based on each of the inspection points, wherein the vehicle inspection position point is a position where an image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection point corresponds one-to-one to the vehicle inspection position point; a calculation unit, used to calculate an inspection route based on at least all of the vehicle inspection position points using a heuristic search algorithm; and an inspection unit, used to control the autonomous driving vehicle to inspect each of the inspection points in turn along the inspection route, and obtain an inspection result corresponding to each of the inspection points, so as to inspect the target road.

[0013] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the road inspection methods.

[0014] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the road inspections.

[0015] Applying the technical solution of the present application, the inspection method of the above-mentioned road first obtains multiple inspection points on the target road, and confirms the vehicle inspection position point of the autonomous driving vehicle according to each inspection point. The vehicle inspection position point is the position where the image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection point corresponds to the vehicle inspection position point one by one; then a heuristic search algorithm is used to calculate the inspection route at least according to all the vehicle inspection position points; finally, the autonomous driving vehicle is controlled to inspect each inspection point in turn according to the inspection route, and the inspection image corresponding to each inspection point is obtained, and all the inspection images are uploaded to the back-end server so that the back-end server determines the inspection result of each inspection point according to each inspection image, so as to inspect the target road. This method solves the problem that the use of drones for road inspection in the prior art is easily affected by extreme weather, environment and electromagnetic interference, and cannot obtain detailed and comprehensive target information. Through autonomous driving path planning, autonomous driving vehicle planning and control, and multi-directional photography and real-time semantic uploading, a more stable and comprehensive inspection of urban pipelines is completed. Autonomous driving vehicles have greater stability, can adapt to more extreme working environments, and collect more stable detection data. At the same time, autonomous driving vehicles themselves have a variety of rich and high-precision sensors that can conduct more comprehensive monitoring of target defects and quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for performing a road inspection method provided in an embodiment of the present application is shown;

[0018] Figure 2 A schematic diagram of a process of a road inspection method provided according to an embodiment of the present application is shown;

[0019] Figure 3 A schematic diagram showing whether a manhole cover on a sidewalk is intact is provided according to an embodiment of the present application;

[0020] Figure 4 A schematic diagram showing a flow chart of another road inspection method provided according to an embodiment of the present application;

[0021] Figure 5 A structural block diagram of a road inspection device provided according to an embodiment of the present application is shown.

[0022] The above drawings include the following reference numerals:

[0023] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As introduced in the background technology, in addition to the manual inspection mode, the existing technical solutions are generally based on drones equipped with special sensors to inspect power transmission and distribution lines or bridges, and realize defect or quality monitoring and management through data collection, processing and analysis. However, the inspection technology based on drones is susceptible to extreme weather and environment, and may encounter problems such as wind and electromagnetic interference during flight, which increases the flight risk. At the same time, it is limited by the aerial photography perspective and load limitations of the drone, and it is impossible to obtain detailed and comprehensive target information. In order to solve the problem that the use of drones for road inspection in the existing technology is susceptible to extreme weather, environment and electromagnetic interference, and it is impossible to obtain detailed and comprehensive target information, the embodiments of the present application provide a road inspection method, a road inspection device, a computer-readable storage medium and an electronic device.

[0028] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a road inspection method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the road inspection method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a road inspection method running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 2 is a flow chart of a road inspection method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0033] Step S201, obtaining a plurality of inspection points on a target road, and confirming a vehicle inspection position point of the autonomous driving vehicle according to each of the inspection points, wherein the vehicle inspection position point is a position where an image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection points correspond to the vehicle inspection position points one by one;

[0034] Specifically, the inspection points on the target road and the vehicle inspection position points on the autonomous driving lane are obtained, and the above inspection points are matched one by one with the vehicle inspection position points to ensure that the on-board camera can effectively capture the inspection points, providing a basis for subsequent processing.

[0035] In one alternative, Figure 3 As shown, the road inspection method of the present application is used to detect whether the manhole covers on the sidewalk are intact, wherein the center points of multiple manhole covers on the sidewalk correspond to the inspection points on the target road, and there are multiple vehicle inspection positions on the autonomous driving lane. The inspection points are determined according to the municipal road pipeline inspection requirements, and the road proximity points, i.e., the vehicle inspection positions, are determined in combination with the road information and the inspection points to check and verify the road connectivity.

[0036] The inspection point is located in the non-motorized vehicle lane on the target road, and a plurality of inspection points on the target road are obtained, and the vehicle inspection position point of the autonomous driving vehicle is confirmed according to each of the inspection points, including the following steps:

[0037] Step S2011, using the vehicle-mounted OTA technology to receive the position coordinates of all the inspection points on the target road;

[0038] Step S2012, based on the position coordinates of the inspection point, determine the position point in the motor vehicle lane on the target road where the image acquisition device can capture the inspection point, and determine the position point as the vehicle inspection position point corresponding to the inspection point.

[0039] Specifically, the inspection points are sent to the operating autonomous vehicles through the vehicle OTA technology, and the corresponding vehicle inspection locations are determined to ensure that all inspection points can be inspected, which helps to complete the inspection tasks efficiently. Compared with the inspection method based on drones, autonomous vehicles are based on multi-sensor fusion technology and combined with software algorithms to adapt to more complex weather environments.

[0040] In addition, this application uses OTA and real-time semantic maps to send and report inspection data and collected information, but in fact, there are many ways to complete data transmission in self-driving cars, including but not limited to uploading to disk and reporting to the operation center. There are many ways to complete data transmission in self-driving cars, including but not limited to uploading to disk and reporting to the operation center.

[0041] Furthermore, combined with the high-precision map information of the autonomous driving, the vehicle inspection location points within the visible range of the inspection point are determined. The vehicle inspection location points refer to the three-dimensional coordinate points on the autonomous driving lane where the on-board camera can effectively capture the inspection point. Check whether the updated vehicle inspection location points are interconnected to ensure that the autonomous driving vehicle can automatically complete the pipeline inspection. The connectivity check process means starting from the starting point, finding the next vehicle inspection location point forward or backward along the direction of the pipeline, marking the nodes that have been visited, and finally determining whether all vehicle inspection location points are reachable to ensure that the autonomous driving vehicle can theoretically drive to the corresponding location.

[0042] Step S202, using a heuristic search algorithm to calculate an inspection route based on at least all of the above vehicle inspection position points;

[0043] Specifically, a heuristic search algorithm is used for navigation route planning to select the optimal path, saving time costs and improving inspection efficiency.

[0044] There are at least three inspection points, and a heuristic search algorithm is used to calculate an inspection route based on at least all of the vehicle inspection position points, including the following steps:

[0045] Step S301, obtaining a pipeline route connecting two adjacent inspection points to obtain a plurality of pipeline routes;

[0046] Step S302, using the A* algorithm, according to the two adjacent vehicle inspection position points and the line direction of the pipeline line between the two corresponding inspection points, determining the inspection sub-route of the autonomous driving vehicle between the two adjacent inspection points, where the line direction of the pipeline line is the tangent direction of the pipeline line;

[0047] Step S303: Determine the inspection route of the autonomous driving vehicle based on all the inspection sub-routes.

[0048] Specifically, the autonomous driving vehicle updates the vehicle inspection location points and generates the inspection navigation route in real time with the help of the on-board map. The A* algorithm based on the pipeline guide line can optimize the inspection route of the autonomous driving vehicle, save time costs and improve inspection efficiency. At the same time, automatic path planning further reduces manual intervention and improves the efficiency of daily inspections.

[0049] The two adjacent inspection points are respectively the first inspection point and the second inspection point, the first inspection point is the inspection point for inspection before the second inspection point, the first inspection point and the second inspection point correspond to the first vehicle inspection position point and the second vehicle inspection position point respectively, and the A* algorithm is used to determine the inspection sub-route of the autonomous driving vehicle between the two adjacent inspection points according to the two adjacent vehicle inspection position points and the line direction of the pipeline line between the two corresponding inspection points, including the following steps:

[0050] Step S401, determining the current position point of the autonomous driving vehicle, where the current position point is the position point that the autonomous driving vehicle passes through when moving from the first vehicle inspection position point to the second vehicle inspection position point at the current moment;

[0051] Step S402, determining a prepared driving route of the autonomous driving vehicle according to the current position of the autonomous driving vehicle and a prepared position of the autonomous driving vehicle at the next moment, wherein there are multiple prepared positions, and each of the prepared positions corresponds to the prepared driving route one by one;

[0052] Step S403, determining the angle between the prepared driving route of the autonomous driving vehicle and the route direction of the pipeline route as a target angle;

[0053] Step S404, determining a path score of the prepared driving route according to the size of the target angle, wherein the path score of the prepared driving route is negatively correlated with the size of the target angle, and the target angle is 0° to 180°;

[0054] Step S405, obtaining the heuristic function h*(x)=h(x)+Score(x,y) of the A* algorithm, wherein h*(x) is the heuristic function of the A* algorithm, h(x) is the estimated cost from the current position point x to the second inspection point, and Score(x,y) is the path score from the current position point x to the preparation position point y;

[0055] Step S406: Based on the above heuristic function, an A* algorithm is used to determine the inspection sub-route of the above-mentioned autonomous driving vehicle between two adjacent inspection points.

[0056] Specifically, the pipeline guide line is added to the heuristic function: Assume that the score function on the guide line is Score(x,y), which represents the score from node x to node y. The score of the guide line is added to the heuristic function of the A* algorithm to better guide path planning. Using the A* algorithm for route planning will connect the inspection points in a more reasonable way than manually specifying the driving route or daily navigation path planning.

[0057] In an optional solution, when inspecting the manhole covers on the sidewalk to see if they are intact, the pipeline guide line is the water pipe line between the two manhole covers. The next point on the path is selected by the heuristic function, and the A* algorithm is used to determine whether this point can reach the end point.

[0058] Step S203, control the autonomous driving vehicle to inspect each of the above inspection points in turn according to the above inspection route, obtain the inspection image corresponding to each of the above inspection points, and upload all of the above inspection images to the back-end server so that the above back-end server determines the inspection result of each of the above inspection points according to the above inspection images, so as to inspect the above target road.

[0059] Specifically, the self-driving vehicle itself carries a variety of high-resolution cameras, laser radars and other acquisition equipment, which can provide low-cost and high-quality inspections. In addition, the self-driving vehicle uses the self-driving system to adaptively control the speed during driving, and controls the speed to be reduced near the inspection point, providing a more stable shooting environment.

[0060] Among them, controlling the automatic driving vehicle to inspect each of the inspection points in turn according to the inspection route to obtain the inspection image corresponding to each of the inspection points includes the following steps:

[0061] Step S501, positioning the autonomous driving vehicle in real time to obtain real-time position data of the autonomous driving vehicle;

[0062] Step S502, when the real-time position data of the autonomous driving vehicle is within a preset range, reducing the driving speed of the autonomous driving vehicle to a target driving speed, the preset range being a range within which the image acquisition device of the autonomous driving vehicle can capture the inspection point, and the target driving speed being a driving speed at which the clarity of the image captured by the image acquisition device of the autonomous driving vehicle is greater than the preset clarity;

[0063] Step S503, when the driving speed of the above-mentioned autonomous driving vehicle is the above-mentioned target driving speed, control the image acquisition device of the above-mentioned autonomous driving vehicle to capture the inspection image including the above-mentioned inspection point.

[0064] Specifically, a complete autonomous driving system will simulate human behavior to drive the car, and the on-board camera can usually support 30 frames per second or higher, so there is no need to stop completely at the inspection point for inspection shooting. The inspection mode of the autonomous driving can be set, and the speed of the vehicle can be controlled based on the distance between the shooting point and the inspection point. By predicting other moving road participants on the road, the autonomous driving car can be controlled to maintain a fixed, low speed through the inspection point. The data collection of the inspection point is completed with the help of the high-precision positioning technology of the autonomous driving system and the front, side and rear cameras on the vehicle.

[0065] The real-time positioning of the autonomous driving vehicle to obtain the real-time position data of the autonomous driving vehicle includes the following steps:

[0066] Step S601, using the computer vision technology of the autonomous driving vehicle to perform feature matching on feature objects in the surrounding area of ​​the autonomous driving vehicle to obtain a feature matching result, wherein the surrounding area is an area with the autonomous driving vehicle as the center and a preset length as the radius;

[0067] Step S602: Based on the feature matching results and in combination with a high-precision positioning algorithm, the autonomous driving vehicle is positioned in real time to obtain real-time position data of the autonomous driving vehicle. The unit of the real-time position data of the autonomous driving vehicle is decimeter or centimeter.

[0068] Specifically, the positioning results of self-driving cars will be an order of magnitude more accurate than traditional GPS positioning. With the assistance of higher-precision positioning results, the camera can more accurately judge the distance and direction to the inspection point, thereby selecting a closer shooting position and direction, and can also better evaluate the timing of shooting under motion.

[0069] The autonomous driving vehicle positioning system uses the continuous image sequence obtained by the vehicle camera and the odometer information to perform visual odometer calculations to estimate the vehicle's motion trajectory and coordinates in the world coordinate system. At the same time, it uses computer vision technology to detect and match feature points to track landmarks, road signs or other feature objects, and then determine the vehicle's position in the world coordinate system. Its positioning accuracy is higher than that of traditional GPS. Among them, GPS high-precision positioning algorithms include slam algorithms, multi-frame sensor fusion, and lidar feature maps (high-precision maps at the decimeter level).

[0070] Among them, all the above-mentioned inspection images are uploaded to the back-end server so that the above-mentioned back-end server determines the inspection results of each of the above-mentioned inspection points according to each of the above-mentioned inspection images, so as to inspect the above-mentioned target road, including: uploading all the above-mentioned inspection images to the above-mentioned back-end server so that the above-mentioned back-end server determines the above-mentioned inspection results of the corresponding inspection points according to the YOLOv8 deep learning model and the above-mentioned inspection images, and the above-mentioned inspection results are that the inspection point is normal or that the inspection point is abnormal, wherein the above-mentioned YOLOv8 deep learning model is trained using multiple groups of training data, and each group of the above-mentioned multiple groups of training data includes: historical inspection images and historical inspection results of the above-mentioned inspection points corresponding to the above-mentioned historical inspection images, which are obtained within a historical time period.

[0071] Specifically, the real-time semantic map update module is used to upload the real-time collected image data to the back-end central server after desensitization, and the inspection data is organized and saved in the form of time, space, and image information. Desensitization specifically refers to the coding of sensitive information such as faces and license plates.

[0072] Among them, based on YOLOv8's deep learning technology and image processing technology, the images are detected and classified. First, data preparation is carried out, and the historically reported image data is manually annotated into three categories: complete, damaged, and missing. The annotated images are adjusted to the corresponding size and format; then the loss function is defined, and the predicted value output by the model is processed by the Sigmoid activation function, and then the loss is calculated by BCELoss; then the model parameters are initialized, and the parameters of the YOLOv8 model are initialized; then the downsampling module structure is optimized, and a dual-branch parallel method is adopted. One branch uses a space-to-depth layer and a convolutional layer without a specific step size. This branch implements downsampling while retaining the channel dimension features as much as possible, and the other branch uses maximum pooling to reduce the size of the feature map. The YOLOv8 model is trained using the training data set, and the model parameters are updated through the back-propagation algorithm to minimize the loss function of the model on the training data set.

[0073] Figure 4 A specific flow chart of the road inspection method is given.

[0074] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0075] The embodiment of the present application also provides a road inspection device. It should be noted that the road inspection device of the embodiment of the present application can be used to execute the road inspection method provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0076] The following is an introduction to the road inspection device provided in an embodiment of the present application.

[0077] Figure 5 Schematic diagram of a road inspection device according to an embodiment of the present application. Figure 5As shown, the device includes: an acquisition unit 10, a calculation unit 20 and an inspection unit 30, the acquisition unit 10 is used to acquire multiple inspection points on the target road, and confirm the vehicle inspection position point of the autonomous driving vehicle according to each of the above inspection points, the above vehicle inspection position point is a position where the image acquisition device of the above autonomous driving vehicle can capture the inspection point, and the above inspection points correspond to the above vehicle inspection position points one by one; the calculation unit 20 is used to use a heuristic search algorithm to calculate an inspection route based on at least all of the above vehicle inspection position points; the inspection unit 30 is used to control the autonomous driving vehicle to inspect each of the above inspection points in turn according to the above inspection route, and obtain the inspection results corresponding to each of the above inspection points, so as to inspect the above target road.

[0078] The inspection device for the above-mentioned road of the present application includes an acquisition unit, a calculation unit and an inspection unit. The acquisition unit is used to acquire multiple inspection points on the target road, and confirm the vehicle inspection position point of the autonomous driving vehicle according to each inspection point. The vehicle inspection position point is the position where the image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection point corresponds to the vehicle inspection position point one by one; the calculation unit is used to use a heuristic search algorithm to calculate the inspection route according to at least all the vehicle inspection position points; the inspection unit is used to control the autonomous driving vehicle to inspect each inspection point in turn according to the inspection route, obtain the inspection results corresponding to each inspection point, and inspect the target road. The device solves the problem that the use of drones for road inspection in the prior art is easily affected by extreme weather, environment and electromagnetic interference, and cannot obtain detailed and comprehensive target information. Through autonomous driving path planning, autonomous driving vehicle planning and control, and multi-directional photography and real-time semantic uploading, a more stable and comprehensive inspection of urban pipelines is completed. Autonomous driving vehicles have greater stability, can adapt to more extreme working environments, and collect more stable detection data. At the same time, autonomous driving vehicles themselves have a variety of rich and high-precision sensors that can conduct more comprehensive monitoring of target defects and quality.

[0079] In some instances, the acquisition unit includes a first acquisition module and a second acquisition module, wherein the first acquisition module is used to receive the position coordinates of all the inspection points on the target road using the vehicle-mounted OTA technology; the second acquisition module is used to determine the position point in the motor vehicle lane on the target road where the image acquisition device can capture the inspection point based on the position coordinates of the inspection point, and determine the position point as the vehicle inspection position point corresponding to the inspection point. The inspection point is sent to the operating autonomous driving vehicle through the vehicle-mounted OTA technology, and the corresponding vehicle inspection position point is determined to ensure that all inspection points can be detected, which helps to complete the inspection task efficiently.

[0080] In some examples, the calculation unit includes a first calculation module, a second calculation module and a third calculation module. The first calculation module is used to obtain the pipeline line connecting the two adjacent inspection points to obtain multiple pipeline lines. The second calculation module is used to use the A* algorithm to determine the inspection sub-route of the autonomous driving vehicle between the two adjacent inspection points according to the line direction of the pipeline line between the two adjacent vehicle inspection positions and the corresponding two inspection points. The line direction of the pipeline line is the tangent direction of the pipeline line. The third calculation module is used to determine the inspection route of the autonomous driving vehicle according to all the inspection sub-routes. In this way, the inspection route of the autonomous driving vehicle can be optimized, time cost can be saved and inspection efficiency can be improved. At the same time, automatic path planning further reduces manual intervention and improves the efficiency of daily inspections.

[0081] In some instances, the computing unit further includes a fourth computing module, a fifth computing module, a sixth computing module, a seventh computing module, an eighth computing module and a ninth computing module, wherein the fourth computing module is used to determine the current position point of the above-mentioned autonomous driving vehicle, wherein the above-mentioned current position point is the position point passed by the above-mentioned autonomous driving vehicle when it moves from the above-mentioned first vehicle inspection position point to the above-mentioned second vehicle inspection position point at the current moment; the fifth computing module is used to determine the prepared driving route of the above-mentioned autonomous driving vehicle according to the current position point of the above-mentioned autonomous driving vehicle and the prepared position point of the above-mentioned autonomous driving vehicle at the next moment, wherein there are multiple prepared position points, and the above-mentioned prepared position points correspond one-to-one to the above-mentioned prepared driving routes; the sixth computing module is used to calculate the distance between the prepared driving route of the above-mentioned autonomous driving vehicle and the route direction of the above-mentioned pipeline line. The angle is determined as the target angle; the seventh calculation module is used to determine the path score of the above-mentioned prepared driving route according to the size of the above-mentioned target angle, and the path score of the above-mentioned prepared driving route is negatively correlated with the size of the above-mentioned target angle, and the above-mentioned target angle is 0°~180°; the eighth calculation module is used to obtain the heuristic function h*(x)=h(x)+Score(x,y) of the above-mentioned A* algorithm, wherein h*(x) is the heuristic function of the above-mentioned A* algorithm, h(x) is the estimated cost from the above-mentioned current position point x to the above-mentioned second inspection point, and Score(x,y) is the path score from the above-mentioned current position point x to the above-mentioned prepared position point y; the ninth calculation module is used to determine the inspection sub-route of the above-mentioned autonomous driving vehicle between the two adjacent inspection points based on the above-mentioned heuristic function and using the A* algorithm. Compared with manually specifying the driving route or daily navigation path planning, the route planning using the A* algorithm will connect the inspection points in series in a more reasonable way.

[0082] In some examples, the inspection unit includes a first inspection module, a second inspection module and a third inspection module. The first inspection module is used to locate the autonomous driving vehicle in real time and obtain the real-time location data of the autonomous driving vehicle. The second inspection module is used to reduce the driving speed of the autonomous driving vehicle to a target driving speed when the real-time location data of the autonomous driving vehicle is within a preset range. The preset range is the range in which the image acquisition device of the autonomous driving vehicle can capture the inspection point. The target driving speed is the driving speed at which the clarity of the image captured by the image acquisition device of the autonomous driving vehicle is greater than the preset clarity. The third inspection module is used to control the image acquisition device of the autonomous driving vehicle to capture the inspection image including the inspection point when the driving speed of the autonomous driving vehicle is the target driving speed. A perfect autonomous driving system will simulate human behavior to drive the car. At the same time, the on-board camera can usually support 30 frames per second or higher. There is no need to stop completely at the inspection point for inspection shooting, which has higher efficiency and more stable data collection effect.

[0083] In some examples, the inspection unit also includes a fourth inspection module and a fifth inspection module. The fourth inspection module is used to use the computer vision technology of the above-mentioned autonomous driving vehicle to perform feature matching on characteristic objects in the surrounding area of ​​the above-mentioned autonomous driving vehicle to obtain feature matching results. The above-mentioned surrounding area is an area with the above-mentioned autonomous driving vehicle as the center and a preset length as the radius; the fifth inspection module is used to locate the above-mentioned autonomous driving vehicle in real time based on the above-mentioned feature matching results in combination with a high-precision positioning algorithm to obtain the real-time position data of the above-mentioned autonomous driving vehicle. The unit of the real-time position data of the above-mentioned autonomous driving vehicle is decimeter or centimeter. With the assistance of higher-precision positioning results, the camera can more accurately judge the distance and direction from the inspection point, thereby selecting a closer shooting position and direction, and can also better evaluate the shooting timing under motion.

[0084] In some instances, the inspection unit is also used to upload all of the above inspection images to the back-end server so that the back-end server determines the inspection results of each of the above inspection points based on the above inspection images to inspect the target road, including: uploading all of the above inspection images to the back-end server so that the back-end server determines the above inspection results of the corresponding inspection points based on the YOLOv8 deep learning model and the above inspection images, and the above inspection results are that the inspection point is normal or abnormal, wherein the above YOLOv8 deep learning model is trained using multiple sets of training data, and each set of training data in the multiple sets of training data includes: historical inspection images and historical inspection results of the inspection points corresponding to the above historical inspection images acquired within a historical time period. The YOLOv8 model is trained using the training data set, and the model parameters are updated through the back-propagation algorithm to minimize the loss function of the model on the training data set.

[0085] The above-mentioned road inspection device includes a processor and a memory, and the above-mentioned acquisition unit and the like are stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0086] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problem that the use of drones for road inspection in the prior art is easily affected by extreme weather, environment and electromagnetic interference, and cannot obtain detailed and comprehensive target information.

[0087] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0088] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the road inspection method.

[0089] Specifically, the road inspection methods include:

[0090] Step S201, obtaining a plurality of inspection points on a target road, and confirming a vehicle inspection position point of the autonomous driving vehicle according to each of the inspection points, wherein the vehicle inspection position point is a position where an image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection points correspond to the vehicle inspection position points one by one;

[0091] Step S202, using a heuristic search algorithm to calculate an inspection route based on at least all of the above vehicle inspection position points;

[0092] Step S203, control the autonomous driving vehicle to inspect each of the above inspection points in turn according to the above inspection route, obtain the inspection image corresponding to each of the above inspection points, and upload all of the above inspection images to the back-end server so that the above back-end server determines the inspection result of each of the above inspection points according to the above inspection images, so as to inspect the above target road.

[0093] Optionally, the inspection point is located in a non-motorized vehicle lane on the target road, multiple inspection points on the target road are obtained, and the vehicle inspection position point of the autonomous driving vehicle is confirmed based on each of the inspection points, including: using on-board OTA technology to receive the position coordinates of all the inspection points on the target road; based on the position coordinates of the inspection points, determining the position point in the motor vehicle lane on the target road where the image acquisition device can capture the inspection point, and determining the position point as the vehicle inspection position point corresponding to the inspection point.

[0094] Optionally, there are at least three inspection points, and a heuristic search algorithm is used to calculate an inspection route based on at least all of the vehicle inspection position points, including: obtaining a pipeline line connecting two adjacent inspection points to obtain a plurality of pipeline lines; using an A* algorithm to determine an inspection sub-route of the autonomous driving vehicle between two adjacent inspection points based on two adjacent vehicle inspection position points and the line direction of the pipeline line between the corresponding two inspection points, the line direction of the pipeline line being the tangent direction of the pipeline line; and determining the inspection route of the autonomous driving vehicle based on all of the inspection sub-routes.

[0095] Optionally, the two adjacent inspection points are respectively the first inspection point and the second inspection point, the first inspection point is the inspection point inspected before the second inspection point, the first inspection point and the second inspection point correspond to the first vehicle inspection position point and the second vehicle inspection position point respectively, and the A* algorithm is used to determine the inspection sub-route of the autonomous driving vehicle between the two adjacent inspection points according to the two adjacent vehicle inspection position points and the line direction of the pipeline line between the corresponding two inspection points, including: determining the current position point of the autonomous driving vehicle, the current position point being the position point passed by the autonomous driving vehicle when it moves from the first vehicle inspection position point to the second vehicle inspection position point at the current moment; determining the prepared driving route of the autonomous driving vehicle according to the current position point of the autonomous driving vehicle and the prepared position point of the autonomous driving vehicle at the next moment, the prepared driving route of the autonomous driving vehicle There are multiple backup position points, and the above-mentioned backup position points correspond to the above-mentioned backup driving routes one by one; the angle between the backup driving route of the above-mentioned autonomous driving vehicle and the route direction of the above-mentioned pipeline line is determined as the target angle; according to the size of the above-mentioned target angle, the path score of the above-mentioned backup driving route is determined, and the path score of the above-mentioned backup driving route is negatively correlated with the size of the above-mentioned target angle, and the above-mentioned target angle is 0°~180°; the heuristic function h*(x)=h(x)+Score(x,y) of the above-mentioned A* algorithm is obtained, wherein h*(x) is the heuristic function of the above-mentioned A* algorithm, h(x) is the estimated cost from the above-mentioned current position point x to the above-mentioned second inspection point, and Score(x,y) is the path score from the above-mentioned current position point x to the above-mentioned backup position point y; based on the above-mentioned heuristic function, the A* algorithm is used to determine the inspection sub-route of the above-mentioned autonomous driving vehicle between two adjacent inspection points.

[0096] Optionally, the autonomous driving vehicle is controlled to inspect each of the above-mentioned inspection points in turn according to the above-mentioned inspection route to obtain an inspection image corresponding to each of the above-mentioned inspection points, including: positioning the above-mentioned autonomous driving vehicle in real time to obtain real-time position data of the above-mentioned autonomous driving vehicle; when the real-time position data of the above-mentioned autonomous driving vehicle is within a preset range, reducing the driving speed of the above-mentioned autonomous driving vehicle to a target driving speed, the above-mentioned preset range is a range within which the image acquisition device of the above-mentioned autonomous driving vehicle can capture the above-mentioned inspection points, and the above-mentioned target driving speed is a driving speed at which the clarity of the image captured by the image acquisition device of the above-mentioned autonomous driving vehicle is greater than the preset clarity; when the driving speed of the above-mentioned autonomous driving vehicle is the above-mentioned target driving speed, controlling the image acquisition device of the above-mentioned autonomous driving vehicle to capture the inspection image including the above-mentioned inspection points.

[0097] Optionally, the autonomous driving vehicle is positioned in real time to obtain real-time position data of the autonomous driving vehicle, including: using computer vision technology of the autonomous driving vehicle to perform feature matching on feature objects in the surrounding area of ​​the autonomous driving vehicle to obtain feature matching results, wherein the surrounding area is an area centered on the autonomous driving vehicle and with a preset length as a radius; based on the feature matching results, combined with a high-precision positioning algorithm, the autonomous driving vehicle is positioned in real time to obtain real-time position data of the autonomous driving vehicle, and the unit of the real-time position data of the autonomous driving vehicle is decimeter or centimeter.

[0098] Optionally, all of the above-mentioned inspection images are uploaded to the back-end server so that the above-mentioned back-end server determines the inspection results of each of the above-mentioned inspection points based on the above-mentioned inspection images, so as to inspect the above-mentioned target road, including: uploading all of the above-mentioned inspection images to the above-mentioned back-end server so that the above-mentioned back-end server determines the above-mentioned inspection results of the corresponding inspection points based on the YOLOv8 deep learning model and the above-mentioned inspection images, and the above-mentioned inspection results are that the inspection point is normal or that the inspection point is abnormal, wherein the above-mentioned YOLOv8 deep learning model is trained using multiple groups of training data, and each group of the above-mentioned multiple groups of training data includes: historical inspection images and historical inspection results of the above-mentioned inspection points corresponding to the above-mentioned historical inspection images, which are obtained within a historical time period.

[0099] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the inspection method of the road is executed when the program is run.

[0100] Specifically, the road inspection methods include:

[0101] Step S201, obtaining a plurality of inspection points on a target road, and confirming a vehicle inspection position point of the autonomous driving vehicle according to each of the inspection points, wherein the vehicle inspection position point is a position where an image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection points correspond to the vehicle inspection position points one by one;

[0102] Step S202, using a heuristic search algorithm to calculate an inspection route based on at least all of the above vehicle inspection position points;

[0103] Step S203, control the autonomous driving vehicle to inspect each of the above inspection points in turn according to the above inspection route, obtain the inspection image corresponding to each of the above inspection points, and upload all of the above inspection images to the back-end server so that the above back-end server determines the inspection result of each of the above inspection points according to the above inspection images, so as to inspect the above target road.

[0104] An embodiment of the present invention provides a device, the device comprising a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0105] Step S201, obtaining a plurality of inspection points on a target road, and confirming a vehicle inspection position point of the autonomous driving vehicle according to each of the inspection points, wherein the vehicle inspection position point is a position where an image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection points correspond to the vehicle inspection position points one by one;

[0106] Step S202, using a heuristic search algorithm to calculate an inspection route based on at least all of the above vehicle inspection position points;

[0107] Step S203, control the autonomous driving vehicle to inspect each of the above inspection points in turn according to the above inspection route, obtain the inspection image corresponding to each of the above inspection points, and upload all of the above inspection images to the back-end server so that the above back-end server determines the inspection result of each of the above inspection points according to the above inspection images, so as to inspect the above target road.

[0108] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0109] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0110] Step S201, obtaining a plurality of inspection points on a target road, and confirming a vehicle inspection position point of the autonomous driving vehicle according to each of the inspection points, wherein the vehicle inspection position point is a position where an image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection points correspond to the vehicle inspection position points one by one;

[0111] Step S202, using a heuristic search algorithm to calculate an inspection route based on at least all of the above vehicle inspection position points;

[0112] Step S203, control the autonomous driving vehicle to inspect each of the above inspection points in turn according to the above inspection route, obtain the inspection image corresponding to each of the above inspection points, and upload all of the above inspection images to the back-end server so that the above back-end server determines the inspection result of each of the above inspection points according to the above inspection images, so as to inspect the above target road.

[0113] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0114] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0115] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0116] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0118] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0119] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0120] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0121] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0122] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0123] 1) The inspection method of the above-mentioned road of the present application first obtains multiple inspection points on the target road, and confirms the vehicle inspection position point of the autonomous driving vehicle according to each inspection point. The vehicle inspection position point is the position where the image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection point corresponds to the vehicle inspection position point one by one; then the heuristic search algorithm is used to calculate the inspection route at least according to all the vehicle inspection position points; finally, the autonomous driving vehicle is controlled to inspect each inspection point in turn according to the inspection route, and the inspection image corresponding to each inspection point is obtained, and all the inspection images are uploaded to the back-end server so that the back-end server determines the inspection result of each inspection point according to each inspection image, so as to inspect the target road. This method solves the problem that the road inspection using drones in the prior art is easily affected by extreme weather, environment and electromagnetic interference, and cannot obtain detailed and comprehensive target information. Through autonomous driving path planning, autonomous driving vehicle planning and control, and multi-directional photography and real-time semantic uploading, a more stable and comprehensive inspection of urban pipelines is completed. Autonomous driving vehicles have greater stability, can adapt to more extreme working environments, and collect more stable detection data. At the same time, autonomous driving vehicles themselves have a variety of rich and high-precision sensors that can conduct more comprehensive monitoring of target defects and quality.

[0124] 2) The inspection device for the above-mentioned road of the present application includes an acquisition unit, a calculation unit and an inspection unit. The acquisition unit is used to acquire multiple inspection points on the target road, and confirm the vehicle inspection position point of the autonomous driving vehicle according to each inspection point. The vehicle inspection position point is the position where the image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection point corresponds to the vehicle inspection position point one by one; the calculation unit is used to use a heuristic search algorithm to calculate the inspection route based on at least all the vehicle inspection position points; the inspection unit is used to control the autonomous driving vehicle to inspect each inspection point in turn according to the inspection route, obtain the inspection results corresponding to each inspection point, and inspect the target road. The device solves the problem that the use of drones for road inspection in the prior art is easily affected by extreme weather, environment and electromagnetic interference, and cannot obtain detailed and comprehensive target information. Through autonomous driving path planning, autonomous driving vehicle planning and control, and multi-directional photography and real-time semantic uploading, a more stable and comprehensive inspection of urban pipelines is completed. Autonomous driving vehicles have greater stability, can adapt to more extreme working environments, and collect more stable detection data. At the same time, autonomous driving vehicles themselves have a variety of rich and high-precision sensors that can conduct more comprehensive monitoring of target defects and quality.

[0125] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A road inspection method, characterized in that: include: Acquire multiple inspection points on the target road, and confirm the vehicle inspection position point of the autonomous driving vehicle according to each of the inspection points, wherein the vehicle inspection position point is a position where the inspection point can be photographed by the image acquisition device of the autonomous driving vehicle, and the inspection point corresponds to the vehicle inspection position point one by one; Using a heuristic search algorithm to calculate an inspection route based on at least all of the vehicle inspection location points; Control the autonomous driving vehicle to inspect each of the inspection points in turn along the inspection route, obtain the inspection image corresponding to each of the inspection points, and upload all of the inspection images to the back-end server so that the back-end server determines the inspection result of each of the inspection points based on the inspection images to inspect the target road.

2. The method according to claim 1, characterized in that The inspection point is located in a non-motorized vehicle lane on the target road, a plurality of inspection points on the target road are obtained, and a vehicle inspection position point of the autonomous driving vehicle is confirmed according to each of the inspection points, including: Using vehicle-mounted OTA technology to receive the location coordinates of all the inspection points on the target road; According to the position coordinates of the inspection point, a position point in the motor vehicle lane on the target road where the image acquisition device can capture the inspection point is determined, and the position point is determined as the vehicle inspection position point corresponding to the inspection point.

3. The method according to claim 1, characterized in that There are at least three inspection points, and a heuristic search algorithm is used to calculate an inspection route based on at least all of the vehicle inspection position points, including: Acquire a pipeline route connecting two adjacent inspection points to obtain a plurality of pipeline routes; Adopting the A* algorithm, according to two adjacent vehicle inspection position points and the line direction of the pipeline line between the corresponding two inspection points, determining the inspection sub-route of the autonomous driving vehicle between two adjacent inspection points, the line direction of the pipeline line being the tangent direction of the pipeline line; The inspection route of the autonomous driving vehicle is determined based on all the inspection sub-routes.

4. The method according to claim 3, characterized in that The two adjacent inspection points are respectively a first inspection point and a second inspection point, the first inspection point is an inspection point for inspection before the second inspection point, the first inspection point and the second inspection point correspond to a first vehicle inspection position point and a second vehicle inspection position point respectively, and an A* algorithm is used to determine an inspection sub-route of the autonomous driving vehicle between the two adjacent inspection points according to the two adjacent vehicle inspection position points and the line direction of the pipeline line between the corresponding two inspection points, including: Determine a current position point of the autonomous driving vehicle, where the current position point is a position point that the autonomous driving vehicle passes through when moving from the first vehicle inspection position point to the second vehicle inspection position point at the current moment; Determine a prepared driving route of the autonomous driving vehicle according to a current position of the autonomous driving vehicle and a prepared position of the autonomous driving vehicle at a next moment, wherein there are multiple prepared position points, and the prepared position points correspond to the prepared driving routes one by one; Determine the angle between the prepared driving route of the autonomous driving vehicle and the route direction of the pipeline route as a target angle; Determining a path score of the prepared driving route according to the size of the target angle, wherein the path score of the prepared driving route is negatively correlated with the size of the target angle, and the target angle is 0° to 180°; Obtain the heuristic function h*(x)=h(x)+Score(x,y) of the A* algorithm, wherein h*(x) is the heuristic function of the A* algorithm, and h(x) is the estimated cost from the current position point x to the second inspection point. Score(x,y) is the path score from the current position point x to the preliminary position point y; Based on the heuristic function, the A* algorithm is used to determine the inspection sub-route of the autonomous driving vehicle between two adjacent inspection points.

5. The method according to claim 1, characterized in that Controlling the automatic driving vehicle to inspect each inspection point in turn according to the inspection route to obtain an inspection image corresponding to each inspection point, including: Positioning the autonomous driving vehicle in real time to obtain real-time position data of the autonomous driving vehicle; When the real-time position data of the autonomous driving vehicle is within a preset range, reducing the driving speed of the autonomous driving vehicle to a target driving speed, wherein the preset range is a range within which the image acquisition device of the autonomous driving vehicle can capture the inspection point, and the target driving speed is a driving speed at which the clarity of the image captured by the image acquisition device of the autonomous driving vehicle is greater than the preset clarity; When the driving speed of the autonomous driving vehicle is the target driving speed, the image acquisition device of the autonomous driving vehicle is controlled to capture an inspection image including the inspection point.

6. The method according to claim 5, characterized in that Positioning the autonomous driving vehicle in real time to obtain real-time position data of the autonomous driving vehicle includes: Using computer vision technology of the autonomous driving vehicle to perform feature matching on feature objects in a surrounding area of ​​the autonomous driving vehicle to obtain a feature matching result, wherein the surrounding area is an area with the autonomous driving vehicle as the center and a preset length as the radius; According to the feature matching result, combined with a high-precision positioning algorithm, the autonomous driving vehicle is positioned in real time to obtain real-time position data of the autonomous driving vehicle. The unit of the real-time position data of the autonomous driving vehicle is decimeter or centimeter.

7. The method according to claim 1, characterized in that Uploading all the inspection images to a back-end server so that the back-end server determines the inspection result of each inspection point according to each inspection image to inspect the target road, including: All the inspection images are uploaded to the back-end server, so that the back-end server determines the inspection result of the corresponding inspection point according to the YOLOv8 deep learning model and the inspection image, and the inspection result is that the inspection point is normal or abnormal, wherein the YOLOv8 deep learning model is trained using multiple sets of training data, and each set of training data in the multiple sets of training data includes: historical inspection images and historical inspection results of the inspection points corresponding to the historical inspection images acquired within a historical time period.

8. A road inspection device, characterized in that: include: An acquisition unit, used to acquire a plurality of inspection points on a target road, and confirm a vehicle inspection position point of the autonomous driving vehicle according to each of the inspection points, wherein the vehicle inspection position point is a position where an image acquisition device of the autonomous driving vehicle can capture the inspection point, and the inspection point corresponds to the vehicle inspection position point one by one; A calculation unit, configured to calculate an inspection route based on at least all of the vehicle inspection position points by using a heuristic search algorithm; The inspection unit is used to control the autonomous driving vehicle to inspect each of the inspection points in turn along the inspection route, obtain the inspection results corresponding to each of the inspection points, and inspect the target road.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the road inspection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the road inspection method described in any one of claims 1 to 7.

Images