Road night lighting information determination method and device, electronic equipment and storage medium

Abstract

The invention provides a road night lighting information determination method and device, electronic equipment and a storage medium, and relates to the technical fields of unmanned aerial vehicle aerial photography, map generation, automatic driving navigation and the like. The method comprises the following steps: determining each target area lacking night lighting information; based on the representative points of each target area, generating a complete graph in which a connection relationship is established between any two representative points; determining a flight planning route based on the complete graph and a preset path optimization algorithm; obtaining an orthographic aerial image shot by the unmanned aerial vehicle flying according to the flight planning route; and determining night illumination information of the target area based on brightness information contained in the orthographic aerial image. According to the method, the flight height advantage of the unmanned aerial vehicle is fully utilized, and an orthographic aerial image of night brightness information is conveniently obtained by arranging a reasonable flight planning path, so that an automatic driving service with higher safety is provided under the condition of increasing night illumination information of each road, and the personal and property safety of night driving is better guaranteed.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, specifically to technical fields such as drone aerial photography, map generation, and autonomous driving navigation, and more particularly to a method, device, electronic device, computer-readable storage medium, and computer program product for determining nighttime road lighting information. Background Art

[0002] In night travel scenarios, good lighting conditions are of great value in ensuring personal safety, avoiding traffic accidents, and calming the emotions of pedestrians. Summary of the invention

[0003] The embodiments of the present disclosure provide a method, device, electronic device, computer-readable storage medium, and computer program product for determining nighttime road lighting information.

[0004] In a first aspect, an embodiment of the present disclosure proposes a method for determining road night lighting information, including: determining each target area that lacks night lighting information; based on representative points of each target area, generating a complete graph in which a connection relationship is established between any two representative points; based on the complete graph and a preset path optimization algorithm, determining a flight planning route; obtaining orthophoto aerial images taken by a drone flying along the flight planning route; and determining the night lighting information of the target area based on brightness information contained in the orthophoto aerial images.

[0005] In the second aspect, an embodiment of the present disclosure proposes a device for determining road night lighting information, including: a target area determination unit, configured to determine each target area that lacks night lighting information; a complete graph generation unit, configured to generate a complete graph with a connection relationship established between any two representative points based on representative points of each target area; a flight route planning unit, configured to determine the flight planning route based on the complete graph and a preset path optimization algorithm; an orthophoto aerial image acquisition unit, configured to acquire orthophoto aerial images taken by a drone flying along the flight planning route; and a night lighting information determination unit, configured to determine the night lighting information of the target area based on the brightness information contained in the orthophoto aerial images.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor, when executing, can implement the method for determining road night lighting information as described in any implementation method in the first aspect.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement a method for determining road nighttime lighting information as described in any implementation method in the first aspect when executed.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the steps of the method for determining road nighttime lighting information as described in any implementation manner in the first aspect.

[0009] The road night lighting information determination scheme provided by the present disclosure, for the target area lacking night lighting information, first abstracts the area into representative points, then constructs a complete graph by connecting the representative points, and then determines the flight planning route suitable for UAV flight based on the complete graph and the preset path optimization algorithm, so that the UAV can obtain the required orthophoto aerial image of the target area by shooting according to the flight planning route, and finally determine the night lighting brightness of each target area from the brightness information contained in the orthophoto aerial image. That is, this scheme makes full use of the UAV's flight altitude advantage, and conveniently obtains orthophoto aerial images of night brightness information by arranging a reasonable flight planning path for it, so as to provide a safer automatic driving service and better protect the personal and property safety of night driving while increasing the night lighting information of each road.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0012] Figure 1 is an exemplary system architecture in which the present disclosure may be applied;

[0013] Figure 2 A flowchart of a method for determining road nighttime lighting information provided by an embodiment of the present disclosure;

[0014] Figure 3 A flowchart of a method for determining a target area based on a user trajectory provided by an embodiment of the present disclosure;

[0015] Figure 4 A flowchart of a method for generating a flight planning route and obtaining orthophoto aerial images provided in an embodiment of the present disclosure;

[0016] Figure 5 A flowchart of a method for obtaining nighttime lighting information of each lane and target area through lane-level data provided by an embodiment of the present disclosure;

[0017] Figure 6 A flowchart of another method for obtaining nighttime lighting information of each lane and target area through lane-level data provided by an embodiment of the present disclosure;

[0018] Figure 7 A structural block diagram of a device for determining nighttime road lighting information provided by an embodiment of the present disclosure;

[0019] Figure 8 A schematic diagram of the structure of an electronic device suitable for executing a method for determining road nighttime lighting information provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.

[0021] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0022] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method, apparatus, electronic device, and computer-readable storage medium for determining nighttime road lighting information of the present disclosure may be applied.

[0023] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, a server 103, a network 104, and a drone 105. The network 104 is used to provide a medium for communication links between different terminal devices 101, 102 and the server 103, and between the server 103 and the drone 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0024] The terminal devices 101 and 102 carried by the user can record the user's relevant information under the premise of user authorization, and send it to the server 103 through the network 104. The server 103 can further generate relevant instructions for controlling the drone 105 based on the analysis and processing of the received user's relevant information, so as to control the drone 105 to fly and receive relevant images taken by the drone 105 during the flight. The terminal devices 101 and 102, the server 103 and the drone 105 can be installed with various applications for realizing information communication between the two, such as trajectory recording and collection applications, road night lighting information determination applications, drone flight control applications, etc.

[0025] The terminal devices 101, 102, the server 103 and the drone 105 are usually hardware, and can also be software in a specific scenario (such as a simulation scenario). When the terminal devices 101 and 102 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, servers or server clusters, etc.; when the terminal devices 101 and 102 are software, they can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here; when the server 103 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; when the server is software, it can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here; when the drone 105 is hardware, it can be implemented as different models of drone devices with flight and shooting capabilities; when the drone 105 is software, it can be implemented as software or software modules integrated with flight and shooting functions, or as a single software or software module, which is not specifically limited here.

[0026] The server 103 can provide various services through various built-in applications. Taking the road night lighting information determination application that can provide road night lighting information determination service as an example, the server 105 can achieve the following effects when running the road night lighting information determination application: first, based on the relevant user trajectories recorded in the terminal devices 101 and 102 and the historical map information recorded locally received through the network 104, the target areas where night lighting information is missing are determined; then, based on the representative points of each target area, a complete graph with a connection relationship established between any two representative points is generated; then, based on the complete graph and a preset path optimization algorithm, a flight planning route is determined; next, the flight planning route is sent to the drone 103 through the network 104, and then the orthophoto aerial image taken by the drone 103 along the flight planning route is received; finally, based on the brightness information contained in the orthophoto aerial image, the night lighting information of the target area is determined.

[0027] It should be noted that the relevant user trajectories recorded in the terminal devices 101 and 102 can be obtained from the terminal devices 101 and 102 through the network 104, or can be pre-stored in the server 103 in various ways. Therefore, when the server 103 detects that the data has been stored locally (for example, starting to process the tasks to be processed before), it can choose to directly obtain the data locally. In this case, the exemplary system architecture 100 may not include the terminal devices 101 and 102.

[0028] Since the generation of road network data requires more computing resources and stronger computing power, the road nighttime lighting information determination method provided in the subsequent embodiments of the present disclosure is generally performed by the server 103 with stronger computing power and more computing resources. Accordingly, the road nighttime lighting information determination device is generally also set in the server 103. However, it should also be pointed out that when the terminal devices 101 and 102 also have computing power and computing resources that meet the requirements, the terminal devices 101 and 102 can also complete the above-mentioned various operations that are handed over to the server 103 through the road nighttime lighting information determination application installed thereon, and then output the same result as the server 103. Especially in the case where there are multiple terminal devices with different computing powers at the same time, but the road nighttime lighting information determination application determines that the terminal device has stronger computing power and more remaining computing resources, the terminal device can be allowed to perform the above-mentioned operations, thereby appropriately reducing the computing pressure of the server 103. Accordingly, the road nighttime lighting information determination device can also be set in the terminal devices 101 and 102. In this case, the exemplary system architecture 100 may also not include the server 103 and the network 104.

[0029] It should be understood that Figure 1The number of terminal devices, networks, servers, and drones in the embodiment is only for illustration. Any number of terminal devices, networks, servers, and drones may be provided as required.

[0030] Please refer to Figure 2 , Figure 2 A flowchart of a method for determining nighttime road lighting information provided by an embodiment of the present disclosure, wherein process 200 includes the following steps:

[0031] Step 201: Determine each target area lacking nighttime lighting information;

[0032] This step is intended to be performed by the execution subject (e.g. Figure 1 The server 103 shown in the figure determines the area that lacks the night lighting information that meets the requirements. For the convenience of the following description, the area that lacks the night lighting information that meets the requirements is referred to as the target area.

[0033] Among them, the night lighting information in this disclosure refers to the lighting brightness provided by an area, section, road or lane at night, which is used to ensure the safe travel of objects traveling in the area, section, road or lane, and can be used as an additional road network data for the road. Road network data generally refers to data that describes the road system and its connection relationship, including information such as the name, type, location, length, intersection, number of lanes, and types of different lanes of the road. These data are generally used in applications such as traffic planning, navigation systems, and geographic information systems to help analyze and optimize traffic flows and routes, and can also be used to support autonomous driving services for autonomous vehicles. By adding night lighting information, it is possible to fully combine the night lighting information of different roads for the night driving part in the subsequent navigation or autonomous driving process to reasonably plan safer routes or trajectories to fully protect the safety of people and property.

[0034] If any night lighting information has never been collected in a certain area (for example, a road that has not yet installed street lights, or street lights have been installed but not yet turned on), it can be considered that the area lacks night lighting information that meets the requirements. If night lighting information has been collected in a certain area before but lacks freshness (for example, it has not been updated for more than 3 months or longer, for example, the street lights are damaged or have not been turned on recently, etc.), then the area can also be considered to lack night lighting information that meets the requirements.

[0035] Specifically, the target area can be determined by screening and determining from a map library or road administration information, or by exclusive determination based on the conditions in which the non-target area has night lighting information that meets the requirements, or by reverse deduction at the practical level based on whether there are sections in the user's trajectory where the travel speed suddenly changes due to poor night lighting.

[0036] An implementation method including but not limited to may be:

[0037] An area that has not recorded any nighttime lighting information may be determined as a target area, and an area that has recorded nighttime lighting information but has not updated the last recorded nighttime lighting information within a preset time period may also be determined as a target area.

[0038] Step 202: Based on the representative points of each target area, a complete graph is generated in which a connection relationship is established between any two representative points;

[0039] On the basis of step 201, this step aims to generate a complete graph in which a connection relationship is established between any two representative points based on the representative points of each target area by the above-mentioned execution subject. That is, the representative point is a point that can represent the corresponding target area. Specifically, any point in the area that can represent the area can be used as the representative point, such as the center point or mass point of the area. The edge connecting any two representative points not only refers to the connection relationship, but also can reflect the path and path length between the objects represented by the two points in a suitable form in the required scenario, for example, by matching the edge length of the path length, or by assigning different distance weights to the edge when it cannot be reflected by different edge lengths. The path length can be calculated based on the plane straight line distance or by using the spherical straight line. On the basis of the distance weight, considering that different moving objects and different moving types will have different safety under the same night lighting brightness, for example, lanes can be divided into: motor vehicle lanes, non-motor vehicle lanes and sidewalks according to type, corresponding to car driving, bicycles, electric vehicle driving and people walking, respectively, and night lighting is more important for walking and riding. Therefore, on the basis of the distance weight, different priority weights can be set for different types of lanes in the target area (used to characterize the importance priority of night lighting to different types of lanes), so as to better calculate a reasonable flight planning route based on the combination of distance weight and priority weight. Specifically, the priority weight can be expressed as follows: from low priority to high priority, the weight decreases by 0.2 for each level increase; from high priority to low priority, the additional weight increases by 0.2 for each level decrease; at the same time, the weight of the highest priority road remains unchanged, and the weight increases by 0.2 for each lower priority level of the road.

[0040] Among them, the complete graph is a basic concept in graph theory. It is an undirected graph in which each pair of different vertices is connected by an edge. In other words, a complete graph is a graph with an edge between each pair of vertices, and all vertices are connected by edges. As the basic graph type in graph theory, the complete graph plays an important role in many application fields due to its complete edge connection characteristics. It is the basis of many theoretical and algorithmic studies, and it can help understand the structural properties of graphs and the complexity of algorithms.

[0041] This step aims to construct a complete graph by connecting the representative points of each target area, so as to generate interconnected paths between all areas through the complete graph, so as to facilitate subsequent path optimization. For example, find the shortest loop so that the drone can visit all cities and pass through only once. This scheme is used in this step to find the optimal path between different areas, and the same principle can also be used to find the optimal path between sub-areas within a region.

[0042] An implementation method including but not limited to may be:

[0043] First, a representative point of each target area is determined, and the representative point may include: a center point or a mass point; then, any two representative points are connected to obtain a complete graph in which any two representative points are connected by an edge, and the edge has a weight matching the distance between the two connected representative points, so as to reflect the distance between the two areas corresponding to the two connected representative points through the size of the weight; and the edge may also have a priority weight matching the difference in lane priority between the two connected representative points, and the types of lanes include motor vehicle lanes, non-motor vehicle lanes and sidewalks, the sidewalk has the highest priority, the motor vehicle lane has the lowest priority, the non-motor vehicle lane has a priority between the sidewalk and the motor vehicle lane, and the number of lanes of each type includes multiple.

[0044] Step 203: Determine the flight planning route based on the complete graph and a preset path optimization algorithm;

[0045] Based on step 202, this step aims to determine the flight planning route by the above-mentioned execution subject based on the complete graph and the preset path optimization algorithm. Since the path between any two representative points has been presented in the complete graph by constructing edges, a suitable path optimization algorithm is used to find a flight planning path suitable for the UAV flight to pass through as many representative points as possible at one time in the shortest route based on the complete graph.

[0046] Among them, the path optimization algorithm aims to find the best path from the starting point to the end point in the graph, usually optimized according to the shortest distance or the lowest cost. For example, Dijkstra algorithm (principle: starting from the starting point, gradually expanding to the unvisited vertices in the graph, selecting the vertices of the current shortest path for processing until the end point is found. Applicable to graphs with non-negative edge weights), Bellman-Ford algorithm (principle: through multiple relaxation operations, the shortest path estimate of each vertex in the graph is updated, which can handle negative weight edges, but cannot handle negative weight cycles), Floyd-Warshall algorithm (principle: using dynamic programming methods, by updating the shortest paths between all vertex pairs in the graph to calculate the shortest paths of all vertex pairs. Applicable to graphs with negative edge weights but no negative weight cycles), simulated annealing algorithm (simulated annealing algorithm is a heuristic algorithm used to find the optimal solution, especially suitable for dealing with combinatorial optimization problems), etc.

[0047] Step 204: obtaining orthophoto aerial images taken by the drone flying along the planned flight route;

[0048] Based on step 203, this step aims to obtain the orthophoto aerial images taken by the drone flying along the flight planning route by the above-mentioned execution subject. Specifically, to achieve the purpose of this step, the above-mentioned execution subject can first send the flight planning path to the drone in the task execution state, so that the drone flies according to the received flight planning path, and uses the carried camera to complete the image shooting of the passing area during the flight.

[0049] Among them, orthophoto aerial images are geometrically corrected aerial or drone-photographed images that accurately reflect the actual surface conditions geographically. This type of image eliminates distortion caused by shooting angles and terrain undulations, ensuring that each point in the image is aligned with the actual geographic coordinate system. The result is an image with true scale and accurate geographic location that can be used for mapping, geographic information system analysis, road network information collection, and spatial planning.

[0050] Step 205: Determine the nighttime lighting information of the target area based on the brightness information contained in the orthophoto aerial image.

[0051] On the basis of step 204, this step aims to enable the above-mentioned execution subject to obtain the nighttime lighting information of each target area through appropriate analysis and processing based on the image content related to nighttime brightness contained in the orthophoto aerial image.

[0052] Specifically, considering that the orthophoto aerial images contain information related to nighttime brightness that can determine the nighttime lighting information of different roads, in order to better extract and more efficiently obtain the nighttime lighting information of each target area, especially when the orthophoto aerial images contain images of all target areas, it is inevitable that some images that do not require road network data will be included when passing through different target areas, and the images of each target area will also contain some images that are irrelevant to determining the nighttime lighting information. Therefore, the images of each target area can be first split by segmentation, and then some key image content related to determining the nighttime brightness of the road can be extracted from the images of each target area, and then the key image content can be combined to perform preliminary confirmation and correction of the nighttime lighting information, and finally obtain accurate nighttime lighting information of each target area. Furthermore, in this process, for a target area containing multiple lanes, it can also be split into images corresponding to each lane before extracting the road network data, and then the corresponding nighttime lighting information can be determined for each lane image.

[0053] The method for determining nighttime lighting information of roads provided by the disclosed embodiment, for target areas lacking nighttime lighting information, first abstracts the area into representative points, then constructs a complete graph by connecting the representative points, and then determines a flight planning route suitable for UAV flight based on the complete graph and a preset path optimization algorithm, so that the UAV can obtain the required orthophoto aerial images of the target area by shooting according to the flight planning route, and finally determines the nighttime lighting brightness of each target area from the brightness information contained in the orthophoto aerial images. That is, this solution makes full use of the UAV's flight altitude advantage, and conveniently obtains orthophoto aerial images of nighttime brightness information by arranging a reasonable flight planning path for it, so as to provide a safer automatic driving service and better protect the personal and property safety of nighttime driving while increasing the nighttime lighting information of each road.

[0054] To deepen the understanding of how to determine the target area, this embodiment also Figure 3 Another implementation is provided, that is, how to determine the target area based on the user trajectory, wherein process 300 includes the following steps:

[0055] Step 301: determining that there is an abnormal trajectory in the user trajectory in which the travel speed decreases by more than a preset degree;

[0056] This step aims to collect and determine the abnormal trajectories in the user trajectories where the speed reduction exceeds the preset level by the above-mentioned execution subject. The reason why the feature of the sharp reduction in speed is used to screen abnormal trajectories is that when the moving object travels from a road section with high brightness at night to a road section with low brightness at night, it will not be able to accurately see the details of the road in the front field of vision due to the weakening of brightness, and then reduce the speed for safety to observe carefully. Therefore, the present disclosure uses this feature to reversely screen the target area.

[0057] Step 302: clustering the abnormal trajectories to obtain fused trajectories;

[0058] On the basis of step 301, this step aims to perform clustering processing on abnormal trajectories by the above-mentioned execution subject to obtain fused trajectories.

[0059] Specifically, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm can be used, and 100,000 times the distance between trajectory points is used as the radius distance of the e-neighborhood to cluster the trajectories in the trajectory set, so that similar trajectories are aggregated into one, and the PV (position and velocity) change information is added.

[0060] Step 303: determining the area in the fused trajectory corresponding to the part of the trajectory where the speed is reduced and the speed is maintained after the reduction as the target area;

[0061] On the basis of step 302, this step aims to determine the area in the fusion trajectory corresponding to the part of the trajectory where the speed is reduced and the speed is maintained after the reduction as the target area by the above-mentioned execution subject. That is, the area where the speed is reduced corresponds to the starting part of the target area, the part where the speed is maintained after the reduction corresponds to the extension part starting from the starting part, and the end part can correspond to the part where the speed is restored.

[0062] An implementation method including but not limited to may be:

[0063] The part of the fusion trajectory where the speed is reduced and the speed is maintained after the reduction is determined as the speed change trajectory; the speed change trajectory is convex-hulled to obtain a convex hull; based on the area covered by the convex hull, the target area is determined. Specifically, the centroid of the convex hull can be first determined, and then the target area is obtained by taking the centroid as the area center and the preset length as the area radius. Of course, other representative points of the convex hull can also be used as the area center and other lengths as the radius to obtain a similar target area, and the target area may not be a circle, but may be other more appropriate shapes that better match the actual situation.

[0064] Among them, the main purpose of convex hull technology when processing trajectory data is to simplify and optimize data representation: by generating the convex hull (the smallest convex polygon) of the trajectory, reducing the number of complex trajectory data points and retaining key boundary information. The role is to reduce the size of the data set, make data processing and storage more efficient, and accurately extract the spatial boundaries of the trajectory to facilitate the analysis of trajectory coverage. It has the following advantages: simplified data reduces computational complexity and improves analysis speed; simplified data is easier to visualize, helping to intuitively understand the overall shape and coverage area of ​​the trajectory; in spatial analysis, path planning and regional monitoring, reducing redundant points improves processing accuracy and efficiency. That is, when processing and analyzing trajectory data, convex hull technology helps to better understand and apply trajectory information by removing redundant details.

[0065] This embodiment provides a method for determining a target area that lacks nighttime lighting information based on a user trajectory through steps 301 to 303, so that the target area can be reversely inferred based on the feedback of the user trajectory, which can complement the method of forwardly determining the target area to make the determined target area more accurate and comprehensive.

[0066] Based on any of the above embodiments, this embodiment further comprises: Figure 4 A method for generating a flight planning route and obtaining orthophoto aerial images is provided to deepen the understanding of how to obtain a flight planning route based on a complete graph path optimization algorithm and then obtain an orthophoto aerial image. The process 400 includes the following steps:

[0067] Step 401: Based on the distance weight, priority weight and simulated annealing algorithm of the edge connecting each representative point in the complete graph, a complete flight planning route passing through all representative points is calculated;

[0068] This step aims to calculate a complete flight planning route passing through all representative points based on the distance weights, priority weights and simulated annealing algorithm of the edges connecting each representative point in the complete graph by the above-mentioned execution subject.

[0069] Among them, the distance weight of the edge connecting each representative point is used to reflect the actual distance of the path, and the priority weight is used to reflect the demand for night lighting of this section of road. When the simulated annealing algorithm is applied to the path optimization problem of the complete graph, simulated annealing can be used to find a path that passes through all destinations and is as short as possible, so as to avoid exceeding the maximum flight distance of the drone.

[0070] The general process of finding the shortest path based on a complete graph using the simulated annealing algorithm can be seen in the following points:

[0071] 1) Initialization: Randomly generate an initial path (visit all vertices in a loop).

[0072] 2) Define the objective function:

[0073] The objective function is usually the total length or total cost of the path. The length of the current path is calculated as an evaluation criterion for the current solution.

[0074] 3) Generate neighborhood solutions: Make small random adjustments to the current path, such as swapping the positions of two vertices in the path, to generate a new path.

[0075] 4) Calculate the objective function value of the new solution: calculate the length or cost of the new path.

[0076] 5) Acceptance criterion: If the new path is better (shorter in length), then accept the new path as the current path. If the new path is worse, then accept it with a certain probability to avoid falling into the local optimal solution. The acceptance probability usually depends on the current temperature and the degree of degradation of the solution.

[0077] 6) Gradually lower the temperature: The temperature controls the probability of accepting a worse solution. As the algorithm proceeds, the temperature is gradually lowered to reduce the possibility of accepting an inferior solution and eventually converge to a better solution.

[0078] 7) Termination condition: When the temperature drops to a very low level or after a certain number of iterations, the algorithm terminates and returns the optimal path currently found.

[0079] That is, the simulated annealing algorithm helps to explore possible solutions and find better solutions in the path optimization of the complete graph through the combination of randomness and step-by-step optimization.

[0080] Step 402: In response to the planned flight distance of the complete flight planning route exceeding the maximum flight distance of the UAV, the complete flight planning route is split into at least two sub-flight planning routes that only pass through some representative points according to the maximum flight distance;

[0081] This step is based on the situation that the planned flight distance of the complete flight planning route calculated in step 401 exceeds the maximum flight distance of the drone. Since it exceeds the maximum flight distance of the drone, it means that the drone cannot pass through all target areas in a single flight. Therefore, at least one drone needs to fly twice to obtain orthophoto aerial images of all target areas. This step aims to split the complete flight planning route into at least two sub-flight planning routes that only pass through some representative points according to the maximum flight distance, and the sum of the number of times each sub-flight planning route passes through any representative point is not less than 1, that is, after being split into multiple sub-flight planning routes, for the sake of safety, the areas passed by different flight planning routes can partially overlap. Furthermore, it can also be ensured that each target area is passed through twice through multiple flights, that is, the orthophoto aerial images of each target area taken during two flights are obtained, so as to avoid as much as possible the possible omissions or missing of key information due to single flight shooting.

[0082] Specifically, when constructing a complete flight planning route or splitting it into a sub-flight planning route, the starting point should be the target area closest to the drone's take-off point or a lane in the target area. From then on, the shortest path tree is constructed. The stopping condition for this tree construction is that the path depth is 40% of the drone's cruising range. If the shortest path is less than 40% of the cruising range, the shortest path is used directly, and the path with the smallest total weight is selected as the starting path. The weight of this path is then set to infinity, and the shortest route is planned from the target area at the end of the path or the lane in the target area to the drone airport. If the shortest route length exceeds 50% of the cruising range, the drone flies directly back to the airport, and no information is collected on the return trip.

[0083] Step 403: Obtain orthophoto aerial images taken by the drone flying along different sub-flight planning routes in sequence.

[0084] Based on step 402, this step aims to enable the above-mentioned execution subject to obtain the orthophoto aerial images taken by the drone flying along different sub-flight planning routes in sequence, that is, when the route has been split into at least two sub-flight planning routes, the same drone can be controlled to fly and shoot successively along different sub-flight planning routes.

[0085] This embodiment provides a solution for calculating a complete flight planning route based on a complete graph and a simulated annealing algorithm through steps 401 to 403, and further splits the complete flight planning path when the flight distance corresponding to the complete flight planning route exceeds the maximum flight distance of the drone, so that the drone can finally obtain orthophoto aerial images of all target areas through multiple consecutive flight shootings. That is, the advantage of the simulated annealing algorithm in calculating the shortest path is fully utilized.

[0086] On the basis of any of the above embodiments, in order to deepen the understanding of how to determine the road network data of each target area according to the orthophoto aerial image, this embodiment further Figure 5 A method for obtaining nighttime lighting information of a target area through lane-level data is provided, wherein a process 500 includes the following steps:

[0087] Step 501: Determine lane boundaries between lanes based on lane-level data of the target area;

[0088] This step aims to first determine the lane dividing lines between lanes according to the lane-level data of the target area by the execution subject, that is, using the lane dividing lines of multiple lanes contained in the target area provided by the lane-level road network data as a segmentation reference.

[0089] Step 502: extracting lane aerial images corresponding to each lane from the orthophoto aerial image using lane demarcation lines;

[0090] On the basis of step 501, this step aims at extracting lane aerial images corresponding to each lane from the orthophoto aerial images by the above-mentioned execution subject using lane dividing lines.

[0091] Step 503: Based on the light source brightness information contained in the aerial image of each lane, determine the nighttime lighting information of the corresponding lane.

[0092] Specifically, the lane boundary lines of each lane in the target area are extracted based on the lane-level road network data, and processed into two boundary lines, one on each side of the road of the lane, so as to segment the images of different lanes with the help of the boundary lines. Furthermore, on the basis of obtaining the lane boundary lines, the center polyline of each lane can also be confirmed based on the lane-level road network data. The center polyline can also be used as the flight route of the drone when flying through this lane, thereby further ensuring that the drone collects the correct orthophoto information of the road.

[0093] Step 504: Determine the nighttime lighting information of the target area according to the nighttime lighting information of each lane included in the target area.

[0094] On the basis of step 503, this step aims to determine the nighttime lighting information of the target area according to the nighttime lighting information of each lane contained in the target area by the above-mentioned execution subject. That is, the nighttime lighting information of the target area can be obtained comprehensively according to the nighttime lighting information of each lane contained in the target area.

[0095] This embodiment provides a solution through steps 501 to 504 for using lane-level road network data to assist in segmenting the aerial images of each lane, and further determining the night lighting information of the corresponding lane based on the aerial images of each lane, and finally obtaining the night lighting information of the target area. Accurate segmentation can improve the accuracy of the final night lighting information of each lane and the target area.

[0096] In the above Figure 5 On the basis of the embodiment shown, in order to further deepen the understanding of the specific implementation process, this embodiment also Figure 6 A more specific implementation is provided, wherein the process 600 includes the following steps:

[0097] Step 601: extracting lane aerial images corresponding to each lane from the orthophoto aerial image using lane dividing lines;

[0098] This step is consistent with step 502 in process 500 and will not be described again here.

[0099] Step 602: preprocessing the lane aerial image to obtain a preprocessed image;

[0100] This step is intended to enable the above-mentioned execution subject to perform preprocessing operations on the lane aerial image that can highlight the key content used for determining more accurate night lighting information in subsequent steps, such as enhancing contrast, removing noise, etc., to obtain a preprocessed image.

[0101] Step 603: converting the preprocessed image into a grayscale image, and determining a brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method;

[0102] Among them, the typical example of the maximum inter-class variance method is the OTSU algorithm, which is a classic method for image binarization. Its core idea is to segment the grayscale image into foreground and background by optimizing the threshold. The core principle of this algorithm is to select an optimal threshold to divide the grayscale of the image into foreground and background so that the between-class variance between the foreground and background is maximized. This maximized between-class variance corresponds to the optimal segmentation effect.

[0103] A specific implementation method may be:

[0104] First, the number of pixels at each preset grayscale level is determined according to the grayscale image; then, the number of pixels, grayscale mean and variance of the foreground and background at each grayscale level are calculated respectively; then, the inter-class variance at different grayscale levels is calculated using the maximum inter-class variance method, and the brightness threshold for segmenting the foreground and background is determined according to the maximum inter-class variance. That is, the brightness threshold is the value that can maximize the inter-class variance, and then the brightness threshold is used to segment the image into the foreground and the background to complete the binarization process. For the solution provided in this embodiment, it is divided into an illuminated area and a non-illuminated area.

[0105] Step 604: determining an illumination area based on a brightness threshold, and determining light source information according to the illumination area;

[0106] On the basis of step 603, this step aims at the execution subject determining the illumination area based on the brightness threshold, and determining the light source information according to the illumination area.

[0107] Among them, the illuminated area can be obtained by performing a connected domain analysis on each pixel point with an actual grayscale value exceeding the brightness threshold. To put it simply, the areas covered by the same type of pixels with similar distances can be connected; the light source information can be determined based on the shape of the illuminated area and the propagation characteristics of light. The light source information can be subdivided into: the position and movement characteristics of the light source, and can also include area information.

[0108] Step 605: excluding non-street light sources based on the light source information to obtain street light sources;

[0109] On the basis of step 604, this step aims to obtain street light sources by excluding non-street light sources based on the light source information by the above-mentioned execution subject. For example, a light source whose position exceeds a preset distance from the lane and / or whose moving characteristic is moving is determined as a non-street light source, and then other light sources that are different from non-street light sources are determined as street light sources.

[0110] The reason why non-street light sources are excluded to obtain street light sources is that the headlights of a passing or started vehicle are also a light source, which will briefly provide moving illumination. This type of short-term and unstable illumination cannot serve as a stable light source that can provide safety protection and a certain visibility for objects traveling on the corresponding lanes. Therefore, it needs to be excluded as an interference object in the present disclosure.

[0111] Furthermore, in addition to excluding non-street light sources, the drone can also be controlled not to take aerial images during periods when there are vehicles with their headlights turned on, and to hover in place during pauses in filming until the vehicle leaves the filming field of view before resuming filming and continuing to move forward. Of course, such operation of the drone requires full use of its remaining energy.

[0112] A specific implementation method can be: first, build a nighttime moving vehicle target detection dataset, and the detection dataset labels are divided into moving vehicles and non-moving vehicles, i.e., roadside parked vehicles, in nighttime scenes. For this dataset, build a nighttime moving vehicle target detection model based on YOLO (a target detection model) to analyze the vehicle operation status in the field of view; then, relying on the nighttime moving vehicle detection model, in order to avoid the lighting information in the field of view being interfered by the light source from the vehicle, if there is a moving vehicle in the field of view, the collection is suspended until there is no moving vehicle in the field of view; then, after generating the orthophoto image along the road, the left and right boundary line segments of the road are used for mask extraction to obtain a refined road surface image and eliminate non-road pixel interference.

[0113] Step 606: Determine the brightness information of the illuminated area provided by the street light source as the nighttime lighting information of the corresponding lane.

[0114] On the basis of step 605, this step aims to determine the brightness information of the illuminated area provided by the street light source as the nighttime lighting information of the corresponding lane by the above-mentioned execution subject.

[0115] This embodiment provides a specific implementation method through steps 601 to 606, which not only improves the presentation effect of brightness information in the image through preprocessing operations, but also uses the maximum inter-class variance method to segment the illuminated area and the non-illuminated area, and further determines the light source according to the illuminated area and eliminates the interference of non-street light sources, thereby accurately obtaining the night lighting information of the corresponding lane based on the remaining street light sources.

[0116] Further references Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for determining road nighttime lighting information. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0117] like Figure 7As shown, the road nighttime lighting information determination device 700 of this embodiment may include: a target area determination unit 701, a complete graph generation unit 702, a flight route planning unit 703, an aerial image acquisition unit 704, and a nighttime lighting information determination unit 705. Among them, the target area determination unit 701 is configured to determine each target area that lacks nighttime lighting information; the complete graph generation unit 702 is configured to generate a complete graph with a connection relationship established between any two representative points based on representative points of each target area; the flight route planning unit 703 is configured to determine the flight planning route based on the complete graph and a preset path optimization algorithm; the aerial image acquisition unit 704 is configured to obtain the orthophoto aerial image taken by the drone flying along the flight planning route; the nighttime lighting information determination unit 705 is configured to determine the nighttime lighting information of the target area based on the brightness information contained in the orthophoto aerial image.

[0118] In this embodiment, in the road nighttime lighting information determination device 700, the specific processing of the target area determination unit 701, the complete map generation unit 702, the flight route planning unit 703, the aerial image acquisition unit 704, and the nighttime lighting information determination unit 705 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of steps 201 - 205 in the corresponding embodiment are not repeated here.

[0119] In some optional implementations of this embodiment, the target area determining unit 701 may be further configured to:

[0120] The area where no nighttime lighting information has been recorded is identified as the target area;

[0121] An area that has recorded nighttime lighting information but has not updated the last recorded nighttime lighting information within a preset time period is determined as a target area.

[0122] In some optional implementations of this embodiment, the target area determining unit 701 may include:

[0123] an abnormal trajectory determination subunit, configured to determine that there is an abnormal trajectory in the user trajectory in which the degree of reduction in travel speed exceeds a preset degree;

[0124] The trajectory fusion subunit is configured to perform clustering on the abnormal trajectories to obtain fused trajectories;

[0125] The target region determination subunit is configured to determine, as the target region, a region in the fused trajectory corresponding to a portion of the trajectory where the speed is reduced and the speed after the reduction is maintained.

[0126] In some optional implementations of this embodiment, the target area determination subunit 701 may include:

[0127] The speed change trajectory determination module is configured to determine the portion of the fused trajectory where the speed reduction occurs and the speed after the reduction is maintained as the speed change trajectory;

[0128] A convex hull processing module is configured to perform convex hull processing on the speed change trajectory to obtain a convex hull;

[0129] The target area determination module is configured to determine the target area based on the area covered by the convex hull.

[0130] In some optional implementations of this embodiment, the target area determination module may be further configured to:

[0131] Determine the centroid of the convex hull;

[0132] Take the centroid as the area center and the preset length as the area radius to get the target area

[0133] In some optional implementations of this embodiment, the complete graph generating unit 702 may be further configured to:

[0134] Determine the representative point of each lane contained in each target area; wherein the representative point includes: a center point or a mass point;

[0135] Connect any two representative points to obtain a complete graph in which any two representative points are connected by an edge; wherein the edge has a distance weight that matches the distance between the two connected representative points.

[0136] In some optional implementations of this embodiment, the edge also has a priority weight that matches the difference in lane priorities between the two connected representative points; wherein the types of lanes include motor vehicle lanes, non-motor vehicle lanes and sidewalks, the sidewalks have the highest priority, the motor vehicle lanes have the lowest priority, the non-motor vehicle lanes have a priority between the sidewalks and the motor vehicle lanes, and the number of lanes of each type includes multiple.

[0137] In some optional implementations of this embodiment, the flight route planning unit 703 may be further configured to:

[0138] Based on the distance weight, priority weight and simulated annealing algorithm of the edges connecting each representative point in the complete graph, the complete flight planning route passing through all representative points is calculated.

[0139] In some optional implementations of this embodiment, the road nighttime lighting information determination device 700 may further include:

[0140] The flight route splitting unit is configured to split the complete flight planning route into at least two sub-flight planning routes that only pass through some representative points according to the maximum flight distance in response to the planned flight distance of the complete flight planning route exceeding the maximum flight distance of the drone; wherein the sum of the number of times each sub-flight planning route passes through any representative point is not less than 1;

[0141] Correspondingly, the aerial image acquisition unit 704 may be further configured as follows:

[0142] Obtain orthophoto aerial images taken by the UAV flying along different sub-flight planning routes in sequence.

[0143] In some optional implementations of this embodiment, the nighttime lighting information determining unit 705 may include:

[0144] a lane dividing line determination subunit, configured to determine a lane dividing line between lanes based on the lane-level data of the target area;

[0145] The lane aerial image segmentation and extraction subunit is configured to extract lane aerial images corresponding to each lane from the orthophoto aerial image using lane dividing lines;

[0146] The lane nighttime lighting information determination subunit is configured to determine the nighttime lighting information of the corresponding lane based on the light source brightness information contained in the aerial image of each lane;

[0147] The regional nighttime lighting information determination subunit is configured to determine the nighttime lighting information of the target area based on the nighttime lighting information of each lane contained in the target area.

[0148] In some optional implementations of this embodiment, the road nighttime lighting information determination device 700 may further include:

[0149] The preprocessing unit is configured to perform a preprocessing operation on the lane aerial image to obtain a preprocessed image; wherein the preprocessing operation includes: enhancing contrast and removing noise;

[0150] Correspondingly, the lane nighttime lighting information determination subunit includes:

[0151] A grayscale conversion and brightness threshold determination module is configured to convert the preprocessed image into a grayscale image, and determine a brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method;

[0152] An illumination area and light source information determination module is configured to determine the illumination area based on a brightness threshold, and determine the light source information according to the illumination area;

[0153] A street light source determination module is configured to exclude non-street light sources based on the light source information to obtain the street light source;

[0154] The lane nighttime lighting information determination module is configured to determine the brightness information of the illuminated area provided by the street light source as the nighttime lighting information of the corresponding lane.

[0155] In some optional implementations of this embodiment, the grayscale conversion and brightness threshold determination module includes a brightness threshold determination submodule configured to determine a brightness threshold for segmenting the foreground and the background based on the grayscale image and the maximum inter-class variance method, and the brightness threshold determination module is further configured to:

[0156] Determine the number of pixels at each preset gray level according to the grayscale image;

[0157] Calculate the number of pixels, grayscale mean and variance of foreground and background at each grayscale level respectively;

[0158] The maximum inter-class variance method is used to calculate the inter-class variance at different gray levels, and the brightness threshold for segmenting the foreground and background is determined based on the maximum inter-class variance.

[0159] In some optional implementations of this embodiment, the illumination area and light source information determination module includes an illumination area determination submodule configured to determine the illumination area based on a brightness threshold, and the illumination area determination submodule is further configured to:

[0160] For each pixel with an actual grayscale value exceeding the brightness threshold, the illuminated area is obtained by connected domain analysis.

[0161] In some optional implementations of this embodiment, the illumination area and light source information determination module includes a light source information determination submodule configured to determine the light source information according to the illumination area, and the light source information determination submodule is further configured to:

[0162] Determine the position and movement characteristics of the light source based on the shape of the illuminated area and the propagation characteristics of light;

[0163] Correspondingly, the street light source determination module is further configured as follows:

[0164] Determine a light source whose position exceeds a preset distance from the lane and / or whose moving characteristic is moving as a non-street light source;

[0165] Other light sources that are different from non-street light sources are identified as street light sources.

[0166] In some optional implementations of this embodiment, the road nighttime lighting information determination device 700 may further include:

[0167] The aerial photography pause control unit is configured to control the drone not to take aerial images during the period when there are vehicles with their lights turned on.

[0168] This embodiment exists as a device embodiment corresponding to the above method embodiment. The device for determining road nighttime lighting information provided by this embodiment first abstracts the target area that lacks nighttime lighting information into representative points, then constructs a complete graph by connecting the representative points, and then determines a flight planning route suitable for UAV flight based on the complete graph and a preset path optimization algorithm, so that the UAV can obtain the required orthophoto aerial images of the target area by shooting according to the flight planning route, and finally determine the nighttime lighting brightness of each target area from the brightness information contained in the orthophoto aerial images. That is, this solution makes full use of the UAV's flight altitude advantage, and conveniently obtains orthophoto aerial images of nighttime brightness information by arranging a reasonable flight planning path for it, so as to provide a safer automatic driving service and better protect the personal and property safety of nighttime driving while increasing the nighttime lighting information of each road.

[0169] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the method for determining road night lighting information described in any of the above embodiments can be implemented when the at least one processor executes the instructions.

[0170] According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the method for determining road nighttime lighting information described in any of the above embodiments when executed.

[0171] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, which, when executed by a processor, can implement the steps of the method for determining road nighttime lighting information described in any of the above embodiments.

[0172] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0173] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0174] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0175] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a road nighttime lighting information determination method. For example, in some embodiments, the road nighttime lighting information determination method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the road nighttime lighting information determination method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the road nighttime lighting information determination method in any other appropriate manner (e.g., by means of firmware).

[0176] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0177] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0178] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0179] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0180] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0181] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0182] According to the technical solution of the disclosed embodiment, for the target area lacking nighttime lighting information, the area is first abstracted into representative points, and then a complete graph is constructed by connecting the representative points, and then a flight planning route suitable for UAV flight is determined based on the complete graph and a preset path optimization algorithm, so that the UAV can obtain the required orthophoto aerial images of the target area by shooting according to the flight planning route, and finally determine the nighttime lighting brightness of each target area from the brightness information contained in the orthophoto aerial images. That is, this solution makes full use of the UAV's flight altitude advantage, and conveniently obtains orthophoto aerial images of nighttime brightness information by arranging a reasonable flight planning path for it, so as to provide a safer automatic driving service and better protect the personal and property safety of nighttime driving while increasing the nighttime lighting information of each road.

[0183] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0184] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for determining road nighttime lighting information, comprising: Identify target areas where nighttime lighting information is missing; Based on the representative points of each target area, a complete graph is generated in which a connection relationship is established between any two representative points; Determine the flight planning route based on the complete graph and a preset path optimization algorithm; Obtaining orthophoto aerial images taken by the drone flying along the planned flight route; Based on the brightness information contained in the orthophoto aerial image, nighttime lighting information of the target area is determined.

2. The method according to claim 1, wherein: The step of determining target areas where nighttime lighting information is missing includes: Determine an area where no nighttime lighting information has been recorded as the target area; An area where nighttime lighting information has been recorded but the last recorded nighttime lighting information has not been updated within a preset time period is determined as the target area.

3. The method according to claim 1, wherein: The step of determining target areas where nighttime lighting information is missing includes: Determining that there is an abnormal trajectory in the user trajectory in which the degree of reduction in travel speed exceeds a preset degree; Performing clustering processing on the abnormal trajectories to obtain fused trajectories; An area in the fused trajectory corresponding to a portion of the trajectory where the speed is reduced and the speed after the reduction is maintained is determined as the target area.

4. The method according to claim 3, wherein: The step of determining the area in the fused trajectory corresponding to the portion of the trajectory where the speed is reduced and the speed after the reduction is maintained as the target area includes: Determine the portion of the fused trajectory where the speed reduction occurs and the speed after the reduction is maintained as a speed change trajectory; Performing convex hull processing on the speed change trajectory to obtain a convex hull; The target area is determined based on the area covered by the convex hull.

5. The method according to claim 4, wherein: The determining the target area based on the area covered by the convex hull comprises: determining the centroid of the convex hull; The target area is obtained by taking the centroid as the area center and the preset length as the area radius.

6. The method according to claim 1, wherein: The step of generating a complete graph in which a connection relationship is established between any two representative points based on the representative points of each target area includes: Determine a representative point of each lane contained in each target area; wherein the representative point includes: a center point or a mass point; Connect any two of the representative points to obtain a complete graph in which any two of the representative points are connected by an edge; wherein the edge has a distance weight that matches the distance between the two connected representative points.

7. The method according to claim 6, wherein: The edge also has a priority weight that matches the difference in lane priorities between the two connected representative points; wherein the types of lanes include motor vehicle lanes, non-motor vehicle lanes and sidewalks, the sidewalk has the highest priority, the motor vehicle lane has the lowest priority, the non-motor vehicle lane has a priority between the sidewalk and the motor vehicle lane, and the number of lanes of each type includes multiple.

8. The method according to claim 7, wherein: Determining the flight planning route based on the complete graph and a preset path optimization algorithm includes: Based on the distance weights, priority weights and simulated annealing algorithm of the edges connecting the representative points in the complete graph, a complete flight planning route passing through all the representative points is calculated.

9. The method according to claim 8, further comprising: In response to the planned flight distance of the complete flight planning route exceeding the maximum flight distance of the UAV, the complete flight planning route is split into at least two sub-flight planning routes that only pass through some of the representative points according to the maximum flight distance; wherein the sum of the number of times each of the sub-flight planning routes passes through any of the representative points is not less than 1; Correspondingly, the obtaining of the orthophoto aerial image taken by the drone flying along the planned flight route includes: Obtain orthophoto aerial images taken by the drone flying along different sub-flight planning routes in sequence.

10. The method according to any one of claims 1 to 9, wherein: The determining the nighttime lighting information of the target area based on the brightness information contained in the orthophoto aerial image includes: determining lane dividing lines between lanes according to the lane-level data of the target area; Extracting lane aerial images corresponding to each lane from the orthophoto aerial images using the lane dividing lines; Determining nighttime lighting information of the corresponding lane based on light source brightness information contained in the aerial images of each lane; The nighttime lighting information of the target area is determined according to the nighttime lighting information of each lane contained in the target area.

11. The method according to claim 10, further comprising: Performing a preprocessing operation on the lane aerial image to obtain a preprocessed image; wherein the preprocessing operation includes: enhancing contrast and removing noise; Correspondingly, determining the nighttime lighting information of the corresponding lane based on the light source brightness information contained in the aerial images of each lane includes: Converting the preprocessed image into a grayscale image, and determining a brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method; Determine an illumination area based on the brightness threshold, and determine light source information according to the illumination area; Eliminate non-street light sources based on the light source information to obtain street light sources; The brightness information of the illuminated area provided by the street light source is determined as the nighttime lighting information of the corresponding lane.

12. The method according to claim 11, wherein: Determining the brightness threshold for segmenting the foreground and the background based on the grayscale image and the maximum inter-class variance method includes: Determining the number of pixels at each preset grayscale level according to the grayscale image; Calculate the number of pixels, grayscale mean and variance of the foreground and background at each grayscale level respectively; The inter-class variance at different gray levels is calculated using the maximum inter-class variance method, and the brightness threshold for segmenting the foreground and the background is determined according to the maximum inter-class variance.

13. The method according to claim 11, wherein: The determining of the illumination area based on the brightness threshold comprises: For each pixel point having an actual grayscale value exceeding the brightness threshold, the illuminated area is obtained by connected domain analysis.

14. The method according to claim 11, wherein: The determining of light source information according to the illumination area includes: Determining the position and movement characteristics of the light source according to the shape of the illumination area and the propagation characteristics of the light; Correspondingly, excluding non-street light sources based on the light source information to obtain street light sources includes: Determine the light source whose position exceeds a preset distance from the lane and / or whose moving characteristic is moving as a non-street light source; Other light sources that are different from the non-street light sources are determined as the street light sources.

15. The method according to claim 14, further comprising: The drone is controlled not to take aerial images during a period when there are vehicles with their lights turned on.

16. A device for determining road nighttime lighting information, comprising: a target area determination unit configured to determine each target area lacking nighttime lighting information; A complete graph generating unit is configured to generate a complete graph in which a connection relationship is established between any two representative points based on the representative points of each target area; A flight route planning unit, configured to determine a flight planning route based on the complete graph and a preset path optimization algorithm; An aerial image acquisition unit is configured to acquire orthophoto aerial images taken by the drone flying along the flight planning route; The nighttime lighting information determination unit is configured to determine the nighttime lighting information of the target area based on the brightness information contained in the orthophoto aerial image.

17. The device according to claim 16, wherein: The target area determination unit is further configured to: Determine an area where no nighttime lighting information has been recorded as the target area; An area where nighttime lighting information has been recorded but the last recorded nighttime lighting information has not been updated within a preset time period is determined as the target area.

18. The device according to claim 16, wherein: The target area determination unit comprises: an abnormal trajectory determination subunit, configured to determine that there is an abnormal trajectory in the user trajectory in which the degree of reduction in travel speed exceeds a preset degree; A trajectory fusion subunit is configured to perform clustering processing on the abnormal trajectory to obtain a fused trajectory; The target region determination subunit is configured to determine, as the target region, a region in the fused trajectory corresponding to a portion of the trajectory where the speed is reduced and the speed after the reduction is maintained.

19. The device according to claim 18, wherein The target area determination subunit comprises: A speed change trajectory determination module is configured to determine a portion of the fused trajectory where the speed reduction occurs and the speed after the reduction is maintained as a speed change trajectory; A convex hull processing module is configured to perform convex hull processing on the speed change trajectory to obtain a convex hull; The target area determination module is configured to determine the target area based on the area covered by the convex hull.

20. The device according to claim 19, wherein The target area determination module is further configured to: determining the centroid of the convex hull; The target area is obtained by taking the centroid as the area center and the preset length as the area radius.

21. The device according to claim 16, wherein: The complete graph generation unit is further configured to: Determine a representative point of each lane contained in each target area; wherein the representative point includes: a center point or a mass point; Connect any two of the representative points to obtain a complete graph in which any two of the representative points are connected by an edge; wherein the edge has a distance weight that matches the distance between the two connected representative points.

22. The device according to claim 21, wherein The edge also has a priority weight that matches the difference in lane priorities between the two connected representative points; wherein the types of lanes include motor vehicle lanes, non-motor vehicle lanes and sidewalks, the sidewalk has the highest priority, the motor vehicle lane has the lowest priority, the non-motor vehicle lane has a priority between the sidewalk and the motor vehicle lane, and the number of lanes of each type includes multiple.

23. The device according to claim 22, wherein: The flight path planning unit is further configured to: Based on the distance weights, priority weights and simulated annealing algorithm of the edges connecting the representative points in the complete graph, a complete flight planning route passing through all the representative points is calculated.

24. The apparatus according to claim 23, further comprising: A flight route splitting unit is configured to, in response to the planned flight distance of the complete flight planning route exceeding the maximum flight distance of the UAV, split the complete flight planning route into at least two sub-flight planning routes that only pass through some of the representative points according to the maximum flight distance; wherein the sum of the number of times each of the sub-flight planning routes passes through any of the representative points is not less than 1; Correspondingly, the aerial image acquisition unit is further configured as follows: Obtain orthophoto aerial images taken by the drone flying along different sub-flight planning routes in sequence.

25. The device according to any one of claims 16 to 24, wherein: The nighttime lighting information determination unit comprises: a lane dividing line determination subunit, configured to determine lane dividing lines between lanes according to the lane-level data of the target area; A lane aerial image segmentation and extraction subunit is configured to extract lane aerial images corresponding to each lane from the orthophoto aerial image using the lane dividing lines; The lane nighttime lighting information determination subunit is configured to determine the nighttime lighting information of the corresponding lane based on the light source brightness information contained in the aerial image of each lane; The regional nighttime lighting information determination subunit is configured to determine the nighttime lighting information of the target area according to the nighttime lighting information of each lane contained in the target area.

26. The apparatus according to claim 25, further comprising: A preprocessing unit is configured to perform a preprocessing operation on the lane aerial image to obtain a preprocessed image; wherein the preprocessing operation includes: enhancing contrast and removing noise; Correspondingly, the lane nighttime lighting information determination subunit includes: A grayscale conversion and brightness threshold determination module is configured to convert the preprocessed image into a grayscale image, and determine a brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method; an illumination area and light source information determination module, configured to determine the illumination area based on the brightness threshold, and determine the light source information according to the illumination area; A street light source determination module is configured to exclude non-street light sources based on the light source information to obtain a street light source; The lane nighttime lighting information determination module is configured to determine the brightness information of the illuminated area provided by the street light source as the nighttime lighting information of the corresponding lane.

27. The device according to claim 26, wherein: The grayscale conversion and brightness threshold determination module includes a brightness threshold determination submodule configured to determine a brightness threshold for segmenting the foreground and the background based on the grayscale map and the maximum inter-class variance method, and the brightness threshold determination module is further configured to: Determining the number of pixels at each preset grayscale level according to the grayscale image; Calculate the number of pixels, grayscale mean and variance of the foreground and background at each grayscale level respectively; The inter-class variance at different gray levels is calculated using the maximum inter-class variance method, and the brightness threshold for segmenting the foreground and the background is determined according to the maximum inter-class variance.

28. The device according to claim 26, wherein The illumination area and light source information determination module includes an illumination area determination submodule configured to determine the illumination area based on the brightness threshold, and the illumination area determination submodule is further configured to: For each pixel point having an actual grayscale value exceeding the brightness threshold, the illuminated area is obtained by connected domain analysis.

29. The device according to claim 26, wherein: The illumination area and light source information determination module includes a light source information determination submodule configured to determine light source information according to the illumination area, and the light source information determination submodule is further configured to: Determining the position and movement characteristics of the light source according to the shape of the illumination area and the propagation characteristics of the light; Correspondingly, the street light source determination module is further configured to: Determine the light source whose position exceeds a preset distance from the lane and / or whose moving characteristic is moving as a non-street light source; Other light sources that are different from the non-street light sources are determined as the street light sources.

30. The apparatus according to claim 29, further comprising: The aerial photography pause control unit is configured to control the drone not to take aerial images during a period when there is a vehicle with its headlights turned on.

31. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for determining road nighttime lighting information according to any one of claims 1-15.

32. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method for determining road nighttime lighting information according to any one of claims 1 to 15.

33. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method for determining road nighttime lighting information according to any one of claims 1 to 15.

Images