Methods, devices, electronic equipment and storage media for determining road nighttime lighting information

Abstract

This disclosure provides a method, apparatus, electronic device, and storage medium for determining road nighttime lighting information, relating to the technical fields of UAV aerial photography, map generation, and autonomous driving navigation. The method includes: identifying target areas lacking nighttime lighting information; generating a complete graph based on representative points of each target area, where any two representative points are connected; determining a flight planning route based on the complete graph and a preset path optimization algorithm; acquiring orthophoto images taken by the UAV following the flight planning route; and determining the nighttime lighting information of the target areas based on the brightness information contained in the orthophoto images. This method fully utilizes the altitude advantage of UAVs, conveniently obtaining orthophoto images with nighttime brightness information through a reasonable flight planning path, thereby providing safer autonomous driving services and better protecting the personal and property safety of drivers at night while increasing the amount of nighttime lighting information for each road.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to the fields of drone aerial photography, map generation, and autonomous driving navigation, and particularly to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for determining road nighttime lighting information. Background Technology

[0002] In nighttime travel scenarios, good lighting conditions are of great value in ensuring personal safety, preventing traffic accidents, and calming pedestrians. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for determining road nighttime lighting information.

[0004] In a first aspect, embodiments of this disclosure propose a method for determining road nighttime lighting information, comprising: determining target areas lacking nighttime lighting information; generating a complete graph based on representative points of each target area, wherein any two representative points are connected; determining a flight planning route based on the complete graph and a preset path optimization algorithm; acquiring orthophoto images taken by a UAV flying along the flight planning route; and determining the nighttime lighting information of the target areas based on the brightness information contained in the orthophoto images.

[0005] Secondly, embodiments of this disclosure propose a road nighttime lighting information determination device, comprising: a target area determination unit configured to determine target areas lacking nighttime lighting information; a complete map generation unit configured to generate a complete map based on representative points of each target area, wherein any two representative points are connected; a flight route planning unit configured to determine a flight planning route based on the complete map and a preset path optimization algorithm; an orthophoto aerial image acquisition unit configured to acquire orthophoto aerial images taken by a drone flying along the flight planning route; and a nighttime lighting information determination unit configured to determine the nighttime lighting information of the target area based on the brightness information contained in the orthophoto aerial images.

[0006] Thirdly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the road nighttime lighting information determination method as described in any implementation of the first aspect.

[0007] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer, when executed, to implement the road nighttime lighting information determination method as described in any implementation of the first aspect.

[0008] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, can implement the steps of the road nighttime lighting information determination method as described in any implementation of the first aspect.

[0009] The road nighttime lighting information determination scheme disclosed herein, for target areas lacking nighttime lighting information, first abstracts the area into representative points, then constructs a complete map by connecting these representative points. Based on this complete map and a pre-defined path optimization algorithm, a flight plan suitable for UAV flight is determined. This allows the UAV to capture orthophotos of the desired target area along the flight plan, and finally, the nighttime lighting brightness of each target area can be determined from the brightness information contained in the orthophotos. In other words, this scheme fully utilizes the UAV's high altitude advantage, conveniently obtaining orthophotos of nighttime brightness information by arranging a reasonable flight plan, thus providing safer autonomous driving services and better protecting the personal and property safety of drivers at night while increasing the availability of nighttime lighting information for each road.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0012] Figure 1 This is an exemplary system architecture to which this disclosure can be applied;

[0013] Figure 2 A flowchart illustrating a method for determining road nighttime lighting information provided in this embodiment of the disclosure;

[0014] Figure 3 A flowchart illustrating a method for determining a target area based on a user trajectory, provided in an embodiment of this disclosure;

[0015] Figure 4 A flowchart illustrating a method for generating flight planning routes and acquiring orthophoto images provided in this disclosure embodiment;

[0016] Figure 5 A flowchart illustrating a method for obtaining nighttime lighting information for each lane and target area using lane-level data, provided in this embodiment of the disclosure;

[0017] Figure 6 A flowchart illustrating another method for obtaining nighttime lighting information of each lane and target area from lane-level data, as provided in this disclosure embodiment;

[0018] Figure 7 This is a structural block diagram of a road nighttime lighting information determination device provided in an embodiment of the present disclosure;

[0019] Figure 8 This is a schematic diagram of the structure of an electronic device suitable for performing a method for determining road nighttime lighting information, provided as an embodiment of the present disclosure. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0021] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0022] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the road nighttime lighting information determination method, apparatus, electronic device, and computer-readable storage medium of the present disclosure can be applied.

[0023] like Figure 1 As shown, system architecture 100 may include terminal devices 101 and 102, server 103, network 104, and drone 105. Network 104 is used as a medium to provide communication links between different terminal devices 101 and 102 and server 103, and between server 103 and drone 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0024] With user authorization, user-carried terminal devices 101 and 102 can record relevant user information and transmit it to server 103 via network 104. Server 103, based on analysis and processing of the received user information, can further generate commands to control drone 105, enabling drone 105 to fly and receive images captured during flight. Various applications can be installed on terminal devices 101 and 102, server 103, and drone 105 to facilitate communication between them, such as trajectory recording and collection applications, road nighttime lighting information determination applications, and drone flight control applications.

[0025] Terminal devices 101 and 102, server 103, and drone 105 are typically hardware, but can also be software in specific scenarios (such as simulation scenarios). When terminal devices 101 and 102 are hardware, they can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, desktop computers, servers, or server clusters. When terminal devices 101 and 102 are software, they can be installed in the aforementioned electronic devices, and can be implemented as multiple software programs or software modules, or as a single software program or software module, without specific limitations. When server 103 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module, without specific limitations. When drone 105 is hardware, it can be implemented as different models of drone devices with flight and shooting capabilities. When drone 105 is software, it can be implemented as software or software modules integrating flight and shooting functions, or as a single software program or software module, without specific limitations.

[0026] Server 103 can provide various services through its built-in applications. Taking a road nighttime lighting information determination application as an example, when server 105 runs this application, it can achieve the following effects: First, based on the relevant user trajectories recorded in terminal devices 101 and 102 and the locally recorded historical map information received through network 104, it determines each target area lacking nighttime lighting information. Then, based on representative points of each target area, it generates a complete graph in which any two representative points are connected. Next, based on the complete graph and a preset path optimization algorithm, it determines the flight planning route. The next step is to send the flight planning route to drone 103 through network 104, and then receive the orthophoto images taken by drone 103 following the flight planning route. Finally, based on the brightness information contained in the orthophoto images, it determines the nighttime lighting information of the target area.

[0027] It should be noted that the relevant user trajectories recorded in terminal devices 101 and 102 can be obtained from terminal devices 101 and 102 via network 104, or they can be pre-stored locally on server 103 through various means. Therefore, when server 103 detects that this data is already stored locally (for example, when it starts processing previously left-behind tasks), it can choose to retrieve this data directly from the local storage. In this case, the exemplary system architecture 100 may not include terminal devices 101 and 102.

[0028] Since generating road network data requires significant computing resources and power, the road nighttime lighting information determination method provided in the subsequent embodiments of this disclosure is generally executed by a server 103 with strong computing power and abundant computing resources. Correspondingly, the road nighttime lighting information determination device is also generally located within the server 103. However, it should also be noted that when terminal devices 101 and 102 also possess sufficient computing power and resources, they can also perform the aforementioned calculations performed by the server 103 through the road nighttime lighting information determination application installed on them, thereby outputting the same results as the server 103. Especially when multiple terminal devices with different computing capabilities exist simultaneously, but the road nighttime lighting information determination application determines that the terminal device has strong computing power and abundant remaining computing resources, it can allow the terminal device to perform the aforementioned calculations, thereby appropriately reducing the computing pressure on the server 103. Accordingly, the road nighttime lighting information determination device can also be located within the terminal devices 101 and 102. In this case, the exemplary system architecture 100 may also exclude the server 103 and the network 104.

[0029] It should be understood that Figure 1The number of terminal devices, networks, servers, and drones shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, servers, and drones can be included.

[0030] Please refer to Figure 2 , Figure 2 A flowchart of a method for determining road nighttime lighting information provided in this disclosure embodiment, wherein process 200 includes the following steps:

[0031] Step 201: Identify the target areas lacking nighttime lighting information;

[0032] This step aims to determine the entity responsible for implementing the method based on road nighttime lighting information (e.g., Figure 1 The server 103 shown determines the area where nighttime lighting information that meets the requirements is missing. For ease of description later, the area where nighttime lighting information that meets the requirements is missing is referred to as the target area.

[0033] In this disclosure, nighttime lighting information refers to the illumination brightness provided by a region, road segment, road, or lane at night. This information is used to ensure the safe travel of objects within that region, road segment, road, or lane, and can serve as additional road network data. Road network data typically refers to data describing the road system and its connections, including information such as road name, type, location, length, intersections, number of lanes, and different lane types. This data is commonly used in traffic planning, navigation systems, and geographic information systems to help analyze and optimize traffic flow and routes. It can also be used to support autonomous driving services for autonomous vehicles. By incorporating nighttime lighting information, subsequent navigation or autonomous driving processes can leverage the nighttime lighting information of different roads to rationally plan safer routes or trajectories for nighttime driving, thereby fully ensuring personal and property safety.

[0034] If no nighttime lighting information has ever been collected for a certain area (e.g., a road without streetlights, or streetlights installed but not yet in use), it can be considered that the area lacks nighttime lighting information that meets the requirements. If nighttime lighting information has been collected for a certain area before, but lacks timeliness (e.g., it has not been updated for more than 3 months or longer, such as streetlights being damaged or not being turned on recently), then it can also be considered that the area lacks nighttime lighting information that meets the requirements.

[0035] Specifically, the target area can be determined by filtering and identifying it from map databases or road administration information. Alternatively, it can be exclusively determined based on the conditions that non-target areas have nighttime lighting information that meets the requirements. Furthermore, it can be determined by reverse deduction from a practical perspective based on whether there are sections of the user's trajectory where the travel speed suddenly changes due to poor nighttime lighting.

[0036] One possible implementation method, including but not limited to:

[0037] The system can identify areas that have not recorded any nighttime lighting information as target areas, as well as areas that have recorded nighttime lighting information but have not updated the previously recorded nighttime lighting information within a preset time period.

[0038] Step 202: Based on the representative points of each target region, generate a complete graph in which any two representative points are connected.

[0039] Building upon step 201, this step aims to generate a complete graph by the aforementioned executing entity, based on representative points of each target region, in which any two representative points are connected. In essence, the representative point is a point that can represent the corresponding target area. Specifically, any point within the area that can represent that area can be used as the representative point, such as the center point or a mass point of the area. The edge connecting any two representative points not only indicates the connection relationship, but can also represent the path and path length between the objects represented by the two points in an appropriate form, depending on the scenario. For example, it can be represented by the edge length matching the path length, or different distance weights can be assigned to the edge when different edge lengths cannot be used. The path length can be calculated based on the straight-line distance in a plane, or it can be calculated using a spherical straight line. Based on the distance weight, considering that different traveling objects and different types of travel will have different safety levels under the same nighttime lighting brightness, for example, lanes can be divided into: motor vehicle lanes, non-motor vehicle lanes, and pedestrian lanes, corresponding to vehicle travel, bicycle travel, electric vehicle travel, and pedestrian travel, respectively. Nighttime lighting is obviously more important for pedestrians and cyclists. Therefore, in addition to the distance weight, different priority weights can be set for different types of lanes within the target area (used to represent the importance priority of nighttime lighting to different types of lanes), so as to better calculate a reasonable flight planning route by combining the distance weight and priority weight. Specifically, priority weights can be expressed as follows: from low priority to high priority, the weight decreases by 0.2 for each level increase; from high priority to low priority, the additional weight increases by 0.2 for each level decrease; at the same time, the weight of the highest priority road remains unchanged, and the weight increases by 0.2 for each road with a lower priority.

[0040] A complete graph is a fundamental concept in graph theory. It is an undirected graph in which every pair of distinct vertices is connected by an edge. In other words, a complete graph is a graph where every pair of vertices is connected by an edge, and all vertices are connected by edges. As a basic graph type in graph theory, the complete edge connectivity property of complete graphs makes them play an important role in many application areas. They are the foundation of many theoretical and algorithmic studies, helping to understand the structural properties of graphs and the complexity of algorithms.

[0041] This step aims to construct a complete graph by connecting representative points in each target region. This complete graph then generates interconnected paths between all regions, facilitating subsequent path optimization. For example, it finds the shortest loop, allowing the drone to visit all cities with only one pass. This approach is used in this step to find optimal paths between different regions; the same principle can also be applied to find optimal paths between sub-regions within a single region.

[0042] One possible implementation method, including but not limited to:

[0043] First, representative points for each target region are determined. These representative points can include the center point or a mass point. Then, any two representative points are connected to obtain a complete graph in which all two representative points are connected by edges. Each edge has a weight that matches the distance between the two connected representative points, so that the distance between the two regions corresponding to the two connected representative points can be reflected by the magnitude of the weight. Additionally, each edge can have a priority weight that matches the difference in lane priority between the two connected representative points. Lane types include motor vehicle lanes, non-motor vehicle lanes, and pedestrian lanes. Pedestrian lanes have the highest priority, motor vehicle lanes have the lowest priority, and non-motor vehicle lanes have a priority between pedestrian lanes and motor vehicle lanes. There are multiple lanes of each type.

[0044] Step 203: Determine the flight planning route based on the complete graph and the preset path optimization algorithm;

[0045] Building upon step 202, this step aims to have the aforementioned executing entity determine the flight planning route based on the complete graph and a preset path optimization algorithm. Since the complete graph already represents the path between any two representative points by constructing edges, a suitable path optimization algorithm is used to find a suitable flight planning route for the UAV to pass through as many representative points as possible in one shortest route based on the complete graph.

[0046] Path optimization algorithms aim to find the optimal path from a starting point to an end point in a graph, typically based on the shortest distance or lowest cost. Examples include Dijkstra's algorithm (principle: starting from the starting point, it progressively expands to unvisited vertices in the graph, selecting the vertex with the shortest path for processing until the end point is found; suitable for graphs with non-negative edge weights), the Bellman-Ford algorithm (principle: through multiple relaxation operations, it updates the shortest path estimate for each vertex in the graph; it can handle negative edge weights but not negative cycle weights), the Floyd-Warshall algorithm (principle: using dynamic programming, it calculates the shortest path for all vertex pairs by updating the shortest path between all vertex pairs in the graph; suitable for graphs with negative edge weights but no negative cycle weights), and simulated annealing (a heuristic algorithm used to find optimal solutions, particularly suitable for combinatorial optimization problems), etc.

[0047] Step 204: Acquire orthophotos of the drone taken by the drone following the planned flight path;

[0048] Building upon step 203, this step aims to obtain orthophotos of the UAV taken by the aforementioned executing entity as it flies along the planned flight path. Specifically, to achieve this objective, the executing entity first sends the planned flight path to the UAV in the mission execution state, enabling the UAV to fly according to the received path and use its onboard camera to capture images of the area it traverses during flight.

[0049] Orthophotos are geometrically corrected aerial or drone images that accurately reflect the actual terrain. This type of imagery eliminates distortions caused by shooting angles and terrain variations, ensuring that every point in the image is aligned with the actual geographic coordinate system. The result is an image with true scale and accurate geographic location, which can be used for cartography, geographic information system analysis, road network information collection, and spatial planning.

[0050] Step 205: Determine the nighttime lighting information of the target area based on the brightness information contained in the orthophoto aerial image.

[0051] Building upon step 204, this step aims to have the aforementioned executing entity analyze and process the nighttime illumination information of each target area based on the image content related to nighttime brightness contained in the orthophoto aerial images.

[0052] Specifically, while orthophoto aerial images contain nighttime brightness information relevant to determining the nighttime lighting of different roads, to better and more efficiently extract nighttime lighting information for each target area, especially when the orthophoto aerial images include images of all target areas, some images passing through different target areas may inevitably be included, without requiring road network data. Furthermore, the images of each target area may also contain images unrelated to determining nighttime lighting information. Therefore, it is possible to first segment the images of each target area, then extract key image content related to determining the nighttime brightness of roads from the images of each target area, and then combine this key image content for preliminary confirmation and correction of nighttime lighting information, ultimately obtaining accurate nighttime lighting information for each target area. Further, in this process, for target areas containing multiple lanes, the images can be segmented into images corresponding to each lane before extracting road network data, and then the corresponding nighttime lighting information can be determined for each lane's image separately.

[0053] The method for determining road nighttime lighting information provided in this disclosure, for target areas lacking nighttime lighting information, first abstracts the area into representative points, then constructs a complete graph by connecting these representative points. Based on this complete graph and a preset path optimization algorithm, a flight plan suitable for UAV flight is determined. This allows the UAV to capture orthophotos of the desired target area along the flight plan, and finally, the nighttime lighting brightness of each target area can be determined from the brightness information contained in the orthophotos. In other words, this solution fully utilizes the UAV's high altitude advantage, conveniently obtaining orthophotos of nighttime brightness information by arranging a reasonable flight plan. This provides safer autonomous driving services and better protects the personal and property safety of drivers at night while increasing the availability of nighttime lighting information for each road.

[0054] To deepen the understanding of how to determine the target area, this embodiment also... Figure 3 Another implementation method is provided, namely, how to determine the target area based on the user's trajectory, wherein process 300 includes the following steps:

[0055] Step 301: Determine if there are abnormal trajectories in the user's trajectory where the speed decreases by more than a preset level;

[0056] This step aims to have the aforementioned implementing entity first collect and identify abnormal trajectories in the user's trajectory where the decrease in travel speed exceeds a preset level. The reason for using the feature of a sharp decrease in travel speed to filter abnormal trajectories is that when a traveler moves from a brightly lit section of road to a darker section at night, the reduced brightness makes it difficult to accurately see the details of the road ahead. Consequently, for safety reasons, the traveler will reduce its travel speed to observe more carefully. Therefore, this disclosure uses this feature to filter target areas in reverse.

[0057] Step 302: Cluster the abnormal trajectories to obtain the fused trajectories;

[0058] Based on step 301, this step aims to have the aforementioned executing entity perform clustering processing on the abnormal trajectories to obtain fused trajectories.

[0059] Specifically, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm can be used. This algorithm uses 100,000 times the distance between trajectory points as the radius of the e-neighborhood to cluster trajectories within the trajectory set, aggregating similar trajectories into a single line and adding PV (position and velocity) change information.

[0060] Step 303: Determine the target region as the area in the fused trajectory that corresponds to the part of the trajectory where the speed decreases and the speed is maintained after the decrease.

[0061] Building upon step 302, this step aims to have the aforementioned executing entity determine the target region within the fused trajectory corresponding to the portion of the trajectory where the travel speed decreases and the portion maintaining the decreased speed. Specifically, the region where the speed decreases corresponds to the starting portion of the target region, the portion maintaining the decreased speed corresponds to the extended portion starting from the starting portion, and the ending portion corresponds to the portion where the speed recovers.

[0062] One possible implementation method, including but not limited to:

[0063] The portion of the fused trajectory that experiences a decrease in speed and maintains that decreased speed is identified as the speed change trajectory. This speed change trajectory is then subjected to convex hull processing to obtain the convex hull. Based on the area covered by the convex hull, the target region is determined. Specifically, the centroid of the convex hull can be determined first, and then the target region can be obtained by using the centroid as the region center and a preset length as the region radius. Alternatively, other representative points of the convex hull can be used as region centers, with other lengths as radii, to obtain similar target regions. Furthermore, the target region does not necessarily have to be circular; it can be any other more suitable shape that better matches the actual situation.

[0064] The primary purpose of convex hull transformation in trajectory data processing is to simplify and optimize data representation. By generating the convex hull (the smallest convex polygon) of the trajectory, the number of complex trajectory data points is reduced while retaining key boundary information. Its functions include: reducing the size of the dataset, making data processing and storage more efficient, and accurately extracting the spatial boundaries of the trajectory, facilitating the analysis of trajectory coverage. It offers the following advantages: simplified data reduces computational complexity and improves analysis speed; simplified data is easier to visualize, helping to intuitively understand the overall shape and coverage area of ​​the trajectory; in spatial analysis, path planning, and area monitoring, reducing redundant points improves processing accuracy and efficiency. In short, convex hull transformation, by removing unnecessary details, helps to better understand and apply trajectory information when processing and analyzing trajectory data.

[0065] This embodiment provides a method for determining target areas lacking nighttime lighting information based on user trajectories through steps 301-303. This allows for the reverse deduction of target areas based on user trajectory feedback, which can complement the forward determination method to make the determined target areas more accurate and comprehensive.

[0066] Based on any of the above embodiments, this embodiment also... Figure 4 A method for generating flight planning routes and acquiring orthorectified aerial imagery is provided to deepen the understanding of how a path optimization algorithm based on a complete graph obtains flight planning routes and then orthorectified aerial imagery. The process 400 includes the following steps:

[0067] Step 401: Based on the distance weights, priority weights, and simulated annealing algorithm of the edges connecting each representative point in the complete graph, calculate the complete flight planning route passing through all representative points;

[0068] This step aims to have the aforementioned execution entity calculate the complete flight planning route passing through all representative points based on the distance weights, priority weights, and simulated annealing algorithm of the edges connecting each representative point in the complete graph.

[0069] In this context, the distance weight of the edge connecting each representative point is used to reflect the actual distance of the path, and the priority weight is used to reflect the importance of the road segment to the nighttime lighting requirement. When the simulated annealing algorithm is applied to the path optimization problem of a complete graph, simulated annealing can be used to find a path that passes through all destinations and is as short as possible, so as to avoid exceeding the maximum flight distance of the drone.

[0070] The general process of finding the shortest path on a complete graph using the simulated annealing algorithm can be summarized in the following points:

[0071] 1) Initialization: Randomly generate an initial path (visit all vertices in a loop).

[0072] 2) Define the objective function:

[0073] The objective function is typically the total length or total cost of the path. The length of the current path is calculated as the evaluation criterion for the current solution.

[0074] 3) Generate neighborhood solutions: Make small random adjustments to the current path, such as swapping the positions of two vertices in the path, to generate a new path.

[0075] 4) Calculate the objective function value of the new solution: calculate the length or cost of the new path.

[0076] 5) Acceptance Criterion: If the new path is better (shorter), then accept the new path as the current path. If the new path is worse, then accept it with a certain probability to avoid getting trapped in a local optimum. The acceptance probability usually depends on the current temperature and the degree of solution degradation.

[0077] 6) Gradually reduce the temperature: Temperature controls the probability of accepting a poor solution. As the algorithm progresses, the temperature is gradually reduced to decrease the possibility of accepting a bad solution, and eventually converges to a better solution.

[0078] 7) Termination condition: When the temperature drops to a very low level or after a certain number of iterations, the algorithm terminates and returns the currently found optimal path.

[0079] Simulated annealing, through its combination of randomness and stepwise optimization, helps to explore possible solutions and find better solutions in path optimization of a complete graph.

[0080] Step 402: In response to the fact that the planned flight distance of the complete flight route exceeds the maximum flight distance of the UAV, the complete flight route is divided into at least two sub-flight routes that only pass through some representative points according to the maximum flight distance;

[0081] This step is based on the assumption that the planned flight distance of the complete flight route calculated in step 401 exceeds the maximum flight distance of the UAV. Since it exceeds the maximum flight distance of the UAV, it means that the UAV cannot pass through all target areas in a single flight. Therefore, at least one UAV needs to fly twice to obtain orthophoto images of all target areas. This step aims to divide the complete flight route into at least two sub-flight routes that only pass through some representative points according to the maximum flight distance. The sum of the number of times each sub-flight route passes through any representative point is not less than 1. That is, after dividing it into multiple sub-flight routes, for safety reasons, the areas passed through by different flight routes can partially overlap. Furthermore, multiple flights can be used to ensure that each target area is passed through at least twice, that is, to obtain orthophoto images of each target area taken during two flights, so as to avoid errors or omissions or loss of key information that may occur due to a single flight.

[0082] Specifically, when constructing a complete flight planning route or breaking it down into sub-flight planning routes, the starting point should be the latest target area or a lane within that target area that is closest to the drone's takeoff point. From there, a shortest path tree is constructed. The tree construction stops when the path depth is 40% of the drone's range. If the shortest path is less than 40% of the range, the shortest path is used directly, and the path with the smallest total weight is selected as the starting path. Then, the weight of this path is set to infinity. Shortest route planning is performed from the target area or lane within the target area at the end of the path to the drone airport. If the shortest route length exceeds 50% of the range, the drone flies directly back to the airport in a straight line, and no information is collected on the return trip.

[0083] Step 403: Acquire orthophotos taken by the drone as it flies along different sub-flight planning routes.

[0084] Based on step 402, this step aims to obtain orthophotos of the drone taken by the aforementioned executing entity as it flies along different sub-flight planning routes. That is, when the flight plan has been divided into at least two sub-flight planning routes, the same drone can be controlled to fly and take pictures sequentially along different sub-flight planning routes.

[0085] This embodiment provides a scheme for calculating a complete flight plan route based on a complete graph and simulated annealing algorithm through steps 401-403. Furthermore, when the flight distance corresponding to the complete flight plan route exceeds the maximum flight distance of the UAV, the complete flight plan path is split, allowing the UAV to acquire orthophotos of the entire target area through multiple successive flight shots. This fully utilizes the advantage of the simulated annealing algorithm in calculating the shortest path.

[0086] Based on any of the above embodiments, to deepen the understanding of the specific process of determining the road network data of each target area based on orthophoto aerial imagery, this embodiment also... Figure 5 A method for obtaining nighttime lighting information of a target area using lane-level data is provided, and its process 500 includes the following steps:

[0087] Step 501: Determine the lane boundaries between lanes based on the lane-level data of the target area;

[0088] This step aims to have the aforementioned implementing entity first determine the lane boundaries between lanes based on lane-level data of the target area. Specifically, it uses the lane boundaries of multiple lanes within the target area provided by lane-level road network data as a segmentation reference.

[0089] Step 502: Extract the lane aerial images corresponding to each lane from the orthophoto aerial images using lane dividing lines;

[0090] Based on step 501, this step aims to have the aforementioned executing entity extract lane aerial images corresponding to each lane from the orthophoto aerial images using lane dividing lines.

[0091] Step 503: Based on the light source brightness information contained in the aerial images of each lane, determine the nighttime lighting information of the corresponding lane.

[0092] Specifically, lane boundary lines are extracted from the lane-level road network data for each lane within the target area, and these are processed into two boundary lines located on both sides of the road for that lane. This allows for the segmentation of images of different lanes using these boundary lines. Furthermore, based on the lane boundary lines, the center polyline of each lane can be identified using the lane-level road network data. This center polyline can also serve as the flight path for the UAV when flying over this lane, thereby further ensuring that the UAV collects accurate orthophoto information of the road.

[0093] Step 504: Determine the nighttime lighting information of the target area based on the nighttime lighting information of each lane contained within the target area.

[0094] Building upon step 503, this step aims to have the aforementioned executing entity determine the nighttime lighting information of the target area based on the nighttime lighting information of each lane within the target area. In other words, the nighttime lighting information of the target area can be obtained by comprehensively considering the nighttime lighting information of each lane within the target area.

[0095] This embodiment provides a scheme through steps 501-504 that uses lane-level road network data to assist in segmenting the aerial images of each lane, and further determines the nighttime lighting information of the corresponding lane based on the aerial images of each lane, and finally obtains the nighttime lighting information of the target area. Accurate segmentation can improve the accuracy of the nighttime lighting information of each lane and the target area obtained in the end.

[0096] In the above Figure 5 Based on the illustrated embodiment, to further deepen the understanding of the specific implementation process, this embodiment also... Figure 6 A more specific implementation method is given, and its process 600 includes the following steps:

[0097] Step 601: Extract the lane aerial images corresponding to each lane from the orthophoto aerial images using lane dividing lines;

[0098] This step is the same as step 502 in process 500, and will not be described again here.

[0099] Step 602: Perform preprocessing on the aerial imagery of the lane to obtain the preprocessed imagery;

[0100] This step involves the aforementioned executing entity performing preprocessing operations on the aerial images of the lanes to highlight key elements that will help determine more accurate nighttime lighting information in subsequent steps. These operations include enhancing contrast and removing noise, resulting in a preprocessed image.

[0101] Step 603: Convert the preprocessed image into a grayscale image, and determine the brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method;

[0102] Among them, the Otsu algorithm is a typical example of the maximum inter-class variance method. It is a classic method for image binarization, and its core idea is to segment a grayscale image into foreground and background by optimizing the threshold. The core principle of this algorithm is to select an optimal threshold to divide the image's grayscale levels into foreground and background, thereby maximizing the between-class variance between the foreground and background. This maximized between-class variance corresponds to the optimal segmentation effect.

[0103] One specific implementation method could be:

[0104] First, the number of pixels at each preset grayscale level is determined based on the grayscale image. Then, the number of pixels, grayscale mean, and variance of the foreground and background are calculated for each grayscale level. Next, the inter-class variance at different grayscale levels is calculated using the maximum inter-class variance method, and a brightness threshold for segmenting the foreground and background is determined based on the maximum inter-class variance. That is, this brightness threshold is the value that maximizes the inter-class variance. Then, the image is segmented into foreground and background using this brightness threshold, completing the binarization process. In the case of the scheme provided in this embodiment, it is divided into illuminated and unilluminated regions.

[0105] Step 604: Determine the illumination area based on the brightness threshold, and determine the light source information based on the illumination area;

[0106] Based on step 603, this step aims to have the aforementioned executing entity determine the illumination area based on the brightness threshold, and determine the light source information based on the illumination area.

[0107] The illuminated area can be obtained by performing connected component analysis on each pixel with an actual gray value exceeding the brightness threshold. In a simpler case, the areas covered by pixels of the same type that are close to each other can be connected. The light source information can be determined based on the shape of the illuminated area and the propagation characteristics of light. The light source information can be further subdivided into: the position and movement characteristics of the light source, and can also include area information.

[0108] Step 605: Eliminate non-streetlight light sources based on light source information to obtain streetlight light sources;

[0109] Building upon step 604, this step aims to have the aforementioned executing entity exclude non-streetlight sources based on light source information and obtain streetlight sources. For example, light sources whose location exceeds a preset distance from their lane and / or whose movement characteristics are moving are identified as non-streetlight sources. Then, other light sources that are distinct from non-streetlight sources are identified as streetlight sources.

[0110] The reason for excluding non-streetlight light sources to obtain streetlight light sources is that the headlights of passing or starting vehicles are also a light source, which will provide moving illumination for a short time. Such short-term and unstable illumination cannot serve as a stable light source that can provide safety protection and a certain level of visibility for objects traveling in the corresponding lane. Therefore, it needs to be excluded as an object of interference in this disclosure.

[0111] Furthermore, in addition to excluding non-streetlight light sources, the drone can also be controlled to avoid taking aerial images of vehicles with their lights on during certain times. During the pause in filming, the drone can hover in place until the vehicle leaves the field of view before resuming filming and continuing its journey. Of course, this operation of the drone requires full utilization of its remaining energy.

[0112] One specific implementation method is as follows: First, a nighttime vehicle target detection dataset is built. The dataset is labeled as either moving vehicles or non-moving vehicles (i.e., parked vehicles) in a nighttime scene. For this dataset, a nighttime vehicle target detection model based on YOLO (a target detection model) is constructed to analyze the vehicle operation status within the field of view. Then, relying on this nighttime vehicle detection model, to avoid interference from vehicle light sources on the lighting information in the field of view, if a moving vehicle is present, data acquisition is paused until no moving vehicle is visible. Next, after generating orthophotos along the road, a mask is used to extract the image by the left and right boundary lines of the road to obtain a refined road surface image and eliminate interference from non-road pixels.

[0113] Step 606: Determine the brightness information of the illuminated area provided by the street light source as the nighttime lighting information for the corresponding lane.

[0114] Based on step 605, this step aims to have the aforementioned executing entity determine the brightness information of the illuminated area provided by the street light source as the nighttime lighting information for the corresponding lane.

[0115] This embodiment, through the specific implementation provided in steps 601-606, not only improves the presentation effect of brightness information in the image through preprocessing operations, but also uses the maximum inter-class variance method to segment the illuminated area and the non-illuminated area, and further determines the light source and eliminates interference from non-street light sources based on the illuminated area, thereby accurately obtaining the nighttime lighting information of the corresponding lane based on the remaining street light sources.

[0116] Further reference Figure 7 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a road nighttime lighting information determination device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0117] like Figure 7As shown, the road nighttime lighting information determination device 700 of this embodiment may include: a target area determination unit 701, a complete map generation unit 702, a flight route planning unit 703, an aerial image acquisition unit 704, and a nighttime lighting information determination unit 705. Specifically, the target area determination unit 701 is configured to determine target areas lacking nighttime lighting information; the complete map generation unit 702 is configured to generate a complete map based on representative points of each target area, establishing connections between any two representative points; the flight route planning unit 703 is configured to determine a planned flight route based on the complete map and a preset path optimization algorithm; the aerial image acquisition unit 704 is configured to acquire orthophotos taken by a drone flying along the planned flight route; and the nighttime lighting information determination unit 705 is configured to determine the nighttime lighting information of the target area based on the brightness information contained in the orthophotos.

[0118] In this embodiment, the specific processing and technical effects of the target area determination unit 701, the complete map generation unit 702, the flight route planning unit 703, the aerial image acquisition unit 704, and the nighttime lighting information determination unit 705 in the road nighttime lighting information determination device 700 can be referred to respectively. Figure 2 The relevant descriptions of steps 201-205 in the corresponding embodiments will not be repeated here.

[0119] In some optional implementations of this embodiment, the target region determination unit 701 can be further configured to:

[0120] The area where no nighttime lighting information has been recorded was identified as the target area;

[0121] Areas that have recorded nighttime lighting information but have not updated the previously recorded nighttime lighting information within a preset time period are identified as target areas.

[0122] In some optional implementations of this embodiment, the target region determination unit 701 may include:

[0123] The abnormal trajectory determination subunit is configured to determine whether there is an abnormal trajectory in the user's trajectory where the speed of travel decreases by more than a preset degree;

[0124] The trajectory fusion subunit is configured to cluster abnormal trajectories to obtain fused trajectories.

[0125] The target region determination sub-unit is configured to determine the region in the fused trajectory that corresponds to the portion of the trajectory where the travel speed decreases and the speed is maintained after the decrease as the target region.

[0126] In some optional implementations of this embodiment, the target region determination subunit 701 may include:

[0127] The speed change trajectory determination module is configured to determine the portion of the fused trajectory that corresponds to the decrease in travel speed and the maintenance of the decreased speed as the speed change trajectory;

[0128] The convex hull processing module is configured to perform convex hull processing on the velocity change trajectory to obtain the convex hull;

[0129] The target region determination module is configured to determine the target region based on the region covered by the convex hull.

[0130] In some optional implementations of this embodiment, the target region determination module can be further configured as follows:

[0131] Determine the centroid of the convex hull;

[0132] The target region is obtained by using the centroid as the center of the region and the preset length as the radius of the region.

[0133] In some optional implementations of this embodiment, the complete graph generation unit 702 can be further configured as follows:

[0134] Identify representative points for each lane within each target area; where representative points include: center point or mass point;

[0135] Connect any two representative points to obtain a complete graph in which all two representative points are connected by an edge; where each edge has a distance weight that matches the distance between the two representative points it connects.

[0136] In some optional implementations of this embodiment, the edge also has a priority weight that matches the difference in lane priority between the two representative points it connects; wherein, the types of lanes include motor vehicle lanes, non-motor vehicle lanes and pedestrian lanes, with pedestrian lanes having the highest priority, motor vehicle lanes having the lowest priority, and non-motor vehicle lanes having a priority between pedestrian lanes and motor vehicle lanes, and the number of each type of lane includes multiple lanes.

[0137] In some optional implementations of this embodiment, the flight route planning unit 703 can be further configured as follows:

[0138] Based on the distance weights, priority weights, and simulated annealing algorithm of the edges connecting each representative point in the complete graph, the complete flight planning route passing through all representative points is calculated.

[0139] In some optional implementations of this embodiment, the road nighttime lighting information determining device 700 may further include:

[0140] The flight route splitting unit is configured to split the complete flight route into at least two sub-flight routes that only pass through some representative points, in response to the planned flight distance of the complete flight route exceeding the maximum flight distance of the UAV; wherein the sum of the number of times each sub-flight route passes through any representative point is not less than 1.

[0141] Correspondingly, the aerial image acquisition unit 704 can be further configured as follows:

[0142] Acquire orthophotos of drones taken sequentially along different sub-flight plans.

[0143] In some optional implementations of this embodiment, the nighttime lighting information determination unit 705 may include:

[0144] The lane boundary determination subunit is configured to determine the lane boundary between each lane based on lane-level data of the target area.

[0145] The lane aerial image segmentation and extraction subunit is configured to extract the lane aerial images corresponding to each lane from the orthophoto aerial images using lane dividing lines.

[0146] The lane nighttime lighting information determination subunit is configured to determine the nighttime lighting information of the corresponding lane based on the light source brightness information contained in the aerial images of each lane;

[0147] The area nighttime lighting information determination subunit is configured to determine the nighttime lighting information of the target area based on the nighttime lighting information of each lane contained within the target area.

[0148] In some optional implementations of this embodiment, the road nighttime lighting information determining device 700 may further include:

[0149] The preprocessing unit is configured to perform preprocessing operations on the lane aerial imagery to obtain a preprocessed imagery; wherein the preprocessing operations include: enhancing contrast and removing noise;

[0150] Correspondingly, the lane nighttime lighting information determination subunit includes:

[0151] The grayscale conversion and brightness threshold determination module is configured to convert the preprocessed image into a grayscale image and determine the brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method.

[0152] The illumination area and light source information determination module is configured to determine the illumination area based on a brightness threshold and determine the light source information based on the illumination area;

[0153] The street light source determination module is configured to exclude non-street light sources based on light source information to obtain street light sources;

[0154] The lane nighttime lighting information determination module is configured to determine the brightness information of the illuminated area provided by the street light source as the nighttime lighting information of the corresponding lane.

[0155] In some optional implementations of this embodiment, the grayscale conversion and brightness threshold determination module includes a brightness threshold determination submodule configured to determine the brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method. The brightness threshold determination module is further configured to:

[0156] The number of pixels at each preset gray level is determined based on the grayscale image;

[0157] Calculate the number of pixels in the foreground and background, the mean gray level, and the variance for each gray level.

[0158] The inter-class variance at different gray levels was calculated using the maximum inter-class variance method, and the brightness threshold for segmenting the foreground and background was determined based on the maximum inter-class variance.

[0159] In some optional implementations of this embodiment, the illumination area and light source information determination module includes an illumination area determination submodule configured to determine the illumination area based on a brightness threshold. The illumination area determination submodule is further configured to:

[0160] For each pixel with an actual gray value exceeding the brightness threshold, the illuminated area is obtained through connected component analysis.

[0161] In some optional implementations of this embodiment, the illumination area and light source information determination module includes a light source information determination submodule configured to determine light source information based on the illumination area. The light source information determination submodule is further configured to:

[0162] Based on the shape of the illuminated area and the propagation characteristics of light, determine the position and movement characteristics of the light source;

[0163] Correspondingly, the street light source determination module is further configured as follows:

[0164] Light sources whose location is more than a preset distance from their lane and / or whose movement characteristics are moving are identified as non-streetlight sources.

[0165] Other light sources, distinct from non-streetlight light sources, are designated as streetlight light sources.

[0166] In some optional implementations of this embodiment, the road nighttime lighting information determining device 700 may further include:

[0167] The aerial photography pause control unit is configured to prevent the drone from taking aerial images of vehicles with their lights on during certain times.

[0168] This embodiment exists as a device embodiment corresponding to the method embodiment described above. The road nighttime lighting information determination device provided in this embodiment, for target areas lacking nighttime lighting information, first abstracts the area into representative points, then constructs a complete map by connecting these representative points. Based on this complete map and a preset path optimization algorithm, a flight planning route suitable for UAV flight is determined. This allows the UAV to capture orthophotos of the desired target area by following the flight planning route, and finally, the nighttime lighting brightness of each target area can be determined from the brightness information contained in the orthophotos. In other words, this solution fully utilizes the UAV's flight altitude advantage, conveniently obtaining orthophotos of nighttime brightness information by arranging a reasonable flight planning path. This provides safer autonomous driving services and better protects the personal and property safety of drivers at night while increasing the availability of nighttime lighting information for each road.

[0169] According to embodiments of this disclosure, this disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to implement the road nighttime lighting information determination method described in any of the above embodiments.

[0170] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that, when executed by a computer, enable the road nighttime lighting information determination method described in any of the above embodiments.

[0171] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when executed by a processor, can implement the steps of the road nighttime lighting information determination method described in any of the above embodiments.

[0172] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0173] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0174] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0175] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the method for determining road nighttime lighting information. For example, in some embodiments, the method for determining road nighttime lighting information may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the method for determining road nighttime lighting information described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the method for determining road nighttime lighting information by any other suitable means (e.g., by means of firmware).

[0176] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0177] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0178] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0179] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0180] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0181] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0182] According to the technical solution of this disclosure, for a target area lacking nighttime lighting information, the area is first abstracted into representative points. Then, a complete graph is constructed by connecting these representative points. Based on this complete graph and a preset path optimization algorithm, a flight planning route suitable for UAV flight is determined. This allows the UAV to capture orthophotos of the desired target area along the flight planning route. Finally, the nighttime lighting brightness of each target area can be determined from the brightness information contained in the orthophotos. In other words, this solution fully utilizes the UAV's high altitude advantage, conveniently obtaining orthophotos of nighttime brightness information by arranging a reasonable flight planning path. This provides safer autonomous driving services and better protects the personal and property safety of drivers at night while increasing the nighttime lighting information of various roads.

[0183] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0184] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining road nighttime lighting information, comprising: Identify the target areas lacking nighttime lighting information; Determine representative points for each lane within each target area, where the representative points include: center points or mass points; Connect any two representative points to obtain a complete graph in which any two representative points are connected by an edge, wherein the edge has a distance weight that matches the distance between the two representative points it connects; Based on the complete graph and the preset path optimization algorithm, the flight planning route is determined; Acquire orthophotos of the drone taken by the drone following the flight plan route; The lane boundaries between lanes are determined based on the lane-level data of the target area; lane aerial images corresponding to each lane are extracted from the orthophoto aerial images using the lane boundaries; nighttime lighting information for the corresponding lanes is determined based on the light source brightness information contained in each lane aerial image; and nighttime lighting information for the target area is determined based on the nighttime lighting information of each lane contained within the target area.

2. The method according to claim 1, wherein, The determination of target areas lacking nighttime lighting information includes: The area where no nighttime lighting information has been recorded is defined as the target area; The area that has recorded nighttime lighting information but has not updated the previously recorded nighttime lighting information within a preset time period is identified as the target area.

3. The method according to claim 1, wherein, The determination of target areas lacking nighttime lighting information includes: The system identifies an abnormal trajectory within the user's path where the rate of decrease in travel speed exceeds a preset threshold. The abnormal trajectories are clustered to obtain fused trajectories; The region in the fused trajectory that corresponds to the portion of the trajectory where the travel speed decreases and the portion that maintains the decreased speed is determined as the target region.

4. The method according to claim 3, wherein, The step of determining the target region as the region corresponding to the portion of the fused trajectory that corresponds to the decrease in travel speed and the portion that maintains the decreased speed includes: The portion of the fused trajectory that corresponds to the decrease in travel speed and the maintenance of the decreased speed is determined as the speed change trajectory; The velocity change trajectory is subjected to convex hull processing to obtain the convex hull; The target region is determined based on the area covered by the convex hull.

5. The method according to claim 4, wherein, Determining the target region based on the region covered by the convex hull includes: Determine the centroid of the convex hull; The target region is obtained by taking the centroid as the region center and the preset length as the region radius.

6. The method according to claim 1, wherein, The edge also has a priority weight that matches the difference in lane priority between the two representative points it connects; wherein, the types of lanes include motor vehicle lanes, non-motor vehicle lanes and pedestrian lanes, with the pedestrian lane having the highest priority, the motor vehicle lane having the lowest priority, and the non-motor vehicle lane having a priority between the pedestrian lane and the motor vehicle lane, and the number of each type of lane includes multiple lanes.

7. The method according to claim 6, wherein, The determination of the flight planning route based on the complete graph and the preset path optimization algorithm includes: Based on the distance weights, priority weights, and simulated annealing algorithm of the edges connecting each representative point in the complete graph, a complete flight planning route passing through all the representative points is calculated.

8. The method according to claim 7, further comprising: In response to the fact that the planned flight distance of the complete flight plan route exceeds the maximum flight distance of the UAV, the complete flight plan route is divided into at least two sub-flight plan routes that only pass through part of the representative points according to the maximum flight distance; wherein the sum of the number of times each of the sub-flight plan routes passes through any of the representative points is not less than 1; Correspondingly, acquiring the orthophoto images taken by the UAV following the flight plan includes: The orthophotos are obtained by the UAV flying sequentially along different sub-flight planning routes.

9. The method according to any one of claims 1-8, further comprising: The aerial image of the lane is preprocessed to obtain a preprocessed image; wherein the preprocessing operation includes: enhancing contrast and removing noise; Correspondingly, determining the nighttime lighting information of the corresponding lane based on the light source brightness information contained in the aerial images of each lane includes: The preprocessed image is converted into a grayscale image, and a brightness threshold for segmenting the foreground and background is determined based on the grayscale image and the maximum inter-class variance method. The illumination area is determined based on the brightness threshold, and the light source information is determined based on the illumination area; Based on the light source information, non-streetlight light sources are excluded to obtain streetlight light sources; The brightness information of the illuminated area provided by the street light source is determined as the nighttime lighting information for the corresponding lane.

10. The method according to claim 9, wherein, The step of determining the brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method includes: The number of pixels at each preset gray level is determined based on the grayscale image. Calculate the number of foreground and background pixels, grayscale mean, and variance for each grayscale level; The inter-class variance at different gray levels is calculated using the maximum inter-class variance method, and the brightness threshold used to segment the foreground and the background is determined based on the maximum inter-class variance.

11. The method according to claim 9, wherein, Determining the illuminated area based on the brightness threshold includes: For each pixel with an actual gray value exceeding the brightness threshold, the illumination area is obtained through connected component analysis.

12. The method according to claim 9, wherein, Determining the light source information based on the illuminated area includes: Based on the shape of the illuminated area and the propagation characteristics of light, determine the position and movement characteristics of the light source; Correspondingly, the step of excluding non-streetlight sources and obtaining streetlight sources based on the light source information includes: Light sources whose location is more than a preset distance from their lane and / or whose movement characteristics are moving are identified as non-streetlight light sources; Other light sources that are distinct from the non-streetlight light sources are identified as the streetlight light sources.

13. The method of claim 12, further comprising: The drone is controlled to avoid taking aerial images of vehicles with their lights on during certain times.

14. A road nighttime lighting information determination device, comprising: The target area determination unit is configured to determine each target area lacking nighttime lighting information; A complete graph generation unit is configured to determine representative points for each lane contained within each target region, the representative points including: center points or mass points; and connect any two representative points to obtain a complete graph in which any two representative points are connected by an edge, the edge having a distance weight matching the distance between the two connected representative points. The flight route planning unit is configured to determine the flight route based on the complete graph and a preset path optimization algorithm. The aerial image acquisition unit is configured to acquire orthophotos of the drone taken by the drone flying along the flight plan route. The nighttime lighting information determination unit includes: a lane boundary line determination subunit, configured to determine lane boundary lines between lanes based on lane-level data of the target area; a lane aerial image segmentation and extraction subunit, configured to extract lane aerial images corresponding to each lane from the orthophoto aerial images using the lane boundary lines; a lane nighttime lighting information determination subunit, configured to determine nighttime lighting information for the corresponding lane based on light source brightness information contained in each lane aerial image; and a region nighttime lighting information determination subunit, configured to determine nighttime lighting information for the target region based on the nighttime lighting information of each lane contained within the target region.

15. The apparatus according to claim 14, wherein, The target region determination unit is further configured to: The area where no nighttime lighting information has been recorded is defined as the target area; The area that has recorded nighttime lighting information but has not updated the previously recorded nighttime lighting information within a preset time period is identified as the target area.

16. The apparatus according to claim 14, wherein, The target region determination unit includes: The abnormal trajectory determination subunit is configured to determine whether there is an abnormal trajectory in the user's trajectory where the speed of travel decreases by more than a preset degree; The trajectory fusion subunit is configured to perform clustering processing on the abnormal trajectories to obtain fused trajectories; The target region determination subunit is configured to determine the region in the fused trajectory corresponding to the portion of the trajectory where the travel speed decreases and the speed is maintained after the decrease as the target region.

17. The apparatus according to claim 16, wherein, The target region determination subunit includes: The speed change trajectory determination module is configured to determine the portion of the fused trajectory that corresponds to the decrease in travel speed and the maintenance of the decreased speed as the speed change trajectory; The convex hull processing module is configured to perform convex hull processing on the velocity change trajectory to obtain a convex hull; The target region determination module is configured to determine the target region based on the region covered by the convex hull.

18. The apparatus according to claim 17, wherein, The target region determination module is further configured to: Determine the centroid of the convex hull; The target region is obtained by taking the centroid as the region center and the preset length as the region radius.

19. The apparatus according to claim 14, wherein, The edge also has a priority weight that matches the difference in lane priority between the two representative points it connects; wherein, the types of lanes include motor vehicle lanes, non-motor vehicle lanes and pedestrian lanes, with the pedestrian lane having the highest priority, the motor vehicle lane having the lowest priority, and the non-motor vehicle lane having a priority between the pedestrian lane and the motor vehicle lane, and the number of each type of lane includes multiple lanes.

20. The apparatus according to claim 19, wherein, The flight route planning unit is further configured to: Based on the distance weights, priority weights, and simulated annealing algorithm of the edges connecting each representative point in the complete graph, a complete flight planning route passing through all the representative points is calculated.

21. The apparatus of claim 20, further comprising: The flight route splitting unit is configured to, in response to the planned flight distance of the complete flight route exceeding the maximum flight distance of the UAV, split the complete flight route into at least two sub-flight routes that only pass through a portion of the representative points, according to the maximum flight distance; wherein the sum of the number of times each of the sub-flight routes passes through any of the representative points is not less than 1. Correspondingly, the aerial image acquisition unit is further configured as follows: The orthophotos are obtained by the UAV flying sequentially along different sub-flight planning routes.

22. The apparatus according to any one of claims 14-21, further comprising: A preprocessing unit is configured to perform preprocessing operations on the aerial imagery of the lane to obtain a preprocessed image; wherein the preprocessing operations include: enhancing contrast and removing noise; Correspondingly, the lane nighttime lighting information determination subunit includes: The grayscale conversion and brightness threshold determination module is configured to convert the preprocessed image into a grayscale image and determine the brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method. The illumination area and light source information determination module is configured to determine the illumination area based on the brightness threshold and determine the light source information based on the illumination area; The street light source determination module is configured to exclude non-street light sources based on the light source information to obtain street light sources; The lane nighttime lighting information determination module is configured to determine the brightness information of the illumination area provided by the street light source as the nighttime lighting information of the corresponding lane.

23. The apparatus according to claim 22, wherein, The grayscale conversion and brightness threshold determination module includes a brightness threshold determination submodule configured to determine a brightness threshold for segmenting the foreground and background based on the grayscale image and the maximum inter-class variance method. The brightness threshold determination module is further configured to: The number of pixels at each preset gray level is determined based on the grayscale image. Calculate the number of foreground and background pixels, grayscale mean, and variance for each grayscale level; The inter-class variance at different gray levels is calculated using the maximum inter-class variance method, and the brightness threshold used to segment the foreground and the background is determined based on the maximum inter-class variance.

24. The apparatus according to claim 22, wherein, The illumination area and light source information determination module includes an illumination area determination submodule configured to determine the illumination area based on the brightness threshold, and the illumination area determination submodule is further configured to: For each pixel with an actual gray value exceeding the brightness threshold, the illumination area is obtained through connected component analysis.

25. The apparatus according to claim 22, wherein, The illumination area and light source information determination module includes a light source information determination submodule configured to determine light source information based on the illumination area, and the light source information determination submodule is further configured to: Based on the shape of the illuminated area and the propagation characteristics of light, determine the position and movement characteristics of the light source; Correspondingly, the streetlight light source determination module is further configured to: Light sources whose location is more than a preset distance from their lane and / or whose movement characteristics are moving are identified as non-streetlight light sources; Other light sources that are distinct from the non-streetlight light sources are identified as the streetlight light sources.

26. The apparatus of claim 25, further comprising: The aerial photography pause control unit is configured to prevent the drone from taking aerial images during periods when vehicles with their lights on are present.

27. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the road nighttime lighting information determination method according to any one of claims 1-13.

28. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method for determining road nighttime lighting information according to any one of claims 1-13.

29. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method for determining road nighttime lighting information according to any one of claims 1-13.

Images