Street lamp identification method and device, storage medium and program product

Abstract

The invention provides a street lamp recognition method and device, a storage medium and a program product, and relates to the technical field of image recognition, and the method comprises the steps: obtaining a road image and the road information of a current position, and carrying out the feature extraction of the road image, and obtaining a global image feature and a local image feature; performing fusion processing on the global image features and the local image features to obtain multi-scale features, and performing street lamp detection based on the multi-scale features to obtain a detection result; the detection result comprises the existence state of the street lamp, the street lamp type and the spatial relationship between the street lamp and the road; under the condition that the existence of the street lamp is determined based on the detection result, the topological relation between the street lamp and the road is reasoned based on the detection result and the road information, the action road and the target position of at least one street lamp are determined, and the accuracy of street lamp detection and road association judgment is improved.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a street light recognition method, device, storage medium, and program product. Background Technology

[0002] In map production, navigation services, and the construction of urban digital twin systems, streetlights along roads serve as key infrastructure elements, not only providing safety for nighttime driving but also serving as an important data foundation for intelligent traffic management, vehicle-road collaboration, and refined urban governance.

[0003] In existing technologies, there are two main ways to supplement street light elements in maps or digital twin models: one is to rely on manual operation, where professionals manually mark the location and attributes of street lights in the map layer based on images or on-site survey results; the other is to use automated recognition methods based on aerial images, which use computer vision algorithms to detect street lights from aerial images and then map the recognition results to geographic space.

[0004] However, manual methods are inefficient and costly due to their reliance on manpower, making them unsuitable for large-scale road networks and dynamic updates, and prone to introducing subjective errors. Aerial image-based recognition methods, on the other hand, are limited by shooting angle, resolution, occlusion, and lighting conditions, resulting in low accuracy in recognizing streetlights of varying shapes and sizes. Summary of the Invention

[0005] This application provides a street light recognition method, device, storage medium, and program product. By automating image processing, recognition, and reasoning, and outputting multi-scale features, it significantly improves data processing and feature extraction efficiency compared to manual operations, while reducing labor costs. Furthermore, since the input image originates from road images captured during vehicle movement, its perspective is closer to the ground and the imaging is clearer, effectively mitigating common problems in aerial photography such as missed recognition, misidentification, incorrect type judgment, and inaccurate inference of road relationships.

[0006] Firstly, this application provides a street light identification method, the method comprising:

[0007] Acquire road images and road information at the current location, and extract features from the road images to obtain global and local image features;

[0008] Global and local image features are fused to obtain multi-scale features, and streetlights are detected based on these multi-scale features to obtain detection results. The detection results include the presence status of streetlights, streetlight type, and the spatial relationship between streetlights and roads.

[0009] If the presence of streetlights is confirmed based on the detection results, the topological relationship between the streetlights and the road is inferred based on the detection results and road information to determine the road and target location of at least one streetlight.

[0010] Secondly, this application provides a street light identification device, the device comprising:

[0011] The acquisition module is used to acquire road images and road information at the current location, and to extract features from the road images to obtain global image features and local image features;

[0012] The processing module is used to fuse global and local image features to obtain multi-scale features, and to detect streetlights based on the multi-scale features to obtain detection results. The detection results include the presence status of streetlights, streetlight type, and spatial relationship between streetlights and roads.

[0013] The reasoning module is used to infer the topological relationship between streetlights and roads based on the detection results and road information when the presence of streetlights is determined, and to determine the road and target location of at least one streetlight.

[0014] Thirdly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0015] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the first aspects.

[0016] This application provides a street light recognition method, device, storage medium, and program product. It employs an automated recognition and reasoning method based on image and road information fusion. By acquiring road images and road information at the current location, feature extraction is performed on the road images to obtain global features reflecting the overall scene and local features focusing on details. Then, these two types of features are fused into multi-scale features, and street light target detection is performed based on these features. The output includes the street light's presence status, type, and spatial relationship with the road. Furthermore, if a street light is detected, the topological relationship between the street light and the road is inferred based on the detection results and road information, thereby determining the road it serves and its specific installation location. This automatic detection and attribute analysis of street lights using feature extraction and fusion technology replaces the manual annotation process, significantly improving processing efficiency, reducing labor costs, and avoiding subjective errors. Moreover, this application uses road images (such as vehicle-mounted or ground-acquired images) instead of aerial images, avoiding occlusion and deformation problems from an aerial perspective. By simultaneously extracting and fusing global and local image features, the model's ability to capture multi-scale and multi-morphological streetlight features is enhanced, improving its robustness in recognizing small-sized and differently shaped streetlights. Furthermore, by incorporating road information for spatial relationship and topological reasoning, the accuracy of streetlight detection and its association with roads is improved. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0019] Figure 2 A schematic flowchart illustrating a street light recognition method provided in an embodiment of this application;

[0020] Figure 3 A flowchart illustrating an optional street light recognition method provided in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the structure of a street light recognition device provided in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0023] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0025] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0026] First, let me explain the terms used in this application:

[0027] Streetlights: These can refer to fixed lighting devices installed along roads or in related areas, primarily for providing public lighting. They are used for road traffic illumination and typically include, but are not limited to: freestanding pole-mounted streetlights erected on both sides of the road, and auxiliary lighting fixtures attached to utility poles, bridges, tunnel walls, or other structures.

[0028] The term "streetlight" can refer to the road section that provides effective lighting, depending on its installation location, lighting direction, and beam coverage.

[0029] Specialty streetlights: A type of streetlight that differs from standard road lighting devices such as conventional single-arm and double-arm lights, with its pole, lamp head, or overall shape having a special design or containing regional characteristics.

[0030] Multidimensional feature recognition refers to the process of extracting and analyzing multi-attribute information of a target object (such as a street lamp) concurrently and collaboratively through integrated perception and computing technologies. For example, it can be the process of simultaneously extracting information from multiple dimensions such as the street lamp's state of existence, type, density distribution, spatial location, and physical height.

[0031] Topological reasoning mechanism: This can refer to a computational process that uses preset logical rules or machine learning models to deduce the geometric relationships, relative positions, and connectivity between target objects (such as streetlights) in images or spatial data, as well as between target objects and known structured environments (such as road networks), in order to determine the road category to which the streetlights belong and their layout logic.

[0032] In related technologies, the identification and labeling of streetlight elements in map and digital twin scenarios mainly rely on two methods. In one possible implementation, operators manually determine the characteristics of streetlights and assess their spatial location based on images or on-site experience, and then manually label them on the map layer.

[0033] However, manual methods are usually inefficient, difficult to cope with large-scale road networks and dynamic update requirements, and limited by the subjective judgment and spatial estimation ability of the operators, making it difficult to unify standards and easily leading to some labeling errors and spatial location estimation errors.

[0034] Another possible implementation is an automated recognition method based on aerial images. This method uses algorithms to detect and select streetlights in aerial images, and then converts the recognition results into geographic coordinates for road sign labeling on maps or city digital twin models.

[0035] However, since aerial photography is usually conducted from a distance, if the streetlights in the image are small or have indistinct features, it is easy to miss the identification of small-sized or low-contrast streetlights and make type identification errors. At the same time, due to the lack of accurate perception of the ground road structure, this method is difficult to accurately determine the relationship between streetlights and specific roads, resulting in incorrect road binding.

[0036] In addition, aerial data collection is costly, and the recognition model has limited accuracy in distinguishing types in complex scenarios, which can easily lead to missed recognition or incorrect road identification. In particular, when multiple roads are intertwined in spatial relationships, the automated binding logic is complex and prone to errors, resulting in low overall recognition accuracy.

[0037] To address the aforementioned issues, this application provides a streetlight recognition method. It employs an automated recognition and reasoning approach based on image and road information fusion. By acquiring road images and road information at the current location, feature extraction is performed on the road images to obtain global features reflecting the overall scene and local features focusing on details. Then, these two types of features are fused into multi-scale features, and streetlight target detection is performed based on these features. The output includes the streetlight's presence status, type, and spatial relationship with the road. Furthermore, if a streetlight is detected, the topological relationship between the streetlight and the road is inferred based on the detection results and road information, thereby determining the road it serves and its specific installation location. This method, utilizing feature extraction and fusion technology to achieve automatic streetlight detection and attribute analysis, replaces the manual annotation process, significantly improving processing efficiency, reducing labor costs, and avoiding subjective errors. Moreover, this application uses road images (such as vehicle-mounted or ground-acquired images) instead of aerial images, avoiding occlusion and deformation problems from an aerial perspective. By simultaneously extracting and fusing global and local image features, the model's ability to capture multi-scale and multi-morphological streetlight features is enhanced, improving its robustness in recognizing small-sized and differently shaped streetlights. Furthermore, by incorporating road information for spatial relationship and topological reasoning, the accuracy of streetlight detection and its association with roads is improved.

[0038] For example, Figure 1 This is a schematic diagram of an application scenario provided in an embodiment of this application, such as... Figure 1 As shown, this application takes an autonomous vehicle as an example. The application scenario includes a vehicle 102 and a cloud-based server 101. The vehicle 102 uses an onboard camera to collect real-time images of the road ahead and to the sides, simultaneously obtaining its current location and associating it with road information in a high-definition map (HD) or road network database, such as lane lines, road geometry, topology, and spatial location information. The vehicle 102 can obtain its current location through a Global Positioning System (GPS), an Inertial Measurement Unit (IMU), or a high-definition positioning module. This application does not specifically limit the method of obtaining the current location; the above is merely an illustrative example.

[0039] Furthermore, vehicle 102 sends road images and road information of its current location to server 101. Server 101 performs preprocessing operations such as distortion correction, scaling, etc., on the acquired road images to adapt them to the model input. Further, server 101 uses a pre-trained dual-branch feature extraction network to extract global features (road scene context) and local features (detail texture) of the road images in parallel, and fuses the global and local features through stitching, weighting, or attention mechanisms to generate multi-scale features.

[0040] Furthermore, server 101 inputs multi-scale features into the detection model (such as an object detection network) and outputs detection results, which may include:

[0041] The presence and location of street light 103 are indicated by a detection frame.

[0042] Streetlight 103 type: used to distinguish the types of streetlights 103, such as single-arm lights, double-arm lights, high-mast lights, etc.;

[0043] Spatial relationship: used to make a preliminary judgment on the orientation and density distribution of streetlights 103 relative to the road, such as whether they are located on the left or right side of the road, and the corresponding arrangement in the road.

[0044] Furthermore, based on the aforementioned detection results and road information, the topological relationship between the streetlight 103 and the road is inferred to obtain the service road and target location of at least one streetlight 103. For example, based on the detection results, a first inference model can be used to determine whether the streetlight 103 serves the current road, its lateral orientation (left and right position relative to the lane lines), and its longitudinal position (relative distance along the road direction). Then, combined with road information (such as lane direction, slope, curvature, etc.), a second inference model is used to determine the following: the driving direction (up / down), precise location coordinates (latitude, longitude, and elevation) served by the streetlight 103, its vertical height (height from the bottom of the light pole to the road surface), and its sequential arrangement along the road (the spatial arrangement of the streetlight 103 on the road).

[0045] Optionally, based on the standardized interface specifications of the map production platform, such as the standardized interface specifications corresponding to the Open Dynamic Road Information for Vehicle Environment (OpenDRIVE) specification, the attributes of streetlight 103, such as the road it serves, its target location, and its streetlight type, are converted into structured data. The standardized streetlight data is then sent to the map production platform, where it is automatically labeled to the corresponding locations on the map, enabling dynamic map updates. This involves adding or updating streetlight layers on the map, containing the precise spatial location, attributes, and topological relationships of each streetlight 103. The server 101 then sends the updated map to the vehicle 102 for use by the autonomous driving vehicle 102 for localization, path planning, and environmental perception.

[0046] It should be noted that this application can be applied not only to maps for autonomous driving, but also to maps on terminal devices used by ordinary users and to city digital twin products. The embodiments of this application do not limit the specific application scenarios.

[0047] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0048] For example, Figure 2 This is a flowchart illustrating a street light recognition method provided in an embodiment of this application, as shown below. Figure 2 As shown, the executing entity of this street light recognition method can be the aforementioned cloud server or a vehicle; this application embodiment does not specifically limit this. The street light recognition method includes the following steps:

[0049] S201. Obtain road images and road information at the current location, and extract features from the road images to obtain global image features and local image features.

[0050] In this embodiment, road images can refer to digital images of roads and their surrounding environment acquired by ground-based acquisition devices (such as vehicle-mounted or fixed cameras). Optionally, the road images can originate from data collection vehicles or crowdsourced data collection; this embodiment does not specifically limit the source of the road images.

[0051] The road information at the current location can refer to structured road data corresponding to the image acquisition location, including but not limited to road geometric attributes, topological relationships, lane information, and spatial coordinates. Optionally, the road information may include spatial location information and road matching information obtained based on the identification information of the road image. For example, the identification information may include a unique code (Identity Document, ID), timestamp, geographic coordinates, etc. This application embodiment does not specifically limit this.

[0052] Global image features can refer to feature vectors extracted from the entire road image that represent the contextual information of the overall layout of the scene.

[0053] Local image features can refer to feature vectors that represent detailed textures, edges, and small-scale objects from specific regions in a road image or extracted through local perception operations.

[0054] Contextual information refers to the features extracted from road images that characterize the overall structure and semantic relationships of the environment in which the streetlights are located. This contextual information can reflect the macroscopic correlations between the streetlights and other elements in the scene (such as roads, lane lines, and surrounding facilities), including spatial layout, relative position, and scene category, and is used to understand the role and constraints of the streetlights in the overall environment. Detailed information focuses on the specific attributes of the streetlights, such as their structural composition, surface morphology, component shape, and micro-texture, and is used to distinguish and identify the precise category and individual state of the streetlights.

[0055] In this step, image data of the current road is acquired using a data acquisition device, and road information corresponding to the current location is acquired simultaneously. The road image and road information are then sent to a cloud server, which performs feature extraction on the road image. This extraction process includes global image features reflecting the overall scene structure, as well as local image features focusing on image details, textures, and small-scale targets.

[0056] For example, global and local image features of a road image can be extracted in parallel using a dual-branch feature extraction network model. This application does not specifically limit the specific algorithm model corresponding to the dual-branch feature extraction network model; for example, the dual-branch feature extraction network model can be a convolutional neural network model, a multi-scale feature extraction network model, etc.

[0057] S202. The global image features and local image features are fused to obtain multi-scale features, and streetlights are detected based on the multi-scale features to obtain detection results. The detection results include the existence status of streetlights, streetlight types, and the spatial relationship between streetlights and roads.

[0058] In this embodiment, multi-scale features can refer to a unified feature representation formed by fusing global and local image features, which simultaneously contains information at different scales (i.e., different ranges and granularities). For example, global image features correspond to the macroscopic scale, representing the overall scene context information of the road image; local image features correspond to the microscopic scale, representing the texture and fine structure details of the detailed areas in the road image.

[0059] It should be noted that global image features can capture the contextual information of streetlights in road images, which helps to understand the overall spatial relationship between streetlights and roads. Local image features, on the other hand, focus on the detailed information of streetlights in road images, which can identify the fine structure of streetlights and thus support the discrimination of the existence and specific type of streetlights. Therefore, by fusing global and local image features to generate multi-scale features, and performing target detection based on multi-scale features, the contextual information provided by global image features and the detailed information provided by local image features complement each other. This not only detects the presence and type of streetlights, but also infers the relative position and arrangement of streetlights and roads, thereby determining the spatial relationship.

[0060] Furthermore, object detection based on multi-scale features can utilize pre-trained detection models. The output of these models is specially trained, and by fusing contextual and detailed information, these models can infer the existence status, type, and spatial relationships of streetlights relative to the road, such as their orientation and distribution.

[0061] The presence status of streetlights can refer to whether there are streetlights in the current road image, which can be determined by whether a detection box for the corresponding target is output.

[0062] The type of street light can refer to a specific category, such as single-arm street lights, double-arm street lights, high-mast lights, ordinary community lights, and special street lights. It can be classified according to its shape or function, or it can be categorized based on street lights in the real world. This application does not specifically limit the type of street light. Among them, special street lights can include various types such as multi-layered top-focusing spheres, single-layered top-focusing spheres, pendant spheres, bottom-focusing spheres, symmetrical double-arm lights, solar panel street lights, and irregularly shaped lights.

[0063] The spatial relationship with the road can refer to the positional relationship of the detected streetlights relative to the road geometry, such as the orientation relationship and density distribution relative to the road. The orientation relationship can include the orientation description in the horizontal direction (such as the left / right side of the road) and the vertical direction (such as the relative position of the streetlights with the carriageway and sidewalk).

[0064] In this step, the extracted global image features and local image features can be fused to generate multi-scale features that simultaneously contain overall scene information and local details. Based on these multi-scale features, a target detection model is used to identify streetlight targets in the image, outputting detection results that include the streetlight's presence status, specific type, and its spatial relationship with the road.

[0065] For example, road images can be input into object detection and classification models to detect the presence and type of streetlights, detect density distribution, and detect road positional relationships.

[0066] The target detection and classification model includes the dual-branch feature extraction network model and the target detection model mentioned in the above embodiments. The target detection and classification model is designed by modifying and training the model structure using data-labeled samples. It can detect the presence or absence of streetlights in the image and perform type classification, density distribution detection, and road position relationship detection for detected streetlights.

[0067] Optionally, the training process of the object detection and classification model includes: obtaining a training sample set, which includes multiple samples, each sample including a road image and labeling information such as the type, density distribution, and road location relationship of streetlights in the road image; inputting the training sample set into the object detection and classification model for training to obtain a trained object detection and classification model.

[0068] S203. If the presence of streetlights is determined based on the detection results, the topological relationship between the streetlights and the road is inferred based on the detection results and road information to determine the road and target location of at least one streetlight.

[0069] In this embodiment of the application, the topological relationship can refer to the spatial connection, adjacency or affiliation between the street light and the road, which is used to describe which road the street light acts on and its relative location.

[0070] The target location can refer to the specific location of the street light in the actual geographic space. It is usually described by mapping the image detection results and road information to geographic coordinates or the precise location relative to the road.

[0071] In this step, assuming the presence of a street light is detected, the detection results can be combined with the road information at the current location. Through spatial analysis and logical reasoning, the topological connection relationship between at least one street light and the road can be determined, thereby identifying the specific road segment served by each street light and its actual installation target location.

[0072] In this application, the determination of the topological connection relationship between at least one street lamp and the road through spatial analysis and logical reasoning can be achieved using a deep learning model or other algorithms, and this application does not specifically limit the specific algorithm used.

[0073] For example, reasoning about the topological relationship between streetlights and roads using a topological reasoning model can include: determining whether a streetlight is acting on the current road based on the relative positional relationship of detection boxes in the detected road image; and determining whether a streetlight is installed on both sides of the current road based on the orientation of the detection box relative to the road centerline, thereby outputting the road on which the streetlight acts and the orientation of its installation.

[0074] Therefore, automatic recognition and reasoning based on road images and road information can replace manual operations, rapidly process large-scale road image data, adapt to the dynamic update needs of road networks, significantly improve processing efficiency, reduce labor and time costs, and meet the needs of large-scale road networks and dynamic updates, avoiding subjective errors and achieving automated and efficient processing. Furthermore, this application uses ground road images to avoid the limitations of aerial photography perspectives and enhances the representation ability of multi-scale and multi-morphological targets through the fusion of global and local image features, effectively overcoming recognition difficulties caused by shooting angle, resolution, occlusion, and lighting conditions, and reducing false detections and missed detections caused by occlusion, lighting, and angle changes. In particular, it improves the detection accuracy of small-sized, diverse-shaped streetlights, enhancing recognition accuracy and robustness. Furthermore, based on detection, it combines road information for reasoning, automatically determining the road on which the streetlight functions and the target location of its installation, completing end-to-end analysis from image recognition to geospatial association, improving the accuracy of judging the spatial relationship between streetlights and roads, providing structured data that can be directly used for map construction and management, and achieving accurate spatial association and topological analysis.

[0075] It should be noted that this application adopts a multi-scale feature fusion recognition method, which combines image semantic segmentation with target detection, and simultaneously identifies the existence status, sub-type, and density distribution of streetlights within a unit road segment. This breaks through the limitation of traditional methods that can only identify a single attribute and significantly improves the recognition accuracy.

[0076] Furthermore, this application employs a road-streetlight topology reasoning mechanism, which, based on image geometric relationships and road network topology logic, infers the road range and specific spatial location of the streetlights, thus achieving accurate spatial mapping.

[0077] Optionally, the global image features and local image features are fused, including at least one of the following methods:

[0078] Global and local image features are convolutionally fused by feature map stitching.

[0079] Global and local image features are fused using a weighted summation of feature maps.

[0080] The global and local image features are weighted and fused using an attention mechanism.

[0081] In this embodiment of the application, the feature map stitching method can refer to connecting global image features and local image features in the channel dimension or spatial dimension to form a merged feature map, and then performing fusion and dimensionality reduction through convolution operation.

[0082] The weighted summation of feature maps can refer to assigning weights (learnable or fixed) to the global feature map and the local feature map respectively, and then linearly combining the two feature maps into a new feature map by weighted summation.

[0083] Attention mechanisms can refer to a method of dynamically weighting and fusing global and local image features by calculating the importance weights of each position in the feature map.

[0084] In this step, global and local image features can be fused in multiple dimensions using feature map concatenation, weighted summation, or attention mechanisms to obtain multi-scale features. For example, in feature map concatenation, global image features obtained from a 50-layer Residual Network (ResNet50) and local image features obtained from a CrossStage Partial Network (CSPNet) are concatenated along the channel dimension to form a fused feature map containing both global contextual and detailed information, which is then used for subsequent classification and topological inference. Alternatively, the extracted global and local image features, after mapping, are added together using an add (concatenation) method, followed by convolutional fusion. The fused multi-scale features are then input into the three feature layers of the prediction head of the object detection model for street light presence / absence detection, density detection, type classification, and output of their positional relationship with the road.

[0085] Because this application provides a flexible and selectable fusion strategy to fuse global and local image features, it can adapt to the needs of different scenarios and model structures, improve the model's ability to represent street light targets and the detection accuracy, and highlight important features through the effective integration of multi-scale features and adaptive fusion methods such as attention mechanisms, thereby further enhancing the robustness of the detection results.

[0086] Optionally, before performing feature extraction on the road image, the method further includes:

[0087] Preprocess the road images.

[0088] In this embodiment of the application, preprocessing may refer to a series of technical operations performed on the original road image before image analysis, which aim to improve image quality, correct geometric deformation, unify data specifications, and provide standardized input for subsequent processing steps.

[0089] Optionally, preprocessing includes, but is not limited to, image distortion correction, geometric correction, resizing, rotation and flipping, cropping, denoising and filtering, etc., to eliminate distortion, noise and inconsistency introduced during image acquisition, so as to standardize the image data and adapt it to subsequent processing procedures.

[0090] In this way, by preprocessing road images, image quality can be improved, feature extraction errors caused by device distortion, viewing angle tilt, noise interference, etc. can be reduced, processing efficiency can be improved and the consistency of feature extraction can be ensured, providing more reliable input for subsequent detection and inference.

[0091] Optionally, feature extraction is performed on the road image to obtain global image features and local image features, including:

[0092] The pre-trained dual-branch feature extraction network model extracts global and local image features of road images in parallel; the dual-branch feature extraction network model includes a first branch and a second branch;

[0093] The first branch is used to extract a first feature map with a first receptive field from the road image, and the first feature map contains local image features; the second branch is used to extract a second feature map with a second receptive field from the road image, and the second feature map contains global image features; the first receptive field is smaller than the second receptive field.

[0094] In this application, the training process of the dual-branch feature extraction network model includes: obtaining a training dataset, which includes different types of road images, as well as global image features and local image features corresponding to each road image; inputting the training dataset into the dual-branch feature extraction network model for training, thereby obtaining a pre-trained dual-branch feature extraction network model.

[0095] In this embodiment of the application, the first feature map may refer to the feature map output by the first branch in the dual-branch feature extraction network, which focuses on capturing local detail features and corresponds to local image features.

[0096] The second feature map can refer to the feature map output by the second branch in the dual-branch feature extraction network, which focuses on capturing global context features and corresponds to global image features.

[0097] The receptive field refers to the size of the region in the original input image corresponding to each pixel in the feature map, including scale and distance. The larger the receptive field, the wider the original image region corresponding to the feature. Therefore, the first receptive field corresponding to the first feature map is smaller than the second receptive field corresponding to the second feature map.

[0098] The first feature map and the second feature map can be extracted using different feature extraction networks.

[0099] Optionally, the dual-branch feature extraction network model can be designed as a dual-backbone structure to improve the model's ability to perceive details at different scales and distances. In this dual-backbone structure, two independent feature extraction networks are used in parallel within the same neural network model. Branch 1 of the backbone structure can employ a large receptive field network, such as ResNet50, for feature extraction, specifically for global image features. Branch 2 of the backbone structure employs a small receptive field network, such as an improved CSPNet, for feature extraction, specifically for local image features.

[0100] It should be noted that due to the structural improvements in the dual-branch feature extraction network model, the accuracy of global recognition is improved, and it is better at distinguishing types of objects that are far apart and small or similar in shape, thereby improving the accuracy of multi-class recognition.

[0101] Therefore, by designing a dual-branch feature extraction network model for parallel feature extraction, we can avoid the loss of local details caused by downsampling of deep features in serial networks, ensuring the spatial accuracy of local features. Furthermore, the first feature map with a small receptive field preserves local details of the streetlight (such as lamp head structure and texture), while the second feature map with a large receptive field captures global contextual information of the road scene (such as road direction and surrounding environment). This overcomes the limitations of single-scale features in dealing with streetlights of varying sizes, shapes, or partial occlusion, improving the completeness of feature representation and thus increasing the detection accuracy for small or varied streetlights. The parallel structure can also adapt to different computational complexities for different receptive field branches, reducing redundant computation while maintaining performance. In addition, acquiring local image features enhances the recognition ability of small-scale or partially occluded streetlights, while acquiring global image features helps understand the spatial relationship between streetlights and roads, thus jointly improving detection stability in complex scenes.

[0102] Optionally, the presence of streetlights is characterized by the presence or absence of detection boxes in the target image; the spatial relationship includes the density distribution of streetlights on the road and their orientation information on the road, wherein the number of detection boxes represents the density distribution and the position of the detection boxes in the target image represents the orientation information.

[0103] In this embodiment of the application, the detection box may refer to the bounding box used to identify the location of the target in target detection, and its existence indicates that the model has recognized the corresponding target.

[0104] Density distribution can refer to the spatial density and distribution pattern of the number of streetlights within a given road area or image range.

[0105] Orientation information refers to the horizontal position of a street light relative to a road reference system (such as lane lines or the center line of the road). This orientation information is used to determine the relative horizontal orientation of the street light, such as whether it is located on the left or right side of the road or next to a specific lane.

[0106] In this application, the number of detection boxes is used to infer the distribution density of streetlights along the road, while the orientation information of the detection boxes in the target image is further analyzed to determine the orientation relationship of the streetlights relative to the road.

[0107] The target image can refer to the visual output image obtained after street light detection processing, which is an original road image with zero or at least one detection box superimposed on it. The target image can intuitively represent the existence of street lights and the spatial relationship between street lights and roads. For example, the presence of detection boxes in the target image indicates the presence of street lights, and the position and number of detection boxes in the target image can represent the orientation information and density distribution of street lights.

[0108] In this way, by detecting the presence, quantity, and position of detection boxes, the existence status, density distribution, and orientation information of streetlights are unified within the target detection framework, realizing a structured and visual representation of the spatial attributes of streetlights. Specifically, the presence status of streetlights is directly determined by the presence or absence of detection boxes, simplifying the status judgment logic and improving processing efficiency. The number of detection boxes is converted into streetlight density distribution data, and the relative orientation of streetlights and roads is inferred based on the position of the detection boxes in the target image, enhancing the interpretability and practicality of spatial relationships in the detection results. This allows for more refined topological relationship reasoning and spatial association analysis in the future.

[0109] Optionally, based on the detection results and road information, the topological relationship between the streetlights and the road is inferred to determine the road and target location of at least one streetlight, including:

[0110] The target image containing the detection box, spatial relationships, and road information are inferred to obtain at least one target topological relationship between a street lamp and a road.

[0111] Spatial location calculations are performed based on the target topology to determine the road and target location of at least one street light.

[0112] In this embodiment, the target topology relationship refers to the logical association between streetlights and roads based on spatial connectivity, attribution, and relative orientation. It describes the road to which the streetlight belongs, the road segment it serves, and its position relative to the road direction, lane, or intersection. This target topology relationship may include the road the streetlight serves, the streetlight's lateral orientation, longitudinal offset, vertical height, and sequence arrangement relative to the current road. This embodiment does not limit the specific content corresponding to the target topology relationship.

[0113] In this optional step, by combining the target image containing the detection box and spatial relationship information with the road information provided by the high-precision map, the spatial association and constraints between the streetlights and the road are analyzed through inference processing to obtain the target topological relationship describing their connection, affiliation, or relative positional relationship. Furthermore, based on this target topological relationship, the functioning road served by each streetlight and its precise target location are determined through spatial location calculation.

[0114] It should be noted that spatial location calculation can refer to the process of calculating the precise three-dimensional coordinates of the streetlights in a specified coordinate system (such as a geographic coordinate system or a planar projection coordinate system) by combining the spatial association attributes between the streetlights and the road based on the target topological relationship and the road's geometric data (such as centerline coordinates, lane width, and road surface elevation) with geometric modeling and coordinate transformation methods.

[0115] For example, the process of determining the service road and target location of each street light through spatial location calculation includes:

[0116] The target topology is analyzed to extract the streetlight's function road identification, lateral orientation, longitudinal position, and sequence arrangement order relative to the road. Further, based on the function road identification, the geometric data of the corresponding road is retrieved from the road information database, including the continuous coordinate sequence of the road centerline and lane width. Based on the longitudinal position, linear interpolation is performed on the road centerline coordinate sequence to determine the reference point coordinates of the streetlight in the road's extension direction. Based on the lateral orientation and lane width, the lateral offset is calculated, and the horizontal position is obtained by translation based on the direction perpendicular to the road tangent at that point. Then, combined with the vertical height in the target topology, the road surface elevation is superimposed to generate the streetlight's three-dimensional coordinates. Further, the calculated three-dimensional coordinates are verified to ensure they are within the current road geometry, confirming the function road affiliation. Streetlights on the same road are sorted and numbered according to their sequence arrangement order, outputting structured location data, including road ID, three-dimensional coordinates, and topological attributes, completing the precise positioning of the streetlights and their association with the road.

[0117] In this way, by combining visual detection results with road information for reasoning, a topological relationship between streetlights and roads is established, replacing traditional methods of manual judgment or simple geometric mapping, thus improving processing efficiency and accuracy. Furthermore, by constraining spatial location calculations through target topological relationships, the allocation of streetlight positions not only depends on image coordinates but also considers the structural logic of the road network, reducing erroneous associations caused by perspective, occlusion, or detection bias. This makes the calculation results more consistent with actual road lighting scenarios, thereby improving the structure and accuracy of map data.

[0118] Optionally, inference processing is performed on the target image containing the detection box, spatial relationships, and road information to obtain at least one target topological relationship between a street lamp and a road, including:

[0119] The target image containing the detection box and the spatial relationship are input into the first inference model to obtain the first-level topological relationship. The first-level topological relationship includes: whether at least one street lamp is active on the current road, and the lateral orientation, longitudinal position and vertical height of at least one street lamp relative to the current road surface.

[0120] Based on the first-level topological relationship and road information, the second-level topological relationship is inferred. The second-level topological relationship includes: the up and down direction information, location information and sequence arrangement information of at least one street light on the current road;

[0121] Construct the target topology based on the first-level and second-level topology relationships.

[0122] In this embodiment of the application, the first inference model may refer to a neural network or computational model used to infer the relationship between the street light and the current road basic space from the target image. For example, the first inference model may be a topological inference model.

[0123] The first-level topological relationship can refer to the direct spatial association between the street light and the current road, including whether the street light acts on the current road, its lateral orientation relative to the road, its longitudinal position, and its vertical height relative to the road surface.

[0124] The lateral orientation can refer to the horizontal position of the street light relative to the center line of the road or the roadway, such as the left, right or middle of the road.

[0125] Longitudinal position can refer to the relative position of a street light along the road's extension direction or the length information of its distance from a reference point (current position).

[0126] Vertical height can refer to the vertical distance from the mounting point of the street light to the road surface.

[0127] The second-level topology can refer to the extended spatial attributes of streetlights at the road network level, derived from the first-level topology and combined with road information. These attributes include the direction of travel, the specific coordinates of the location, and the arrangement order along the road.

[0128] The direction information can refer to the direction of traffic on the road served by the streetlight, such as the direction of travel up, down, or both directions.

[0129] Location information can refer to the specific coordinates or unique identifier of a street light within the road network.

[0130] The direction of extension can refer to the orientation of a road or the geometric direction in which a path extends.

[0131] Sequence arrangement information can refer to the order and spacing of multiple streetlights arranged along the direction of road extension.

[0132] For example, the target image containing the detection box and the spatial relationship between the street lamp and the road are input into the first inference model. The first inference model is integrated into the target detection and classification model. It outputs at least one first-level topological relationship between the street lamp and the current road through the position inference head. The first-level topological relationship includes at least one of the following: whether the street lamp acts on the current road, the lateral orientation of the street lamp relative to the current road, the longitudinal position of the street lamp relative to the current road, and the vertical height. The first inference model can learn and constrain the spatial inference results through the angle loss function and the spatial distance loss function to improve the inference accuracy.

[0133] The first-level topological relationship can be obtained by reasoning using the feature results (detection boxes) of the extracted target image and the spatial relationship of the inferred path.

[0134] Furthermore, based on the first-level topological relationship and road information, a second-level topological relationship of at least one streetlight in the road network is inferred. The second-level topological relationship includes at least one of the following: the streetlight's up / down direction information on the current road, its location information, and its sequential arrangement information along the road's extension direction. Further, a target topological relationship is constructed using the first-level and second-level topological relationships for subsequent calculation of spatial location relationships.

[0135] It should be noted that, compared to existing models, the object detection and classification model described above can integrate multiple sub-models, resulting in a significant improvement in the average accuracy of the multiple sub-models on multi-classification tasks, and more precise detection bounding boxes. These multiple sub-models include a bi-branch feature extraction network model, an object detection model, and a first inference model, etc. It is understood that these multiple sub-models may not be integrated into a single object detection and classification model, but rather set up separately; this application does not specifically limit this approach.

[0136] Therefore, through a two-level progressive analysis of topological relationships, the analysis gradually expands from the local spatial relationships of the current road network to the global structural relationships of the road network. This decomposes the complex spatial association problem into a hierarchical judgment from local to global, improving the systematicity and accuracy of the reasoning process and achieving hierarchical and precise reasoning. The first-level topological relationship focuses on the direct spatial relationship between streetlights and the current road, while the second-level topological relationship further clarifies the functional orientation and sequence logic of streetlights within the road network. The combination of these two levels makes the target topological relationship closer to the actual needs of road lighting management, thus providing a comprehensive and fine-grained spatial description of streetlights and enhancing the integrity of spatial attributes. Furthermore, by integrating the first-level topological relationship and road information for second-level reasoning, the complementarity of image and road data is fully utilized to enhance the robustness of topological relationship inference, making it easier to adapt to different road structures or detection conditions and improving adaptability to different scenarios.

[0137] Optionally, the method also includes:

[0138] The target image containing the detection box is converted into a target mask map, which is used to identify the pixel region of at least one street lamp in the road image;

[0139] The road image and target mask image are input into the second inference model to obtain the depth estimation result of at least one street lamp;

[0140] Based on the depth estimation results, some data in the target topology are corrected and the target topology is updated.

[0141] In this embodiment of the application, the target mask image may refer to a binary or class mask image with the same size as the original image, wherein the pixel area corresponding to the street lamp is marked with a specific value to accurately identify the pixel-level position of the detection box in the image.

[0142] The second inference model can refer to a neural network or geometric computation model used to estimate the target depth information from the original image and then from the target mask image. This application does not limit the specific model type corresponding to the second inference model. For example, the second inference model can be a modified monocular depth estimation (Visual Geometry Grounded Transformer, VGGT) model; this modified monocular depth estimation model is configured to: utilize the pixel region information identified by the target mask image to constrain the depth inference process, thereby enhancing the depth estimation accuracy of the target region.

[0143] It should be noted that the accuracy of the vertical height of at least one street lamp relative to the road surface and the longitudinal position of at least one street lamp relative to the current road obtained by the first inference model may not be high. Therefore, this application may also introduce a second inference model to correct the vertical height and longitudinal position of the street lamp.

[0144] For example, the target image containing the detection box is converted into a target mask map (box mask map). The road image and the target mask map are input into the second inference model. The second inference model is based on the improved VGGT architecture. It performs depth map inference by introducing the mapping relationship between the target recognition region and the 3D point cloud to generate a depth estimation result for at least one street lamp. Then, based on the depth estimation result and prior reference information, the vertical height of the street lamp relative to the road surface and its longitudinal position relative to the current road are corrected. The corrected longitudinal position and vertical height are used to update the corresponding data in the target topology relationship.

[0145] The depth estimation results include the estimated vertical height relative to the road surface and the longitudinal position of the streetlights relative to the current road. The vertical height is calculated using distance information provided by the depth map, combined with the pixel height of the streetlights in the road image (obtained through object detection boxes) and camera geometry (such as intrinsic parameters and pose), through triangulation or the principle of similar triangles. The longitudinal position is calculated using the distance information of the streetlights relative to the image acquisition location provided by the depth map, combined with the relative position of the image acquisition location and the road (using road information or prior reference information), to further estimate the spacing of the streetlights in the longitudinal direction of the road.

[0146] Therefore, this application achieves pixel-level streetlight area localization through target masking and a second inference model, and recovers 3D information by combining depth estimation, significantly improving the computational accuracy of depth estimation results and enhancing spatial positioning precision. Updating target topology relationships using corrected data provides more accurate streetlight area information, reduces positional errors caused by detection boxes, enhances the reliability of determining the road function of streetlights and calibrating target locations, and optimizes topology relationship inference. Furthermore, the depth estimation results can correct and compensate for geometric errors caused by viewpoint, occlusion, or image deformation, thereby providing more accurate streetlight spatial attributes, enhancing the accuracy and reliability of spatial data output, and providing high-quality structured data for high-precision map construction.

[0147] Optionally, this application may also employ a lightweight depth estimation method, utilizing the perspective variation relationship between the street lamp outline and the ground texture in a monocular road image, and based on a pre-designed low computational complexity height estimation algorithm, to achieve effective estimation of the physical height of the street lamp without the need for additional sensors.

[0148] Optionally, the method also includes:

[0149] Obtain the standardized data interface specifications of the target platform, as the standardized data interface specifications of different platforms are at least partially different;

[0150] Based on the standardized data interface specifications of the target platform, the street light type, the road it serves, and the target location of at least one street light are converted into a standardized output data.

[0151] Distribute standardized output data to the corresponding target platforms.

[0152] In this embodiment of the application, the standardized data interface specification may refer to the standardized requirements of the target platform for external data input, such as data structure, field definition, encoding format and transmission protocol.

[0153] Format conversion can refer to processing raw data according to the target interface specifications, such as structural adjustment, field mapping, and encoding conversion, to make it conform to the data input specifications of a specific platform.

[0154] For example, after determining the road and target location of at least one street light, the standardized data interface specifications of the target platform (such as map service, facility management system, etc.) are obtained; according to the standardized data interface specifications, the multi-dimensional feature data such as the type of street light, the road it serves, and the target location are fused and converted into a format to generate standardized output data that meets the platform requirements and is represented in structured JSON form; further, the standardized data is distributed to the corresponding target platform.

[0155] Therefore, by adapting to standardized interface specifications across different platforms, seamless integration of streetlight recognition data with various downstream systems (such as map services and facility management platforms) can be achieved, reducing integration complexity and cost while improving compatibility and efficiency. Furthermore, by converting the format of streetlights, the roads they serve, and the target locations, standardized data conforming to the requirements of each platform can be generated, ensuring a unified output data structure and clearly defined fields, guaranteeing data standardization and usability. This automated data conversion and distribution mechanism can adapt to the data synchronization needs of multiple platforms, enabling real-time or near-real-time push of recognition results, meeting the application requirements of dynamic updates and multi-terminal collaboration. In addition, standardized output makes streetlight recognition results easily received and processed by different platform systems, enhancing practical value and scalability.

[0156] In conjunction with the above embodiments, Figure 3 A flowchart illustrating an optional streetlight recognition method provided in an embodiment of this application is shown below. Figure 3 As shown, the street light identification method includes the following steps:

[0157] Step A: Obtain the original image (road image), preprocess the original image, and obtain the road information for the current location based on the original image.

[0158] Step B: Use object detection and classification models to detect and classify streetlights in the preprocessed original image to obtain the detection results.

[0159] Step C: Based on the detection results and road information, infer the topological relationship between streetlights and roads, and extract the streetlight topological relationship (first-level topological relationship). Correspondingly, the target image containing the detection box and the original image can also be processed to extract the streetlight depth information (depth estimation result). Combine the streetlight topological relationship, streetlight depth information and road information for processing to obtain the road and target location of at least one streetlight, and bind each streetlight to its road and target location.

[0160] The process of combining street light topology, street light depth information, and road information includes: correcting the street light topology using street light depth information, and inferring a second-level topology based on the corrected data and road information, and then using the street light topology and the second-level topology to calculate the spatial location.

[0161] It should be noted that the above processing procedure can be referred to the description of the above embodiments, and will not be repeated here.

[0162] Step D: Convert the street light type in the binding results and detection results into structured information for output, that is, convert it into standardized output data for output to different target platforms.

[0163] In this way, this application can achieve detailed type identification, existence status determination, distribution density analysis, target location calibration, road association, and height information estimation of streetlights along roads, thereby basically completing the full-element perception of streetlights in map services and urban digital twin systems. In particular, by extracting multi-dimensional feature elements of streetlights and transforming them into structured elements that can be directly used for map construction, it supports automated map construction and 3D production, thereby significantly improving the feature extraction capability and automated processing efficiency of original images.

[0164] It should be noted that the specific implementation principles and effects of the above steps can be found in the relevant descriptions and effects of the above embodiments, and will not be elaborated further here.

[0165] In the foregoing embodiments, the street light identification method provided by the embodiments of this application has been described. To implement the functions of the methods provided by the embodiments of this application, the electronic device serving as the execution subject may include hardware structures and / or software modules, implementing the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is executed in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0166] For example, Figure 4 This is a schematic diagram of the structure of a street light recognition device provided in an embodiment of this application, as shown below. Figure 4 As shown, the street light recognition device 400 includes: an acquisition module 401, used to acquire road images and road information at the current location, and to extract features from the road images to obtain global image features and local image features;

[0167] The processing module 402 is used to fuse global image features and local image features to obtain multi-scale features, and to perform street light detection based on the multi-scale features to obtain detection results; the detection results include the existence status of street lights, street light type, and spatial relationship between street lights and roads;

[0168] The reasoning module 403 is used to reason about the topological relationship between the streetlight and the road based on the detection results and road information when the presence of a streetlight is determined based on the detection results, and to determine the road and target location of at least one streetlight.

[0169] Optionally, the acquisition module 401 includes a feature extraction module, which is used for:

[0170] The pre-trained dual-branch feature extraction network model extracts global and local image features of road images in parallel; the dual-branch feature extraction network model includes a first branch and a second branch;

[0171] The first branch is used to extract a first feature map with a first receptive field from the road image, and the first feature map contains local image features; the second branch is used to extract a second feature map with a second receptive field from the road image, and the second feature map contains global image features; the first receptive field is smaller than the second receptive field.

[0172] Optionally, the presence of streetlights is characterized by the presence or absence of detection boxes in the target image; the spatial relationship includes the density distribution of streetlights on the road and their orientation information on the road, wherein the number of detection boxes represents the density distribution and the position of the detection boxes in the target image represents the orientation information.

[0173] Optional, inference module 403, specifically used for:

[0174] The target image containing the detection box, spatial relationships, and road information are inferred to obtain at least one target topological relationship between a street lamp and a road.

[0175] Spatial location calculations are performed based on the target topology to determine the road and target location of at least one street light.

[0176] Optionally, the inference module 403 includes an inference unit for:

[0177] The target image containing the detection box and the spatial relationship are input into the first inference model to obtain the first-level topological relationship. The first-level topological relationship includes: whether at least one street lamp is active on the current road, and the lateral orientation, longitudinal position and vertical height of at least one street lamp relative to the current road surface.

[0178] Based on the first-level topological relationship and road information, the second-level topological relationship is inferred. The second-level topological relationship includes: the up and down direction information, location information and sequence arrangement information of at least one street light on the current road;

[0179] Construct the target topology based on the first-level and second-level topology relationships.

[0180] Optionally, the street light recognition device 400 also includes a correction module, which is used for:

[0181] The target image containing the detection box is converted into a target mask map, which is used to identify the pixel region of at least one street lamp in the road image;

[0182] The road image and target mask image are input into the second inference model to obtain the depth estimation result of at least one street lamp;

[0183] Based on the depth estimation results, some data in the target topology are corrected and the target topology is updated.

[0184] Optionally, the street light identification device 400 further includes a distribution module, which is used for:

[0185] Obtain the standardized data interface specifications of the target platform, as the standardized data interface specifications of different platforms are at least partially different;

[0186] Based on the standardized data interface specifications of the target platform, the street light type, the road it serves, and the target location of at least one street light are converted into a standardized output data.

[0187] Distribute standardized output data to the corresponding target platforms.

[0188] It should be noted that the specific implementation principle and effect of the street light recognition device 400 can be found in the relevant description and effect of the above embodiments, and will not be elaborated further here.

[0189] For example, Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5As shown, the electronic device provided in the embodiments of this application may include: at least one processor 501; and a memory 502 communicatively connected to at least one processor; wherein the memory 502 stores instructions that can be executed by at least one processor 501, which are executed by at least one processor 501 to cause the electronic device to perform the method as described in any of the above embodiments.

[0190] Optionally, the memory 502 can be either standalone or integrated with the processor 501.

[0191] The memory 502 and the processor 501 can be connected via bus 503.

[0192] The implementation principle and technical effects of the electronic device provided in this application can be found in the foregoing embodiments, and will not be repeated here.

[0193] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any of the foregoing embodiments.

[0194] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the foregoing embodiments.

[0195] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0196] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application.

[0197] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor. The memory may include RAM (Random Access Memory), and may also include NVM (Non-Volatile Memory), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disc, etc.

[0198] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0199] The aforementioned storage media can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0200] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0201] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0202] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0203] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0204] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A street light identification method, characterized in that, The method includes: Acquire road images and road information at the current location, and extract features from the road images to obtain global image features and local image features; The global image features and the local image features are fused to obtain multi-scale features, and streetlights are detected based on the multi-scale features to obtain detection results; the detection results include the presence status of streetlights, streetlight type, and spatial relationship between streetlights and roads; If the presence of streetlights is determined based on the detection results, the topological relationship between the streetlights and the road is inferred based on the detection results and the road information to determine the road and target location of at least one streetlight.

2. The method according to claim 1, characterized in that, The step of extracting features from the road image to obtain global image features and local image features includes: The global and local image features of the road image are extracted in parallel using a pre-trained dual-branch feature extraction network model, which includes a first branch and a second branch. The first branch is used to extract a first feature map with a first receptive field from the road image, the first feature map containing local image features; the second branch is used to extract a second feature map with a second receptive field from the road image, the second feature map containing global image features; the first receptive field is smaller than the second receptive field.

3. The method according to claim 1, characterized in that, The presence of the streetlights is characterized by the presence or absence of detection frames in the target image; the spatial relationship includes the density distribution of the streetlights on the road and their orientation information on the road, wherein the number of detection frames represents the density distribution, and the position of the detection frames in the target image represents the orientation information.

4. The method according to claim 3, characterized in that, The step of reasoning about the topological relationship between streetlights and roads based on the detection results and road information to determine the road and target location of at least one streetlight includes: The target image containing the detection box, the spatial relationship, and the road information are subjected to reasoning processing to obtain the target topological relationship between the at least one street lamp and the road. Spatial location calculations are performed based on the target topology to determine the effective road and target location of the at least one street light.

5. The method according to claim 4, characterized in that, The step of reasoning through the target image containing the detection box, the spatial relationship, and the road information to obtain the target topological relationship between the at least one street lamp and the road includes: The target image containing the detection box and the spatial relationship are input into the first inference model to obtain the first-level topological relationship. The first-level topological relationship includes: whether the at least one street lamp is acting on the current road, and the lateral orientation, longitudinal position, and vertical height of the at least one street lamp relative to the current road surface. Based on the first-level topological relationship and the road information, a second-level topological relationship is inferred. The second-level topological relationship includes: the up and down direction information, location information and sequence arrangement information of the at least one street light on the current road; Construct the target topology based on the first-level topology and the second-level topology.

6. The method according to claim 4, characterized in that, The method further includes: The target image containing the detection box is converted into a target mask image, which is used to identify the pixel region of the at least one street lamp in the road image; The road image and the target mask image are input into the second inference model to obtain the depth estimation result of the at least one street lamp; Based on the depth estimation results, some data in the target topology are corrected, and the target topology is updated.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the standardized data interface specifications of the target platform, which are at least partially different for different platforms; Based on the standardized data interface specifications of the target platform, the street light type, the road it serves, and the target location of the at least one street light are formatted and standardized output data is generated. The standardized output data is distributed to the corresponding target platform.

8. A street light identification device, characterized in that, The device includes: The acquisition module is used to acquire road images and road information at the current location, and to extract features from the road images to obtain global image features and local image features; The processing module is used to fuse the global image features and the local image features to obtain multi-scale features, and to perform street light detection based on the multi-scale features to obtain detection results; the detection results include the existence status of the street light, the type of street light, and the spatial relationship between the street light and the road. The reasoning module is used to, based on the detection results and the road information, infer the topological relationship between the streetlights and the road, and determine the road and target location of at least one streetlight when the presence of a streetlight is determined.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

Images