Method and apparatus for generating a road sign model

Abstract

The present disclosure provides a method and device for generating a road sign model, electronic equipment, storage medium and computer program product, relates to the technical field of artificial intelligence, specifically to the technical field of large model, agent, machine vision, electronic map and the like, and can be applied to the generation of a road sign model in an electronic map. The specific implementation scheme is as follows: according to the road network data associated with the deviation point in the road, the scene image of the target area where the deviation point is located is determined; according to the visual field constraint data of the scene image, the sign image of the road sign is determined from the scene image; the road sign in the sign image is identified, and the structural parameters of the sign board and the support structure in the road sign are determined; a three-dimensional model of the sign board and the support structure is constructed according to the structural parameters, and texture data of the sign image is added in the three-dimensional model of the sign board, and a sign model is generated. The present disclosure guarantees the generation quality of the road sign model and improves the generation efficiency of the road sign model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically to the technical fields of large models, intelligent agents, machine vision, and electronic maps, and in particular to a method, apparatus, electronic device, storage medium, and computer program product for generating road sign models, which can be applied to the generation of road sign models in electronic maps. Background Technology

[0002] Road signs are a crucial road infrastructure element in map navigation, in-vehicle navigation, intelligent driving, and the construction of digital twin cities. Road signs not only carry road names, directional guidance, distance information, and traffic management information, but also enhance the realism and spatial representation of map scenes. They are of significant value in improving the navigation product experience, assisting driving decision-making, and building a high-precision digital road environment. Compared to traditional map symbols and text prompts, road signs provide users with visual reference information consistent with the real road environment, helping them quickly confirm driving direction and target exits. This reduces problems such as deviations, missed exits, or temporary lane changes caused by inaccurate intersection information recognition, effectively reducing navigation deviation rates and improving navigation decision-making efficiency and user experience. Therefore, road signs are a vital data element for map navigation products to enhance route guidance capabilities and user experience. However, the sheer number, wide distribution, and frequent updates of road signs mean that traditional production methods relying on manual collection, modeling, and configuration suffer from high costs, long cycles, and insufficient scalability, making it difficult to meet the demands of rapid map data updates and large-scale production. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, storage medium, and computer program product for generating road sign models.

[0004] According to the first aspect, a method for generating a road sign model is provided, comprising: determining a scene image of a target area where a detour point is located based on road network data associated with a detour point in the road; determining a sign image representing a road sign from the scene image based on the view constraint data of the scene image; identifying the road sign in the sign image and determining the structural parameters of the signboard and supporting structure in the road sign; constructing a three-dimensional model of the signboard and supporting structure based on the structural parameters, and adding texture data of the signboard to the three-dimensional model of the signboard based on the sign image to generate a sign model of the road sign.

[0005] According to a second aspect, a road sign model generation apparatus is provided, comprising: a scene determination unit configured to determine a scene image of a target area where a detour point is located based on road network data associated with a detour point in a road; a sign determination unit configured to determine a sign image representing a road sign from the scene image based on the field-of-view constraint data of the scene image; a parameter determination unit configured to identify the road sign in the sign image and determine the structural parameters of the signboard and supporting structure in the road sign; and a model generation unit configured to construct a three-dimensional model of the signboard and supporting structure based on the structural parameters, and add texture data of the signboard to the three-dimensional model of the signboard based on the sign image to generate a sign model of the road sign.

[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method as described in any implementation of the first aspect.

[0007] According to a fourth aspect, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to perform the method described in any implementation of the first aspect.

[0008] According to a fifth aspect, a computer program product is provided, comprising: a computer program that, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0009] According to the technology disclosed herein, a method and apparatus for generating road sign models are provided. The method involves determining a scene image of the target area where a detour point is located based on road network data associated with a detour point in the road; determining a sign image representing a road sign from the scene image based on the view constraint data of the scene image; identifying the road sign in the sign image and determining the structural parameters of the signboard and supporting structure within the road sign; constructing a three-dimensional model of the signboard and supporting structure based on the structural parameters; and adding texture data of the signboard to the three-dimensional model of the signboard based on the sign image to generate the road sign model. This method uses the detour point as the trigger for road sign recognition and automatically completes a series of operations such as road sign discovery, structural analysis, parametric modeling, and automatic texturing based on the map scene data of the detour point. This achieves large-scale automatic production of road sign models, improving both the quality and efficiency of road sign model generation.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is an exemplary system architecture diagram that can be applied to an embodiment of this disclosure; Figure 2 This is a flowchart of an embodiment of the method for generating road sign models according to the present disclosure; Figure 3 This is a schematic diagram showing the separation of the signboard and the supporting structure in the road sign according to this embodiment; Figure 4 This is a schematic diagram of the assembly model according to this embodiment; Figure 5 This is a schematic diagram of the signage model according to this embodiment; Figure 6 This is a schematic diagram illustrating an application scenario of the road sign model generation method according to this embodiment; Figure 7 This is a flowchart of yet another embodiment of the method for generating road sign models according to the present disclosure; Figure 8 This is a structural diagram of an embodiment of the road sign model generation apparatus according to the present disclosure; Figure 9 This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] The technical solutions disclosed herein involve the collection, storage, use, processing, transmission, provision, and disclosure of various types of information, such as user personal information, in accordance with relevant laws and regulations and do not violate public order and good morals.

[0014] Figure 1 An exemplary architecture 100 is shown for a method and apparatus for generating road sign models to which this disclosure can be applied.

[0015] like Figure 1As shown, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The communication connections between terminal devices 101, 102, and 103 form a network topology. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0016] Terminal devices 101, 102, and 103 can be hardware or software that supports network connectivity for data acquisition, interaction, and processing. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices that support network connectivity, information acquisition, interaction, display, and processing functions, including but not limited to smartphones, tablets, e-book readers, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are imposed here.

[0017] Server 105 can be a server that provides various services, such as collecting historical navigation trajectory data and historical route planning data from terminal devices 101, 102, and 103 during the navigation process based on map applications, determining deviation points during the navigation process, using deviation points as the trigger for road sign recognition, and automatically completing a series of operations such as road sign discovery, structure analysis, parametric modeling, and automatic mapping based on map scene data from the deviation points. As an example, server 105 can be a cloud server.

[0018] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (such as software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0019] It should also be noted that the method for generating road sign models provided in the embodiments of this disclosure is generally executed by a server, but the possibility of it being executed by a terminal device, or by a combination of the server and the terminal device, is not excluded. Accordingly, the various parts (e.g., various units) included in the road sign model generation device can be all located in the server, all located in the terminal device, or separately located in the server and the terminal device.

[0020] It should be understood that Figure 1The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. When the electronic equipment on which the road sign model generation method runs does not require data transmission with other electronic equipment, the system architecture may only include the electronic equipment (e.g., servers or terminal devices) on which the road sign model generation method runs.

[0021] Please refer to Figure 2 , Figure 2 A flowchart illustrating a method for generating a road sign model according to an embodiment of this disclosure. Flowchart 200 includes the following steps: Step 201: Determine the scene image of the target area where the deviation point is located based on the road network data associated with the deviation point in the road.

[0022] In this embodiment, the execution entity of the road sign model generation method (e.g., Figure 1 The server in the middle determines the scene image of the target area where the detour point is located based on the road network data associated with the detour point in the road.

[0023] A vehicular point is the location where, during navigation, the vehicle's actual route deviates from the preset route planned by the navigation system. The vehicular point typically corresponds to the geographical location on the road where the driver changes their original direction and leaves the planned path; it serves as the spatial anchor point for navigation deviation events.

[0024] Road network data refers to the fundamental data describing the various road elements in a road system and their interrelationships, including road topology data, road attribute data, and road spatial location data. Road topology data characterizes the connectivity and accessibility between road segments; road attribute data includes road grade, road name, number of lanes, speed limit information, and intersection type; road spatial location data characterizes the coordinates and orientation of road segments and road nodes in geographic space. For example, road network data associated with a detour point may include: the road segment identifier where the detour point is located, the start and end nodes of the road segment, the road grade and number of lanes of the road segment, adjacent road segments connected to the road segment, and their intersection types.

[0025] The target region refers to the road space area where the deviation point is located and scene images need to be acquired. The target region uses the deviation point as a spatial reference and covers the road segment where the deviation point is located and the surrounding related roads. It is the spatial basis for subsequent determination of sign images and road sign recognition and modeling.

[0026] Scene images refer to image data obtained by capturing or acquiring images of a target area, reflecting the visual information of the road environment in that area. Scene images record the appearance and spatial distribution of environmental elements such as roads, roadside facilities, buildings, and vegetation within the target area. As an example, a scene image can be a street view image, which is an image containing multi-angle visual information of the road and roadside environment, captured by a panoramic camera mounted on a data acquisition vehicle along the direction of road travel.

[0027] As an example, the navigation system continuously records the vehicle's driving status and route execution during operation. When the vehicle's actual route deviates from the preset route planned by the navigation system, the system records the location where the deviation occurred; this location is called the detour point. The detour point carries corresponding geographic coordinate information, which characterizes the specific location in the road space where the detour occurred.

[0028] After determining the deviation point, the road segment containing the deviation point is retrieved from the road network data based on its geographic coordinates. The road network data records the spatial range, start and end nodes, and connectivity between each road segment. By matching the geographic coordinates of the deviation point with the spatial range of each road segment in the road network data, the road segment into which the deviation point falls can be determined, and further road attribute information of that road segment can be obtained, including road grade, road name, number of lanes, and the road nodes connecting the two ends of the road segment.

[0029] After determining the road segment where the veergency point is located, the spatial range covered by that road segment is defined as the target area where the veergency point is located. This target area uses the road segment where the veergency point is located as the basic unit and encompasses its entire length. After determining the target area, scene images corresponding to the target area are retrieved from pre-collected scene image data based on the spatial location and extent of the target area.

[0030] As another example, statistical analysis was performed on historical navigation trajectory data of a large number of vehicles in the road network. This historical navigation trajectory data records the actual driving trajectories of vehicles at different times and the corresponding planned route information. By comparing the actual driving trajectories with the planned routes segment by segment, the locations where the trajectories deviate from the planned routes were identified. The frequency of deviation events at the same location was statistically analyzed, and points whose deviation frequencies met preset conditions were identified as deviance points in the road.

[0031] After determining the velocity point, the associated road network data is read. This road network data stores the topological connections, spatial coordinates, and attribute descriptions of each road node and road segment. Based on the geographical coordinates of the velocity point, the road node containing the velocity point and its adjacent road segments extending outwards are located within the road network data, and their spatial coordinate sequences, road orientation, and road attribute information are obtained.

[0032] Based on this, the target area where the deviation point is located is delineated according to the spatial coordinates of the road nodes and the spatial extension range of the adjacent road segments, so that the target area covers the complete spatial information of the deviation point and its surrounding road environment.

[0033] After defining the target area, scene images corresponding to the target area are retrieved from the scene image database based on the spatial coordinate range of the target area. The scene image database stores scene images of various areas acquired along the road, and each scene image is associated with the spatial coordinates of its acquisition location and acquisition direction. By matching the spatial coordinate range of the target area with the acquisition locations of each scene image, the scene images covering the target area are determined.

[0034] In some optional implementations of this embodiment, the execution entity performs step 201 above to determine the scene image of the target area where the deviation point is located in the following manner: First, based on historical navigation trajectory data and historical route planning data, the deviation point in the road is determined; then, based on the road network data associated with the deviation point, the acquisition conditions of the scene image are determined; finally, according to the acquisition conditions, the scene image of the target area is determined from the complete scene image of the road.

[0035] Acquisition conditions refer to the set of spatiotemporal and perspective parameters derived from the road network data associated with the veergence point, used to accurately extract scene images of the target area from the complete scene image. Essentially, it transforms the abstract veergence behavior into specific image acquisition commands. These conditions include dimensions such as geographic coordinate range, shooting azimuth angle, pitch angle, and time window. For example, when historical navigation trajectory data shows frequent veergence in the right-turn lane at an intersection, and road network data indicates that there is a dedicated right-turn signal light and directional arrow markings at that location, the acquisition conditions can be specified as follows: using the center of the intersection as a reference, a rectangular area offset 15 meters east and 8 meters south is defined as the geographic coordinate range; the shooting azimuth angle is set to 210° (facing southwest, directly facing the right-turning traffic flow); the pitch angle is set to -5° (simulating a driver's slightly downward-looking eye level); and the time window is limited to weekday morning rush hour 7:30–9:00 to ensure the capture of effective images under typical veergence scenarios.

[0036] As an example, a trajectory point density clustering analysis is performed on historical navigation trajectory data to identify abnormal clustering areas where the actual driving path deviates from the preset path in the historical route planning data, and the geometric center coordinates of the abnormal clustering area are marked as the deviation point.

[0037] The road network data at the location of the deviation point is queried, and the lane topology, traffic sign distribution, and line-of-sight occlusion relationship at that location are extracted. Combined with the average driving direction and speed vector of the deviation point in the historical navigation trajectory data, the geographical coordinate range, shooting azimuth angle, pitch angle, and time window covering the deviation cause factors are calculated and combined into the conditions for obtaining scene images.

[0038] In the database of complete road scene images that have been collected, candidate images that meet the acquisition conditions of geographic coordinate range, shooting azimuth angle, pitch angle and time window are retrieved. Boundary cropping operation is performed on the candidate images to retain only the content of the target area defined by the acquisition conditions, and the output is the scene image of the target area.

[0039] In this implementation, the conditions are obtained by back-deriving the image from the yaw point and the scene image of the target area is captured in a directional manner, avoiding redundancy in full image processing and ensuring that the materials used for subsequent sign modeling accurately correspond to the actual yaw cause location and viewpoint.

[0040] Step 202: Based on the field-of-view constraint data of the scene image, determine the sign image representing the road sign from the scene image.

[0041] In this embodiment, the execution entity determines the sign image representing the road sign from the scene image based on the field-of-view constraint data of the scene image.

[0042] View constraint data refers to a set of parameters that limit the effective observation range and conditions of a scene image. It is used to constrain which areas in the scene image belong to the effective visible range and which road elements in which directions can be included in the observation.

[0043] For example, the field-of-view constraint data includes: the street view acquisition location, i.e., the spatial coordinates of the acquisition device when acquiring images on the road, which determines the observation origin of the scene image; the shooting direction, i.e., the orientation and pitch angle of the acquisition device at the time of acquisition, which determines the effective imaging azimuth range in the scene image; the direction of the road centerline, i.e., the extension direction of the centerline of the road segment where the yaw point is located, and the angle relationship between it and the shooting direction determines the observable area on both sides and above the road; and the spatial distribution relationship of road facilities, i.e., the usual placement and orientation of facilities such as road signs and traffic lights relative to the road centerline and roadside, which are used to further constrain the possible location area of ​​signs within the field of view.

[0044] As an example, the view constraint data corresponding to the scene image is obtained, including the street view acquisition location, shooting direction, road centerline direction, and spatial distribution of road facilities. The observation origin of the scene image is determined based on the street view acquisition location, and the effective imaging azimuth range of the scene image is determined based on the shooting direction. Image areas outside the effective imaging azimuth range are excluded from subsequent processing.

[0045] Within the effective imaging azimuth range, and considering the direction of the road centerline, the expected distribution area of ​​road signs in the scene image is determined. Road signs are typically placed on either side of the road centerline or above the road, and their orientation has a relatively fixed spatial relationship with the road centerline direction. Based on the angle between the road centerline direction and the shooting direction, the azimuth range in which the road signs should appear in the scene image is calculated.

[0046] Within the aforementioned directional range, the expected location of the sign is further constrained by the spatial distribution of road facilities. The spatial distribution of road facilities characterizes the typical placement distance and height range of the sign relative to the road centerline and roadside edges. Accordingly, the specific area where the sign may exist is delineated within the directional range of the scene image, and the image content within this area is determined as the sign image representing the road sign.

[0047] As another example, the view constraint data associated with the scene image is determined. The view constraint data records the street view acquisition location and shooting direction of the scene image, and also includes the direction of the road centerline of the road segment where the yaw point is located, as well as the spatial distribution of road facilities on that road segment.

[0048] The view constraint data and scene image are input into a pre-trained sign detection model, which then identifies the sign image representing the road sign from the scene image. The sign detection model represents the correspondence between the view constraint data, the scene image, and the sign image within the scene image.

[0049] In some optional implementations of this embodiment, the execution entity performs step 202 in the following manner: The first step is to perform a view visibility analysis on the scene images based on the view constraint data, and then select the valid scene images with view visibility.

[0050] Visibility within the field of view refers to the attribute of a road facility that can be actually observed by the acquisition device within the spatial range covered by the shooting direction and field of view of the acquisition device, starting from the street view acquisition location. Having visibility within the field of view indicates that the road facility is within the effective observation range of the acquisition device, and its image content in the scene image can accurately reflect its actual existence and appearance. Lacking visibility within the field of view indicates that the road facility is located in the blind spot of the acquisition device or outside the shooting direction, and its effective image cannot be obtained in the scene image.

[0051] Based on the street view acquisition location and shooting direction in the field-of-view constraint data, the actual imaging area covered by the acquisition device in the scene image is determined. The portion of the scene image located within the imaging area is retained, while the portion outside the imaging area is removed. The retained scene image is determined as a valid scene image with field-of-view visibility.

[0052] The second step is to use a large visual understanding model to determine whether road signs are included in the valid scene images.

[0053] A large-scale visual understanding model refers to a multimodal foundational model that has been jointly pre-trained with large-scale image and text data, possessing the ability to perform semantic understanding, target recognition, and spatial localization of image content. In this scheme, the large-scale visual understanding model is used to perform content understanding on valid scene images, determine whether road signs exist in the image, and output the location information of the road signs in the image when they exist.

[0054] The third step is to determine the sign image from the valid scene image in response to the inclusion of a road sign in the valid scene image.

[0055] The valid scene image is input into the visual understanding model. The visual understanding model performs semantic parsing on the valid scene image and outputs the category and location identifiers of each target in the valid scene image. Based on the category identifiers, it is determined whether there is a target of the category "road sign" in the valid scene image. If there is no target of the category "road sign" in the output of the visual understanding model, it is determined that the valid scene image does not include road signs, and no further processing is performed on the valid scene image. If there is a target of the category "road sign" in the output of the visual understanding model, it is determined that the valid scene image includes road signs. Based on the location identifier corresponding to the target output by the visual understanding model, the image region where the road sign is located in the valid scene image is located, and this image region is extracted from the valid scene image and identified as the sign image.

[0056] In this implementation, invalid images in the blind zone are pre-selected through visual visibility analysis, and semantic-level discrimination is performed with the help of a large visual understanding model. This avoids performing subsequent recognition and modeling operations on unsigned images and reduces unnecessary computational overhead.

[0057] In some optional implementations of this embodiment, the execution entity performs the third step described above to determine the sign image from the valid scene image in the following manner: First, in response to the valid scene image including a road sign, the integrity of the road sign in the valid scene image is determined by a large visual understanding model; then, in response to determining that the road sign in the valid scene image has integrity, the sign image is determined from the valid scene image.

[0058] The valid scene image, including the road sign, is input into the visual understanding model. The model performs a semantic-level integrity assessment of the road sign, determining whether the signboard is occluded, whether the text and symbols are clearly legible, and whether the supporting structure is intact and visible. If the model determines the road sign is complete, it locates and crops the area containing the sign within the valid scene image based on the sign's location identifier output by the model, thus identifying it as the sign image. If the model determines the sign is incomplete, it discards the valid scene image and does not perform the sign image identification process.

[0059] In this implementation, the integrity of road signs is verified by using a large visual understanding model. Only signs with clear surfaces and complete structures are selected as input for subsequent modeling. This effectively avoids structural parameter recognition errors caused by occlusion or incompleteness, and improves the accuracy and usability of sign model generation.

[0060] Step 203: Identify the road signs in the sign image and determine the structural parameters of the signboard and supporting structure in the road signs.

[0061] In this embodiment, the aforementioned execution entity identifies road signs in the sign image and determines the structural parameters of the signboard and supporting structure in the road sign.

[0062] A road sign is a plate-shaped component that carries and displays traffic information. Its surface is printed with text, symbols, arrows, or other traffic instructions, and it is the main functional component of road signs that conveys information to road users. Road signs are usually rectangular, circular, triangular, or other regular geometric shapes, and are fixed to the side or above the road by a supporting structure.

[0063] The supporting structure refers to the collective term for the rods, brackets, and connecting components that fix and support the signboard at a predetermined position in the road space. One end of the supporting structure is connected to the signboard, and the other end is anchored to the ground or attached to other road structures to ensure the stable display of the signboard at a predetermined height and orientation.

[0064] Structural parameters refer to a set of quantitative parameters used to describe the geometric shape, spatial orientation, and assembly relationship between the signboard and the supporting structure.

[0065] For example, the structural parameters of the signboard may include the shape type, width, height, thickness, and orientation of the signboard; the structural parameters of the support structure may include the height of the support pole, the diameter of the support pole, the length of the support arm, the extension direction of the support arm, and the installation height of the bottom edge of the signboard from the ground.

[0066] As an example, a sign image is input into a pre-trained road sign recognition model. This model is trained on a large number of sample images labeled with signboard and support structure regions, enabling it to locate and segment road signs within the image. The model outputs a detection box for the road sign in the image and separately segments and labels the signboard and support structure regions within the detection box.

[0067] Based on the segmented signboard area, the outline boundary of the signboard is determined, and structural parameters such as the signboard shape, width, height, and orientation are obtained. Based on the segmented support structure area, the extension direction and length range of the support poles and support arms are determined, and structural parameters such as the height of the support poles, the length of the support arms, and the installation height of the bottom edge of the signboard from the ground are obtained.

[0068] As another example, edge extraction is performed on the sign image to obtain a set of contour edges in the sign image. In the set of contour edges, based on the geometric closure features of the sign plate edges, the contour line segments that constitute the closed plate area are identified. This closed plate area is determined as the imaging area of ​​the sign plate in the sign image, and structural parameters such as the shape type, plate width, plate height, and plate orientation of the sign plate are obtained accordingly.

[0069] In the contour edge set, based on the linear extension characteristics of the support structure edge, a linear contour line segment that is connected to the signboard imaging area and extends towards the ground is identified. This linear contour line segment is determined as the imaging area of ​​the support structure in the signboard image, and structural parameters such as the height of the support pole, the length of the support crossarm, and the installation height of the bottom edge of the signboard from the ground are obtained accordingly.

[0070] In some optional implementations of this embodiment, the execution entity can perform step 203 to determine the structural parameters of the signboard and supporting structure in the road sign as follows: The first step is to identify road signs from sign images using a large image segmentation model.

[0071] First, the sign image is input into a large-scale image segmentation model. Based on the prompts generated by the automatic prompt generation mechanism, the main body region of the road sign is obtained. Then, using the pixel-level segmentation capability of the large-scale image segmentation model, the overall outline of the road sign is accurately extracted to obtain the binary mask and boundary contour information of the main body of the road sign.

[0072] The second step is to identify the overall structural parameters of the road signs.

[0073] As an example, a morphological closing operation is performed on the binarized mask to fill the internal holes, obtaining a complete and effective mask for the road sign. Based on the effective mask, a set of boundary contour points is extracted, and polygon fitting is performed on this set to obtain a sequence of fitted polygon vertices that matches the actual shape of the road sign. Based on this sequence of fitted polygon vertices, the overall structural parameters of the road sign, such as width and height, are determined.

[0074] The third step is to identify the structural parameters of the signboard using a target object detection model.

[0075] As an example, firstly, target bounding box detection is performed on rectangular signboards, circular signboards, arrow areas, and traffic icon areas in road signs using predefined prompt words to obtain the location range and bounding box information of each component structure.

[0076] Based on the target bounding box detection results, different signboard areas are further classified and structurally represented to determine the geometric shape, size ratio, and spatial distribution relationship of each signboard.

[0077] For complex road signs composed of multiple signboards, the topological structure of the signboards is established by utilizing the positional relationship between the target boxes, describing the vertical, horizontal, and containment relationships between the signboards, thereby forming a complete layout structure.

[0078] The fourth step is to identify the areas of the road signs other than the signboards and determine the structural parameters of the supporting structure.

[0079] Continue to refer to Figure 3 This diagram shows the separation of the signboard and supporting structure in a road sign.

[0080] For example, the overall outline of a road sign is represented as The outline of the signboard is shown as The region of the supporting structure is obtained through the difference operation:

[0081] Subsequently, the supporting structure area was binarized, and a skeleton extraction algorithm was used to calculate and identify the structural parameters of the signboard, such as the centerline of the supporting structure.

[0082] Further topology analysis was performed on the skeleton network to extract skeleton nodes, connections, and centerline structure, and a set of parameterized modeling parameters was constructed:

[0083] in: Signboard The structural parameters are defined as follows:

[0084] in, The coordinates of the center of the content board; Width of the content board; For the height of the content sign; For the thickness of the content board.

[0085] Vertical support rod The structural parameters are defined as follows:

[0086] in, The coordinates of the starting point of the centerline; The coordinates of the endpoint of the centerline; Let be the radius of the cylinder.

[0087] lateral support beam The structural parameters are defined as follows:

[0088] in, The coordinates of the starting point of the centerline; The coordinates of the endpoint of the centerline; Let be the radius of the cylinder.

[0089] The horizontal support beams and vertical support rods are automatically derived from the center line of the skeleton, eliminating the need for separate testing.

[0090] In this implementation, the overall and surface parameters are obtained by using a large image segmentation model and a target object detection model, respectively. Then, the support structure parameters are derived through geometric operations. This avoids the identification bias of a single model for complex structures and improves the accuracy and consistency of the structural parameters of each component.

[0091] In some optional implementations of this embodiment, the execution entity can perform the second step described above to identify the structural parameters of the signboard in the following manner: First, the signboard is identified by a target object detection model to obtain the signboard detection result; then, the contour extraction algorithm is used to optimize the contour of the signboard in the signboard detection result to obtain the optimized detection result; finally, the structural parameters of the signboard are determined based on the optimized detection result.

[0092] Cropping a partial region image from the sign image based on the initial bounding box, performing adaptive threshold segmentation on the partial region image to generate a binarized edge map, extracting the edge point set in the binarized edge map using the Canny operator, performing polygon approximation fitting on the edge point set to obtain an optimized contour polygon that fits the actual shape of the sign plate, and taking the optimized contour polygon as the optimized detection result. According to the optimized detection result, the structural parameters of the sign plate are determined, for example, the minimum area circumscribed rectangle of the optimized contour polygon is calculated, the pixel length of the long side of the minimum area circumscribed rectangle is determined as the pixel width of the sign plate, and the pixel length of the short side is determined as the pixel height of the sign plate, both of which together constitute the structural parameters of the sign plate.

[0093] In this implementation, a contour extraction algorithm is introduced for secondary optimization on the basis of the output result of the target object detection model, which corrects the deviation between the detection frame and the actual edge of the sign plate, makes the structural parameters more consistent with the real physical contour, and improves the accuracy and robustness of the size recognition of the sign plate.

[0094] In some optional implementations of this embodiment, the above execution subject further performs the following operation: performing high-definition redrawing on the missing part of the sign image through an image-to-image large model, and optimizing the visual attributes of the redrawn sign image according to the preset road sign format specification constraints.

[0095] The image-to-image large model refers to a deep learning model that is based on a diffusion probability model or a generative adversarial network architecture, uses an input image as conditional guidance, and generates a new image consistent with the input image in semantics and coherent in structure through a denoising process, and has the capability of context-aware completion and high-definition detail reconstruction for the missing area of the image.

[0096] The road sign format specification constraints refer to a set of sign design rules defined in accordance with current national or industrial standards (such as GB 5768), including but not limited to structural parameters such as layout proportion, character font and font size, color value range, graphic symbol style and spacing, which are used to ensure that the generated sign image meets the legal visual identification requirements.

[0097] The visual attributes refer to image characteristic parameters in the sign image that can be quantitatively or qualitatively evaluated and affect human eye recognition and information transmission effects. They cover dimensions such as color accuracy, contrast, edge sharpness, text stroke completeness, and graphic symbol standardization. For example, the color accuracy is reflected in whether the hue deviation of the red border of a prohibition sign is within the standard tolerance range; the contrast is reflected in whether the black text on a white background still meets the minimum brightness ratio threshold after low-illumination redrawing; the edge sharpness is reflected in whether the contour of the redrawn arrow symbol has blurring or sawteeth; the text stroke completeness is reflected in whether the horizontal-fold-hook of the Chinese character "Stop" is deformed or broken due to missing completion.

[0098] As an example, the original sign image containing the missing area is used as a conditional input to the pre-trained graph-to-image model. At the same time, a mask annotation of the missing area is provided. Based on the missing range defined by the mask and the pixel context information of the surrounding complete area, the model performs a multi-step denoising and generation process, and outputs a high-definition redraw result with the missing content filled in and the resolution no lower than that of the original image.

[0099] The system loads the standard color palette, font template, and graphic symbol library defined in the preset road sign layout specifications, and performs area-by-area verification on the high-definition redrawing results. If the color of the redrawing area deviates from the tolerance range of the standard color palette, it performs forced correction based on the nearest neighbor color value. If the font or strokes of the redrawing text do not conform to the font template, it replaces them with standard fonts. If the proportion or line width of the graphic symbol exceeds the specification threshold, it re-renders the symbol area according to the specification parameters, and finally outputs the sign image optimized by the road sign layout specifications.

[0100] In this implementation, the missing parts are filled in by using a large image-based model and the visual attributes are corrected according to the specifications. This ensures that the redrawn sign image not only restores high-definition details but also strictly conforms to the legal format, guaranteeing the compliance and recognizability of subsequent 3D texture mapping.

[0101] Step 204: Construct a three-dimensional model of the signboard and supporting structure based on the structural parameters, and add the texture data of the signboard to the three-dimensional model of the signboard based on the sign image to generate the sign model of the road sign.

[0102] In this embodiment, the execution entity constructs a three-dimensional model of the signboard and supporting structure based on the structural parameters, and adds the texture data of the signboard to the three-dimensional model of the signboard based on the sign image to generate the sign model of the road sign.

[0103] A 3D model is a physical model that digitally represents the geometry, dimensions, and spatial assembly relationship of a signboard and its supporting structure in a three-dimensional coordinate system. The 3D model of the signboard represents its shape, dimensions, and orientation; the 3D model of the supporting structure represents the shape, dimensions, and connection positions of components such as support poles and crossarms to the signboard. When the 3D models of the signboard and the supporting structure are combined according to the assembly relationships indicated by the structural parameters, they constitute a complete 3D representation of the road sign.

[0104] Texture data refers to the visual information carried on the surface of a signboard, including the background color, text content, symbols, patterns, arrows, and border styles. The texture data originates from the image of the signboard area, is processed, and then applied to the surface of the signboard's 3D model, allowing the 3D model to visually reproduce the actual appearance of the road sign.

[0105] As an example, based on the shape, width, height, and thickness of the signboard in the structural parameters, a 3D model of the signboard is constructed, ensuring that the outline, dimensions, and thickness of the 3D model match the structural parameters. Similarly, based on the height and diameter of the support poles, the length of the support crossarms, and the installation height of the signboard's bottom edge from the ground, a 3D model of the support structure is constructed, ensuring that the dimensions and spatial orientation of each component in the 3D model of the support structure match the structural parameters. Finally, the 3D model of the signboard is combined with the 3D model of the support structure according to the assembly positions indicated by the structural parameters to form the overall 3D framework of the road sign.

[0106] The image region corresponding to the signboard is extracted from the sign image. Perspective correction is performed on this image region to eliminate the distortion of the signboard surface caused by the shooting angle, resulting in a front view image of the signboard. The front view image is used as texture data and mapped and pasted onto the surface of the 3D model of the signboard, so that the surface of the 3D model of the signboard presents the same background color, text, and symbol content as the sign image, generating the sign model of the road sign.

[0107] As an example, structural parameters and sign images are input into a pre-trained 3D sign generation model. The 3D sign generation model is trained on a large amount of sample data, which includes the structural parameters of the road sign, the sign image, and the corresponding ground truth value of the sign model. Based on the input structural parameters, the 3D sign generation model generates a 3D model of the sign panel and a 3D model of the supporting structure, and combines them according to the assembly relationship indicated by the structural parameters. Simultaneously, the 3D sign generation model extracts texture data from the input sign image of the sign panel, fills the surface of the 3D model of the sign panel with the texture data, and directly outputs the sign model of the road sign.

[0108] In some optional implementations of this embodiment, the execution entity performs step 204 above to generate the road sign model in the following manner: The first step is to construct a three-dimensional model of the signboard based on its structural parameters.

[0109] Establish the mapping relationship between parameters and three-dimensional geometry:

[0110] For the signboard structure, parametric cuboid primitives are used for automatic modeling. The pixel width, pixel height and preset thickness of the signboard are used as input parameters to generate the corresponding cuboid mesh model, and the cuboid mesh model is determined as the three-dimensional model of the signboard.

[0111] The second step is to construct a three-dimensional model of the supporting structure based on its structural parameters.

[0112] For the support structure, parametric cylindrical primitives are used for automatic modeling. The start and end points of the support rod's centerline are defined as follows:

[0113] The length of the cylinder is:

[0114] The spatial attitude of the cylinder is automatically calculated based on the centerline direction, and a corresponding cylindrical mesh structure is generated based on the cylinder radius parameter. This cylindrical mesh structure is then used as the three-dimensional model of the supporting structure.

[0115] The third step is to assemble the three-dimensional models of the signboard and the supporting structure based on the topological relationship between them, thus obtaining the assembled model.

[0116] Continue to refer to Figure 4 A schematic diagram of the assembled model is shown.

[0117] Based on the connection point coordinates and assembly angles defined in the topological relationship, the top center point of the 3D model of the supporting structure is aligned to the back mounting position of the 3D model of the signboard. Rigid body transformation is performed to complete spatial assembly. The vertex, normal, and index data of the two are merged to generate the signboard model.

[0118] in, This represents an array of vertex coordinates, storing the spatial position of each vertex in the 3D model. This represents an array of normal vectors, storing the orientation of each vertex or face, used for lighting calculations and rendering; This represents an index array. Triangle faces are defined by referencing the vertex indices, avoiding duplicate vertex storage and improving rendering efficiency.

[0119] The fourth step is to map the texture data of the signboard to the 3D model of the signboard in the assembly model to obtain the signboard model.

[0120] Continue to refer to Figure 5 The diagram shows a schematic of the sign model.

[0121] Obtain the texture region corresponding to each signboard from the scene image:

[0122] in, Represents the texture image of the signboard; This indicates the coordinates of the signboard boundary.

[0123] At the same time, the structural parameters of the corresponding signboard are obtained:

[0124] Further establish a one-to-one mapping relationship between the signboard texture and the signboard's structural parameters:

[0125] The detected signboard texture area is automatically mapped to the surface of the corresponding content sign model.

[0126] Subsequently, UV coordinates are established using a planar mapping method:

[0127] After the texturing is completed, the sign model will be automatically generated:

[0128] in, This represents a texture coordinate array that stores the two-dimensional texture mapping coordinates corresponding to each vertex and defines how the texture image is attached to the model surface. This represents texture data.

[0129] The following signage model file will be output further:

[0130] in, File storage geometry; File storage material information; The files store high-resolution textures. The final product is a standard 3D asset that can be directly used in map rendering engines.

[0131] In this implementation, a three-dimensional model is automatically constructed and assembled based on structural parameters and topological relationships, and then texture data is mapped to achieve accurate reconstruction and appearance restoration of the sign model.

[0132] In some optional implementations of this embodiment, the execution entity performs the fourth step described above in the following manner, mapping the texture data of the signboard to the three-dimensional model of the signboard in the assembly model: First, the texture data is scaled according to the size of the three-dimensional model of the signboard to obtain a scaled texture that matches the size of the three-dimensional model of the signboard; then, the scaled texture is mapped to the three-dimensional model of the signboard in the assembly model.

[0133] The actual width and height of the signboard's 3D model in 3D space are obtained, along with the original pixel width and height of the texture data. The ratio of the actual width to the original pixel width is calculated as the horizontal scaling factor, and the ratio of the actual height to the original pixel height is calculated as the vertical scaling factor. Based on the horizontal and vertical scaling factors, the texture data is resampled using bilinear interpolation to generate a scaled texture that matches the size of the signboard's 3D model.

[0134] Read the UV coordinate data of the 3D model of the signboard in the assembly model, bind the scaled texture to the surface mesh of the 3D model of the signboard in the assembly model according to the UV coordinate data, complete the texture mapping operation, and output the texture-adapted signboard model.

[0135] In this implementation, texture data is adaptively scaled and remapped according to the actual size of the 3D model, avoiding texture stretching or compression distortion, and ensuring the geometric proportion accuracy and visual fidelity of the sign appearance restoration.

[0136] See also Figure 6 , Figure 6 This is a schematic diagram of an application scenario 600 of the road sign model generation method according to this embodiment. Terminal devices 601 and 602 collect historical navigation trajectory data and historical route planning data during the navigation process based on map applications to determine deviation points in the navigation process. Server 603 determines the scene image of the target area where the deviation point is located based on road network data associated with the deviation point; based on the view constraint data of the scene image, it determines the sign image representing the road sign from the scene image; it identifies the road sign in the sign image and determines the structural parameters of the signboard and supporting structure in the road sign; based on the structural parameters, it constructs a three-dimensional model 6031 of the signboard and supporting structure, and adds texture data of the signboard to the three-dimensional model of the signboard based on the sign image to generate the road sign model 6032.

[0137] In some optional implementations of this embodiment, the above-mentioned execution entity may also perform the following operations: based on the absolute spatial information of the road sign, perform coordinate transformation and spatial registration on the sign model to transform the sign model from the relative modeling coordinate system to the geographic coordinate system of the map application.

[0138] Absolute spatial information refers to the location and attitude parameters of road signs in the real physical world that can be uniquely determined by a global or regional geodetic system. It typically includes latitude and longitude coordinates, altitude, and azimuth, pitch, and roll angles relative to geographic north. This information is independent of any local modeling process and can be directly collected by high-precision positioning equipment or provided by an authoritative geographic database.

[0139] The relative modeling coordinate system refers to the local rectangular coordinate system used in the process of building the sign model. Its origin is usually set as the geometric center of the sign board or a reference point at the bottom of the supporting structure. The coordinate axis directions are customized according to the convenience of modeling (such as the X-axis along the width of the board, the Y-axis along the height, and the Z-axis perpendicular to the board and outward). This coordinate system only describes the relative geometric relationship between the components inside the model and does not have the semantic meaning of real geographical location.

[0140] A geographic coordinate system is a standard spatial reference frame used in map applications to uniformly express the absolute position of surface features. Common forms include latitude, longitude, and elevation systems under the WGS84 ellipsoid or plane rectangular coordinate systems obtained by projecting from this ellipsoid. All map rendering, route planning, and spatial query operations are performed under this coordinate system.

[0141] As an example, firstly, the absolute spatial information of the road sign is obtained, including the latitude and longitude, altitude, and azimuth, pitch, and roll angles of the sign installation point. Secondly, the origin position and local coordinates of each vertex of the sign model in the relative modeling coordinate system are analyzed. Then, a three-dimensional rotation matrix is ​​constructed based on the azimuth, pitch, and roll angles in the absolute spatial information, and a rotation transformation is applied to the coordinates of all vertices in the relative modeling coordinate system to make the model posture consistent with the actual installation posture. Next, the three-dimensional rectangular coordinates of the installation point in the geographic coordinate system of the map application are calculated based on the latitude, longitude, and altitude, and are superimposed on the vertices of the rotated model as translation vectors. Finally, it is verified whether the positions of the key points of the transformed model (such as the four corners of the sign) in the geographic coordinate system match the absolute spatial information. If the deviation exceeds a preset threshold, the rotation and translation parameters are iteratively corrected to complete the spatial registration and output the sign model located in the geographic coordinate system of the map application.

[0142] In this implementation, coordinate transformation and spatial registration are driven by absolute spatial information, which enables the sign model to be accurately anchored from the local modeling space to the map geographic coordinate system, ensuring the authenticity and consistency of its position and attitude in the navigation scenario.

[0143] In some optional implementations of this embodiment, the execution entity transforms the sign model from the relative modeling coordinate system to the geographic coordinate system of the map application in the following manner: First, based on the distance between the spatial location points of the road sign in the geographic coordinate system and the distance between the corresponding spatial location points of the sign model in the relative modeling coordinate system, the scaling factor of the sign model is determined; then, the sign model is scaled according to the scaling factor to obtain the scaled model; then, according to the spatial orientation constraints of the road sign in the geographic coordinate system, the rotation angle and spatial position of the scaled model are adjusted to obtain the adjusted model; finally, the adjusted model is transformed from the relative modeling coordinate system to the geographic coordinate system.

[0144] Spatial orientation constraints refer to the set of attitude and positional limitations that road signs must meet in a geographic coordinate system, determined by actual installation specifications or on-site measurement data. These constraints ensure that the orientation, tilt, and installation height of the model in the real environment conform to physical facts. These constraints typically include the desired direction of the sign's surface normal vector in the geographic coordinate system (e.g., facing the direction of travel), allowable attitude deviation tolerances (e.g., pitch angle ±2°, roll angle ±1°), and the precise three-dimensional coordinates of key anchor points (e.g., the bottom of the support rod or the center of the sign). For example, the spatial orientation constraints of a speed limit sign can be specified as follows: the azimuth angle of the surface normal vector is 180°±3° (i.e., due south, tolerance 3°), the pitch angle is 0°±1° (i.e., perpendicular to the ground), the roll angle is 0°±1°, and the Z-axis elevation of the sign's center point in the geographic coordinate system is 5.2 meters ± 0.1 meters.

[0145] As an example, select at least two spatial locations of the road sign with a clear physical correspondence in the geographic coordinate system (such as the locations of two longitudinal support rods), and calculate their Euclidean distance as the true reference length; simultaneously select two vertices of the sign model corresponding to it in the relative modeling coordinate system, and calculate their Euclidean distance as the model reference length; divide the true reference length by the model reference length to obtain the scaling factor of the sign model.

[0146] The scaling factor is applied evenly to all vertex coordinates of the sign model in the relative modeling coordinate system to ensure the overall size of the model matches the scale of the real sign, and the scaled model is output. The desired direction, pitch angle, roll angle, and key anchor point coordinates of the board surface normal vector defined in the spatial orientation constraints are read; the rotation difference between the current and desired attitudes of the scaled model is calculated, and inverse rotation correction is applied; the key anchor points of the scaled model are translated to the 3D coordinate positions specified by the spatial orientation constraints, completing the joint adjustment of attitude and position, and the adjusted model is output.

[0147] Using the key anchor points of the adjusted model as the origin, and combining the absolute orientation parameters in the spatial orientation constraints, a rigid body transformation matrix is ​​constructed from the relative modeling coordinate system to the geographic coordinate system. Coordinate transformation is then performed on all vertices of the model to obtain a sign model located in the geographic coordinate system with dimensions, orientation, and position consistent with the real road sign. Simultaneously, the model's geometry, texture mapping, and topological relationship data are updated.

[0148] In this implementation, the scaling factor is determined by the distance ratio and the attitude and position are finely adjusted in combination with spatial orientation constraints. This ensures that the model has completed size normalization and attitude alignment before being transferred to the geographic coordinate system, thus guaranteeing the geometric realism and spatial consistency of the final model in the map.

[0149] In some optional implementations of this embodiment, the execution entity may also perform the following operations: apply a structural constraint mechanism to the generated sign model, automatically verify the model based on the preset range of horizontal support beam length, range of longitudinal support rod length, range of content sign area and overall structural proportion, and perform parameter correction or result rollback when an anomaly is detected.

[0150] As an example, after the sign model is generated, the geometric parameters of key components such as the transverse support beams, longitudinal support rods, and content signs are automatically extracted from the model. The actual lengths of the transverse support beams, longitudinal support rods, and content signs, as well as the ratio of the overall height to width of the model, are calculated. The calculated values ​​are then compared with preset ranges for the lengths of the transverse support beams, longitudinal support rods, content signs, and the overall structural proportions. If any parameter exceeds the corresponding allowable range, it is considered a structural anomaly, and a correction process is initiated: For linearly adjustable single-dimensional exceedances (such as the transverse support beam length slightly exceeding the upper limit), the parameter is reassigned according to the boundary value, and the dimensions of related components are updated synchronously; for cases of multi-parameter coupling imbalance or deviations exceeding the correction threshold, the current generation result is abandoned, and the model is reverted to the previous valid model state or regeneration is triggered; if all parameters fall within the preset range, the model is confirmed to have passed the structural constraint verification, and a qualified sign model is output.

[0151] In this implementation, the geometric parameters of the model are automatically verified and corrected through a structural constraint mechanism, which effectively prevents dimensional distortion and structural anomalies caused by detection errors, and ensures the physical rationality of the sign model and the visual consistency of the map rendering.

[0152] Continue to refer to Figure 7 The illustration shows a schematic flow 700 of yet another embodiment of the method for generating road sign models according to the present disclosure. Flow 700 includes the following steps: Step 701: Determine the deviation points in the road based on historical navigation trajectory data and historical route planning data.

[0153] Step 702: Determine the conditions for acquiring scene images of the target area where the yaw point is located based on the road network data associated with the yaw point.

[0154] Step 703: Determine the scene image from the complete scene image of the road according to the acquisition conditions.

[0155] Step 704: Based on the field of view constraint data, perform field of view visibility analysis on the scene images and filter out the valid scene images with field of view visibility.

[0156] Step 705: Determine whether road signs are included in the valid scene images using a large visual understanding model.

[0157] Step 706: In response to the inclusion of road signs in the valid scene image, determine the completeness of the road signs in the valid scene image using a visual understanding large model.

[0158] Step 707: In response to determining that the road sign in the valid scene image has integrity, determine the sign image from the valid scene image.

[0159] Step 708: Determine the road signs from the sign images using a large image segmentation model.

[0160] Step 709: Identify the overall structural parameters of the road sign.

[0161] Step 710: Identify the structural parameters of the signboard using the target object detection model.

[0162] Step 711: Identify the areas of the road signs other than the signboard and determine the structural parameters of the supporting structure.

[0163] Step 712: Construct a three-dimensional model of the signboard based on its structural parameters.

[0164] Step 713: Construct a three-dimensional model of the support structure based on the structural parameters of the support structure.

[0165] Step 714: Based on the topological relationship between the signboard and the supporting structure, assemble the three-dimensional models of the signboard and the supporting structure to obtain the assembled model.

[0166] Step 715: Map the texture data of the signboard to the 3D model of the signboard in the assembly model to obtain the signboard model.

[0167] Step 716: Based on the absolute spatial information of the road sign, perform coordinate transformation and spatial registration on the sign model to transform the sign model from the relative modeling coordinate system to the geographic coordinate system used in map applications.

[0168] The process 700 of the road sign model generation method in this embodiment, compared with the above process 200, specifically describes the generation and conversion process of the sign model, realizes the accurate mapping, automatic splicing and map scene fusion of the road sign model, and forms standardized three-dimensional road sign data that can be directly applied to the navigation map production system.

[0169] Continue to refer to Figure 8 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a road sign model generation device, which is similar to... Figure 2 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0170] like Figure 8As shown, the road sign model generation device 800 includes: a scene determination unit 801, configured to determine a scene image of the target area where the detour point is located based on road network data associated with the detour point in the road; a sign determination unit 802, configured to determine a sign image representing the road sign from the scene image based on the field-of-view constraint data of the scene image; a parameter determination unit 803, configured to identify the road sign in the sign image and determine the structural parameters of the signboard and supporting structure in the road sign; and a model generation unit 804, configured to construct a three-dimensional model of the signboard and supporting structure based on the structural parameters, and add texture data of the signboard to the three-dimensional model of the signboard based on the sign image to generate a sign model of the road sign.

[0171] In some optional implementations of this embodiment, the sign determination unit 802 is further configured to: perform visual visibility analysis on the scene image based on visual constraint data, and filter out valid scene images with visual visibility; determine whether the valid scene image includes a road sign through a large visual understanding model; and determine the sign image from the valid scene image in response to the valid scene image including a road sign.

[0172] In some optional implementations of this embodiment, the sign determination unit 802 is further configured to: in response to the inclusion of road signs in the valid scene image, determine the integrity of the road signs in the valid scene image through a visual understanding large model; and in response to determining that the road signs in the valid scene image have integrity, determine the sign image from the valid scene image.

[0173] In some optional implementations of this embodiment, the parameter determination unit 803 is further configured to: determine the road sign from the sign image using an image segmentation large model; identify the overall structural parameters of the road sign; identify the structural parameters of the sign plate using a target object detection model; and identify the area of ​​the road sign other than the sign plate to determine the structural parameters of the supporting structure.

[0174] In some optional implementations of this embodiment, the parameter determination unit 803 is further configured to: identify the signboard through the target object detection model to obtain the signboard detection result; optimize the contour of the signboard in the signboard detection result using a contour extraction algorithm to obtain the optimized detection result; and determine the structural parameters of the signboard based on the optimized detection result.

[0175] In some optional implementations of this embodiment, the model generation unit 804 is further configured to: construct a three-dimensional model of the signboard based on the structural parameters of the signboard; construct a three-dimensional model of the support structure based on the structural parameters of the support structure; assemble the three-dimensional models of the signboard and the support structure based on the topological relationship between the signboard and the support structure to obtain an assembled model; and map the texture data of the signboard to the three-dimensional model of the signboard in the assembled model to obtain a sign model.

[0176] In some optional implementations of this embodiment, the model generation unit 804 is further configured to: scale the texture data according to the size of the three-dimensional model of the signboard to obtain a scaled texture that is adapted to the size of the three-dimensional model of the signboard; and map the scaled texture to the three-dimensional model of the signboard in the assembly model.

[0177] In some optional implementations of this embodiment, the above-mentioned device further includes an image optimization unit (not shown in the figure), which is configured to: redraw the missing parts of the sign image in high definition using a large-scale image-to-image model, and optimize the visual attributes of the redrawn sign image according to preset road sign layout specifications.

[0178] In some optional implementations of this embodiment, the scene determination unit 801 is further configured to: determine the detour point in the road based on historical navigation trajectory data and historical route planning data; determine the acquisition conditions of the scene image based on the road network data associated with the detour point; and determine the scene image of the target area from the complete scene image of the road according to the acquisition conditions.

[0179] In some optional implementations of this embodiment, the above-mentioned device further includes a model conversion unit (not shown in the figure), which is configured to: perform coordinate transformation and spatial registration on the sign model according to the absolute spatial information of the road sign, so as to transform the sign model from the relative modeling coordinate system to the geographic coordinate system of the map application.

[0180] In some optional implementations of this embodiment, the model conversion unit is further configured to: determine the scaling factor of the sign model based on the distance between the spatial location points of the road sign in the geographic coordinate system and the distance between the corresponding spatial location points of the sign model in the relative modeling coordinate system; scale the sign model according to the scaling factor to obtain the scaled model; adjust the rotation angle and spatial position of the scaled model according to the spatial orientation constraints of the road sign in the geographic coordinate system to obtain the adjusted model; and convert the adjusted model from the relative modeling coordinate system to the geographic coordinate system.

[0181] In this embodiment, a road sign model generation device is provided. A scene determination unit determines the scene image of the target area where the detour point is located based on road network data associated with detour points in the road. A sign determination unit determines the sign image representing the road sign from the scene image based on the view constraint data of the scene image. A parameter determination unit identifies the road sign in the sign image and determines the structural parameters of the signboard and supporting structure within the road sign. A model generation unit constructs a three-dimensional model of the signboard and supporting structure based on the structural parameters, and adds texture data of the signboard to the three-dimensional model of the signboard based on the sign image, generating the road sign model. Thus, using the detour point as the trigger for road sign recognition, and based on the map scene data of the detour point, a series of operations such as road sign discovery, structural analysis, parametric modeling, and automatic texturing are automatically completed, achieving large-scale automatic production of road sign models. This ensures the quality of road sign model generation while improving the generation efficiency.

[0182] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the method for generating a road sign model as described in any of the above embodiments.

[0183] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the method for generating a road sign model as described in any of the above embodiments when executed.

[0184] This disclosure provides a computer program product that, when executed by a processor, can implement the method for generating road sign models as described in any of the above embodiments.

[0185] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0186] like Figure 9As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0187] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0188] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the method for generating a road sign model. For example, in some embodiments, the method for generating a road sign model may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the method for generating a road sign model described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform a method for generating a road sign model by any other suitable means (e.g., by means of firmware).

[0189] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0190] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other device for generating programmable road sign models, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0191] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0192] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0193] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0194] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are hosting products within the cloud computing service system to address the management difficulties and weak business scalability inherent in traditional physical hosts and Virtual Private Servers (VPS) services; they can also be servers for distributed systems or servers incorporating blockchain technology.

[0195] According to the technical solution of this disclosure, a road sign model generation device is provided. Based on road network data associated with detour points in a road, a scene image of the target area where the detour point is located is determined. Based on the view constraint data of the scene image, a sign image representing the road sign is determined from the scene image. The road sign in the sign image is identified, and the structural parameters of the signboard and supporting structure in the road sign are determined. A three-dimensional model of the signboard and supporting structure is constructed based on the structural parameters, and texture data of the signboard is added to the three-dimensional model of the signboard based on the sign image to generate the road sign model. Thus, using the detour point as the trigger for road sign recognition, and based on the map scene data of the detour point, a series of operations such as road sign discovery, structural analysis, parametric modeling, and automatic texturing are automatically completed, realizing the large-scale automatic production of road sign models. While ensuring the quality of road sign model generation, the generation efficiency of road sign models is improved.

[0196] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0197] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating a road sign model, comprising: Based on the road network data associated with the detour points in the road, determine the scene image of the target area where the detour points are located; Based on the view constraint data of the scene image, determine the sign image representing the road sign from the scene image; Identify the road signs in the sign image and determine the structural parameters of the signboard and supporting structure in the road signs; A three-dimensional model of the signboard and the supporting structure is constructed based on the structural parameters, and the texture data of the signboard is added to the three-dimensional model of the signboard based on the sign image to generate the sign model of the road sign.

2. The method according to claim 1, wherein, The step of determining the sign image representing the road sign from the scene image based on the view constraint data of the scene image includes: Based on the field of view constraint data, the scene images are subjected to field of view visibility analysis to filter out valid scene images with field of view visibility. The visual understanding model is used to determine whether the valid scene image includes road signs. In response to the inclusion of a road sign in the valid scene image, the sign image is determined from the valid scene image.

3. The method according to claim 2, wherein, The step of responding to the inclusion of a road sign in the valid scene image and determining the sign image from the valid scene image includes: In response to the inclusion of road signs in the effective scene image, the completeness of the road signs in the effective scene image is determined by the visual understanding big model; In response to determining that the road sign in the valid scene image has integrity, the sign image is determined from the valid scene image.

4. The method according to claim 1, wherein, The process of identifying road signs in the sign image and determining the structural parameters of the signboard and supporting structure of the road signs includes: The road sign is determined from the sign image using a large image segmentation model; Identify the overall structural parameters of the road sign; The structural parameters of the signboard are identified using a target object detection model; Identify the area of ​​the road sign other than the signboard and determine the structural parameters of the supporting structure.

5. The method according to claim 4, wherein, The step of identifying the structural parameters of the signboard using a target object detection model includes: The signboard is identified using the target object detection model, and the signboard detection result is obtained. The contour of the signboard in the signboard detection result is optimized by using a contour extraction algorithm to obtain the optimized detection result; Based on the optimized test results, the structural parameters of the signboard are determined.

6. The method according to claim 1, wherein, The step of constructing a three-dimensional model of the signboard and the supporting structure based on the structural parameters, and adding the texture data of the signboard to the three-dimensional model of the signboard based on the sign image to generate the sign model of the road sign includes: A three-dimensional model of the signboard is constructed based on its structural parameters; Construct a three-dimensional model of the supporting structure based on its structural parameters; Based on the topological relationship between the signboard and the supporting structure, assemble the three-dimensional models of the signboard and the supporting structure to obtain the assembled model; The texture data of the signboard is mapped to the three-dimensional model of the signboard in the assembly model to obtain the signboard model.

7. The method according to claim 6, wherein, The step of mapping the texture data of the signboard to the three-dimensional model of the signboard in the assembly model includes: The texture data is scaled according to the size of the three-dimensional model of the signboard to obtain a scaled texture that is adapted to the size of the three-dimensional model of the signboard. The scaled texture is mapped onto the 3D model of the signboard in the assembled model.

8. The method according to claim 1, wherein, Before identifying the road sign in the sign image and determining the structural parameters of the signboard and supporting structure in the road sign, the method further includes: The missing parts of the sign image are redrawn in high definition using a graph-based large-scale model, and the visual attributes of the redrawn sign image are optimized according to the preset road sign layout specifications.

9. The method according to claim 1, wherein, The step of determining the scene image of the target area where the deviation point is located based on road network data associated with the deviation point in the road includes: Based on historical navigation trajectory data and historical route planning data, determine the deviation points in the road; The conditions for acquiring the scene image are determined based on the road network data associated with the yaw point; Based on the acquisition conditions, the scene image of the target area is determined from the complete scene image of the road.

10. The method according to any one of claims 1-9, wherein, Also includes: Based on the absolute spatial information of the road sign, coordinate transformation and spatial registration are performed on the sign model to transform the sign model from the relative modeling coordinate system to the geographic coordinate system used in map applications.

11. The method according to claim 10, wherein, The step of performing coordinate transformation and spatial registration on the sign model based on the absolute spatial information of the road sign, so as to transform the sign model from a relative modeling coordinate system to the geographic coordinate system of the map application, includes: The scaling factor of the sign model is determined based on the distance between the spatial locations of the road signs in the geographic coordinate system and the distance between the corresponding spatial locations of the sign model in the relative modeling coordinate system. The sign model is scaled according to the scaling factor to obtain the scaled model; Based on the spatial orientation constraints of the road sign in the geographic coordinate system, the rotation angle and spatial position of the scaled model are adjusted to obtain the adjusted model. The adjusted model is transformed from the relative modeling coordinate system to the geographic coordinate system.

12. A device for generating road sign models, comprising: The scene determination unit is configured to determine the scene image of the target area where the deviance point is located based on road network data associated with the deviance point in the road; The sign determination unit is configured to determine a sign image representing a road sign from the scene image based on the field-of-view constraint data of the scene image; The parameter determination unit is configured to identify road signs in the sign image and determine the structural parameters of the signboard and supporting structure in the road signs. The model generation unit is configured to construct a three-dimensional model of the signboard and the supporting structure based on the structural parameters, and to add the texture data of the signboard to the three-dimensional model of the signboard based on the sign image, thereby generating a sign model of the road sign.

13. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

15. A computer program product comprising: A computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images