Vehicle repositioning method, device and equipment based on image character recognition

Abstract

The application discloses a vehicle repositioning method, device and equipment based on image character recognition. The method comprises the following steps: collecting an original image containing fixed character identification when vehicle positioning is not initialized or lost, performing character recognition after distortion removal and bird's-eye image conversion, and aggregating adjacent detection results to obtain aggregated character content, an aggregated area and an aggregated detection angle; matching a pre-built character point cloud map according to the character content, determining a map character point cloud and a map character direction, extracting an image character feature point cloud, determining a vehicle heading angle search range in combination with an angle difference, obtaining a repositioning pose of the vehicle in a global coordinate system through point cloud registration, and taking the repositioning pose as an initialization pose of vehicle positioning. The technical scheme of the application can complete point cloud registration, quickly obtain a global initial pose when positioning is not initialized or lost, and improve repositioning efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a vehicle relocation method, apparatus, and device based on image character recognition. Background Technology

[0002] Autonomous vehicles require continuous and accurate global pose for autonomous navigation, path planning, and safety control. Existing positioning methods mainly include Global Navigation Satellite System (GNSS) / Real-time Dynamic Differential (RTK) positioning, LiDAR positioning, Visual SLAM positioning, and Inertial Navigation positioning. Among these, GNSS / RTK positioning relies on satellite signals, which are prone to weak or unusable signals in obstructed environments such as tunnels, under viaducts, urban canyons, underground parking lots, and enclosed parks. While LiDAR positioning can achieve high accuracy through point cloud matching, it usually requires accurate initial pose after cold start or positioning loss. Otherwise, it needs to perform a global search or multiple matching attempts, which is time-consuming and has an unstable success rate. Visual SLAM depends on environmental texture and lighting conditions, and its robustness is insufficient in scenarios with poor texture or large lighting variations. Inertial navigation suffers from cumulative errors and cannot provide reliable global position information independently over a long period of time.

[0003] In application scenarios such as closed parks, industrial parks, ports, underground parking lots, and warehousing and logistics sites, the ground is usually marked with fixed character labels such as parking space numbers, area codes, road numbers, or cargo space numbers. These character labels have the characteristics of fixed location, clear semantic information, obvious visual features, and no reliance on satellite signals, and can serve as important references for vehicle relocation. However, current technologies still do not fully utilize fixed character labels. When vehicle cameras collect ground characters, they are easily affected by perspective distortion, leading to character deformation and decreased recognition accuracy. Furthermore, character recognition results may be split into multiple fragments, affecting the stability of the matching between character content and pre-built character point cloud maps. At the same time, it is difficult to accurately establish the geometric correspondence between image character features and map point clouds. Point cloud registration involves a large angle search range and high computational cost, and direct registration is prone to getting trapped in local optima, making it difficult to obtain a reliable global initial pose in a short time. How to properly solve the above problems is an urgent issue for the industry to address. Summary of the Invention

[0004] This invention provides a vehicle repositioning method, apparatus, and device based on image character recognition, which can quickly obtain the vehicle's global initial pose, narrow the heading search range, and improve repositioning efficiency, accuracy, and robustness.

[0005] According to a first aspect of the present invention, a vehicle relocation method based on image character recognition is provided, the vehicle relocation method based on image character recognition includes: When the vehicle positioning is not initialized or lost, the original image containing fixed character identifiers is acquired, the original image is distorted, and the distorted image is converted into a bird's-eye view image. Character recognition is performed on the bird's-eye view image to obtain at least one detection result, the detection result including character content, detection box and detection angle, and spatially adjacent detection results are aggregated to obtain an aggregated character result including aggregated character content, aggregated region and aggregated detection angle; Based on the character content in the aggregated character result, the corresponding map character point cloud and map character direction are determined in the pre-built character point cloud map, and the image character feature point cloud is extracted from the character region corresponding to the aggregated character result. Calculate the angle difference between the map character direction and the aggregated detection angle in the aggregated character result, determine the vehicle heading angle search range based on the angle difference, register the image character feature point cloud with the map character point cloud within the vehicle heading angle search range, obtain the vehicle's repositioning pose in the global coordinate system, and use the repositioning pose as the vehicle's initial positioning pose.

[0006] In one embodiment, converting the distortion-corrected image into a bird's-eye view image includes: Determine the four corner points of the bird's-eye view area in the vehicle coordinate system; The four corner points are transformed from the vehicle coordinate system to the camera coordinate system based on the inverse matrix of the camera extrinsic parameters, and the transformed three-dimensional points are projected onto the pixel coordinate system based on the camera intrinsic parameter matrix. The perspective transformation matrix is ​​calculated based on the four projection points in the pixel coordinate system and the target plane point corresponding to the bird's-eye view area. The distortion-free image is then processed using the perspective transformation matrix to obtain the bird's-eye view image.

[0007] In one embodiment, aggregating the detection results of spatial adjacency to obtain aggregated character results includes: The character regions in the bird's-eye view image are identified to obtain the detection box, detection angle, and character content of each character region. Calculate the center point coordinates of each detection box, perform breadth-first search clustering based on the distance between the center points, and classify the detection boxes that are connected and whose distance is less than a preset threshold into the same category. The character content is merged according to the arrangement order of the detection boxes of the same category in the bird's-eye view image. The average value of the detection angles within the same category is calculated, and the envelope rectangle of each detection box within the same category is calculated to obtain the aggregated character result, which includes the merged character content, the aggregated detection angle, and the aggregated region.

[0008] In one embodiment, determining the corresponding map character point cloud and map character direction, and extracting image character feature point cloud from the character region corresponding to the aggregated character result, includes: Using the merged character content as an index, the point cloud data file and direction record corresponding to the merged character content are searched in the pre-built character point cloud map. The pre-built character point cloud map includes point cloud data files and direction records associated with the character content. Load the point cloud data file to obtain a map character point cloud located in the global coordinate system, and read the direction record to obtain the map character direction; Within the aggregation region, grayscale conversion, Gaussian blurring, adaptive thresholding, morphological opening operation, Canny edge detection, and contour extraction are performed to obtain the character edge contour point cloud; Based on the pixel coordinates of the character edge contour point cloud in the bird's-eye view image, the pixel offset of the aggregation region, and the scale relationship between the bird's-eye view image and the vehicle coordinate system, the character edge contour point cloud is converted into an image character feature point cloud in the vehicle coordinate system.

[0009] In one embodiment, determining the vehicle heading angle search range and registering the image character feature point cloud with the map character point cloud within the vehicle heading angle search range includes: Voxel downsampling is performed on the image character feature point cloud and the map character point cloud respectively to obtain downsampled image point cloud and downsampled map point cloud; Calculate the angle difference between the map character direction and the aggregated detection angle, and determine the vehicle heading angle search range with the angle difference as the center and a preset angle threshold as the angle half-width; Multiple candidate heading angles are generated within the vehicle heading angle search range. The downsampled image point cloud is rotated according to each candidate heading angle. The nearest neighbor distance between the rotated downsampled image point cloud and the downsampled map point cloud is calculated using a KD tree. The candidate heading angle with the smallest average nearest neighbor distance is selected as the optimal heading angle.

[0010] In one embodiment, obtaining the repositioning pose of the vehicle in the global coordinate system includes: The image character feature point cloud is rotated according to the optimal heading angle, and the centroids of the rotated image character feature point cloud and the map character point cloud are calculated respectively. The translation amount is determined based on the difference between the two centroids, and an initial transformation matrix is ​​constructed using the optimal heading angle and the translation amount. Using the initial transformation matrix as the initial value, under the constraints of the maximum number of iterations, the maximum corresponding distance, and the transformation convergence threshold, a two-dimensional constraint iterative nearest point registration is performed on the image character feature point cloud and the map character point cloud to obtain a fine registration transformation matrix; The translation and yaw angles in the fine registration transformation matrix are used as the planar pose components of the three-dimensional rigid body pose, and the repositioning pose of the vehicle in the global coordinate system is determined based on the preset height, roll angle and pitch angle.

[0011] According to a second aspect of the present invention, a vehicle relocation device based on image character recognition is provided, comprising: The acquisition module is used to acquire an original image containing fixed character identifiers when the vehicle positioning is not initialized or lost, to perform distortion correction on the original image, and to convert the distortion-corrected image into a bird's-eye view image. The recognition module is used to perform character recognition on the bird's-eye view image to obtain at least one detection result. The detection result includes character content, detection box and detection angle. It also aggregates spatially adjacent detection results to obtain an aggregated character result including aggregated character content, aggregated region and aggregated detection angle. The extraction module is used to determine the corresponding map character point cloud and map character direction in the pre-built character point cloud map based on the character content in the aggregated character result, and to extract the image character feature point cloud from the character region corresponding to the aggregated character result. The registration module is used to calculate the angle difference between the direction of the map character and the aggregated detection angle in the aggregated character result, determine the vehicle heading angle search range based on the angle difference, register the image character feature point cloud with the map character point cloud within the vehicle heading angle search range, obtain the vehicle repositioning pose in the global coordinate system, and use the repositioning pose as the vehicle positioning initialization pose.

[0012] According to a third aspect of the present invention, an electronic device is provided, comprising: a communication interface, a processor, and a memory; The memory is used to store program instructions, which, when executed by the processor that is connected to the memory via the communication interface, implement any of the above-described vehicle relocation methods based on image character recognition.

[0013] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed by a computer (e.g., a processor in the computer), implement any of the above-described vehicle relocation methods based on image character recognition.

[0014] In summary, the vehicle relocalization method based on image character recognition provided by the present invention includes: when vehicle localization is not initialized or lost, acquiring an original image containing fixed character identifiers, performing distortion correction on the original image, and converting the distortion-corrected image into a bird's-eye view image; performing character recognition on the bird's-eye view image to obtain at least one detection result, the detection result including character content, detection box, and detection angle, and aggregating spatially adjacent detection results to obtain an aggregated character result including aggregated character content, aggregated region, and aggregated detection angle; determining the corresponding map character point cloud and map character direction in a pre-built character point cloud map based on the character content in the aggregated character result, and extracting image character feature point cloud from the character region corresponding to the aggregated character result; calculating the angle difference between the map character direction and the aggregated detection angle in the aggregated character result, determining the vehicle heading angle search range based on the angle difference, registering the image character feature point cloud with the map character point cloud within the vehicle heading angle search range to obtain the relocalization pose of the vehicle in the global coordinate system, and using the relocalization pose as the vehicle localization initialization pose. The technical solution of this invention performs distortion correction, bird's-eye view image conversion, and character aggregation recognition on fixed character identification images. Combined with a pre-built character point cloud map, it determines the heading angle search range and completes point cloud registration. This allows for rapid acquisition of the global initial pose when the positioning is not initialized or lost, thereby improving repositioning efficiency and accuracy.

[0015] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and drawings.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a vehicle relocation method based on image character recognition, provided as an embodiment of the present invention; Figure 2 A flowchart of another vehicle relocation method based on image character recognition provided for an embodiment of the present invention; Figure 3A flowchart illustrating yet another vehicle relocation method based on image character recognition provided for an embodiment of the present invention; Figure 4 A flowchart illustrating yet another vehicle relocation method based on image character recognition provided for an embodiment of the present invention; Figure 5 A flowchart illustrating yet another vehicle relocation method based on image character recognition provided for an embodiment of the present invention; Figure 6 A flowchart illustrating yet another vehicle relocation method based on image character recognition provided for an embodiment of the present invention; Figure 7 A structural diagram of a vehicle repositioning device based on image character recognition is provided for an embodiment of the present invention; Figure 8 This is a structural diagram of a vehicle relocation electronic device based on image character recognition, provided as an embodiment of the present invention. Detailed Implementation

[0019] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention and not to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the invention.

[0020] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0021] The table below provides a comparison of English abbreviations, their full English expressions, and their Chinese terms.

[0022]

[0023] like Figure 1As shown, the present invention provides a vehicle relocation method based on image character recognition, which includes: In step S11, when the vehicle positioning is not initialized or lost, an original image containing fixed character identifiers is acquired, the original image is distorted, and the distorted image is converted into a bird's-eye view image. In step S12, character recognition is performed on the bird's-eye view image to obtain at least one detection result. The detection result includes character content, detection box, and detection angle. Spatially adjacent detection results are aggregated to obtain an aggregated character result including aggregated character content, aggregated region, and aggregated detection angle. In step S13, based on the character content in the aggregated character result, the corresponding map character point cloud and map character direction are determined in the pre-built character point cloud map, and the image character feature point cloud is extracted from the character region corresponding to the aggregated character result. In step S14, the angle difference between the map character direction and the aggregated detection angle in the aggregated character result is calculated, the vehicle heading angle search range is determined based on the angle difference, the image character feature point cloud and the map character point cloud are registered within the vehicle heading angle search range to obtain the vehicle's repositioning pose in the global coordinate system, and the repositioning pose is used as the vehicle's initial positioning pose.

[0024] In one embodiment, when vehicle positioning is not initialized or lost, an original image containing fixed character identifiers is acquired. This original image is acquired by an onboard image acquisition device while the vehicle is moving or stationary, covering a spatially fixed character identifier area on the ground. The fixed character identifiers can be located on the ground or on a sloping wall. The original image suffers from distortion introduced by the imaging lens. Distortion correction processing is performed on the original image using the camera intrinsic parameter matrix to eliminate the influence of radial and tangential distortion on the geometric structure representation of the characters. The distorted image still retains perspective projection characteristics in its spatial representation. Perspective transformation is performed based on the geometric mapping relationship between the vehicle coordinate system and the imaging plane, converting the distorted image into a bird's-eye view. This ensures that the ground plane area maintains an approximately orthogonal projection relationship in the image representation, thereby reducing the impact of perspective compression on the stability of character structure recognition.

[0025] The technical solution in this embodiment is applicable to the following scenarios: unmanned delivery vehicles in closed parks: industrial parks, logistics parks, campuses, etc., with clear area numbers or parking space numbers on the ground; unmanned container trucks in port terminals: container yards with container space number markings on the ground; automatic parking systems in underground parking lots: parking space numbers are clear, and GNSS signals are unavailable; autonomous driving in tunnel environments: with mileage markers or area numbers; unmanned vehicles in warehousing and logistics: with cargo space number markings in the warehouse.

[0026] A character recognition process is performed on an aerial view image to obtain at least one detection result, which includes character content, detection bounding boxes, and detection angles. Character content represents the semantic information obtained through recognition, detection bounding boxes represent the spatial distribution range of the character in the image, and detection angles represent the rotational posture of the character in the aerial view image. To address the issue of fragmented character output, spatially adjacent detection results are aggregated to form aggregated character results. Aggregated character results include aggregated character content, aggregated region, and aggregated detection angles. Aggregated character content is obtained by merging character content of the same spatially connected category, the aggregated region is determined by the spatial envelope relationship of the detection bounding boxes, and the aggregated detection angle is represented by the statistical values ​​of detection angles of the same category.

[0027] Based on the aggregated character results, index matching is performed in a pre-built character point cloud map according to the aggregated character content to determine the corresponding map character point cloud and map character orientation. The pre-built character point cloud map stores the correspondence between character content and spatial point cloud structure and orientation information, achieving semantic indexing and positioning through character content. After obtaining the map character point cloud through matching, image character feature point clouds are extracted from the character regions corresponding to the aggregated character results. The image character feature point clouds are obtained by performing grayscale processing, Gaussian blurring, adaptive thresholding, morphological opening operation processing, Canny edge detection processing, and contour extraction processing on the character regions, and are then transformed into a vehicle coordinate system representation based on the scale mapping relationship between the bird's-eye view image and the vehicle coordinate system.

[0028] After constructing the image character feature point cloud and the map character point cloud, the angle difference between the map character direction and the aggregated detection angle in the aggregated character result is calculated. Based on this angle difference, the vehicle heading angle search range is determined. The vehicle heading angle search range is centered on the angle difference and limited by a preset angle threshold, with a preferred preset angle threshold of 15 degrees. Within the vehicle heading angle search range, the image character feature point cloud undergoes rotation transformations under different heading angle conditions, and is matched with the map character point cloud. The candidate heading angles are evaluated using a nearest neighbor distance evaluation function to determine the optimal heading angle.

[0029] Under the constraint of optimal heading angle, spatial alignment is performed on the image character feature point cloud and the map character point cloud. The translation amount is determined by centroid alignment to form an initial transformation matrix. Based on the initial transformation matrix, an iterative nearest-point registration process is performed to iteratively optimize the transformation matrix under the two-dimensional plane constraint, so that the distance error between the two types of point clouds gradually converges to within a preset threshold range, thereby obtaining the vehicle's repositioning pose in the global coordinate system.

[0030] After obtaining the repositioned pose of the vehicle in the global coordinate system, the repositioned pose is used as the initial pose for vehicle localization, serving as the initial input for the subsequent LiDAR localization process. This method improves the stability and convergence speed of the initial pose solution through joint constraints of character semantic information and geometric point cloud information, achieving reliable global pose recovery capability in scenarios of lost or uninitialized localization.

[0031] Regarding the use of BEV transform to eliminate perspective distortion, existing technologies perform OCR recognition directly on the original image. However, due to perspective distortion, character deformation is severe, resulting in low recognition accuracy. The technical solution in this embodiment converts the ground image into a top-down view using BEV transform, making the character shape closer to the front view. The achieved technical effects include improving OCR recognition accuracy by 10-20%; the BEV image and map point cloud are at the same top-down perspective, facilitating subsequent registration; and eliminating perspective distortion, resulting in more consistent character size and shape.

[0032] Regarding the detection result aggregation algorithm, in existing technologies, OCR may recognize a complete character as multiple fragments, leading to matching failures or duplicate matches. The technical solution in this embodiment is to design a spatial distance-based BFS clustering algorithm to automatically merge adjacent detection results. The achieved technical effects are: merging scattered recognition results into complete characters, improving the matching success rate; using the BFS algorithm to ensure the connectivity and integrity of clusters; and reducing false matches, improving system robustness.

[0033] Regarding character orientation-assisted angle estimation, existing point cloud registration methods typically require searching within a 360-degree range, resulting in significant computational overhead. This embodiment utilizes pre-stored character orientation information to narrow the search range to ±15 degrees. The resulting improvements include reducing the search range to 1 / 12 of its original size, drastically reducing computational overhead, decreasing search time from several seconds to milliseconds, and improving real-time performance to meet the response speed requirements of autonomous driving.

[0034] Regarding edge feature extraction strategies, existing technologies often use the center point of the OCR detection box directly as a feature, resulting in a limited number of feature points. However, the technical solution in this embodiment extracts character edge contour points as features, resulting in a rich number of feature points (hundreds of points per character). The achieved technical benefits include a more than 100-fold increase in the number of feature points and a significant improvement in registration accuracy; the use of Canny edge detection combined with contour extraction provides strong robustness to changes in illumination; and edge features better represent the geometric shape of the character.

[0035] Regarding the two-stage optimization of coarse estimation and fine registration, existing techniques directly using ICP registration are prone to getting trapped in local optima and involve large computational costs. The technical solution in this embodiment involves rapidly obtaining the initial pose through centroid alignment in the coarse estimation stage, and using the ICP algorithm for precise optimization in the fine registration stage. The achieved technical effects are that the two-stage strategy balances efficiency and accuracy; coarse estimation provides good initial values, avoiding ICP getting trapped in local optima; and the overall computation time is controlled within 1-3 seconds.

[0036] In summary, the overall technical advantages are: rapid initialization: pose calculation is completed within 1-3 seconds, which is 5-10 times faster than traditional methods (10-30 seconds); high-precision positioning: through ICP fine registration, the pose estimation accuracy reaches the centimeter level; strong environmental adaptability: it does not rely on GNSS signals and is suitable for obstructed environments such as tunnels and underground parking lots; low cost: it utilizes existing vehicle-mounted cameras and does not require additional sensors; and high reliability: the text labels have clear semantic information and the probability of mismatch is low.

[0037] The technical solution in this embodiment performs distortion correction, bird's-eye view transformation, and character aggregation recognition on fixed character identifier images. Combined with a pre-built character point cloud map, it determines the heading angle search range and completes point cloud registration. This allows for rapid acquisition of the global initial pose when positioning is not initialized or lost, improving repositioning efficiency and accuracy. Fixed character identifiers are identified by a camera, and the initial pose is quickly calculated (1-3 seconds) through optical character recognition and point cloud registration, providing reliable initialization information for laser positioning. Using fixed character identifiers as positioning features, the text possesses clear semantic information and geometric structure, resulting in high recognition robustness. Through a pre-established character-position mapping relationship, the global position is directly obtained from the recognition results, without relying on global navigation satellite system signals. A multi-level optimization strategy combining bird's-eye view transformation and edge feature extraction with iterative nearest-point algorithm registration improves matching accuracy to the centimeter level. Pre-stored character direction information assists in angle estimation, reducing the search range from 360 degrees to ±15 degrees, significantly improving efficiency.

[0038] In one embodiment, such as Figure 2 As shown, the process of converting the distortion-reduced image into a bird's-eye view image includes the following steps S21-S23: In step S21, the four corner points of the bird's-eye view area are determined in the vehicle coordinate system; In step S22, the four corner points are transformed from the vehicle coordinate system to the camera coordinate system according to the inverse matrix of the camera extrinsic parameters, and the transformed three-dimensional points are projected to the pixel coordinate system according to the camera intrinsic parameter matrix. In step S23, a perspective transformation matrix is ​​calculated based on the four projection points in the pixel coordinate system and the target plane points corresponding to the bird's-eye view area. The distortion-free image is then processed using the perspective transformation matrix to obtain the bird's-eye view image.

[0039] In one embodiment, when vehicle positioning is not initialized or lost, an original image containing fixed character identifiers is acquired. The original image undergoes distortion correction processing to eliminate radial and tangential distortions introduced by the imaging lens, ensuring the fixed character identifiers in the image maintain a relatively stable geometric shape. The distortion-corrected image still exhibits perspective projection characteristics in its spatial representation; therefore, a bird's-eye view image transformation is required based on the geometric mapping relationship between the vehicle coordinate system, camera coordinate system, and pixel coordinate system to obtain an approximately orthogonal representation of the ground plane in image space.

[0040] During the bird's-eye view image transformation, four corner points of the bird's-eye view area are determined in the vehicle coordinate system. These four corner points define the spatial range of the ground area in front of the vehicle and constitute the constraint boundary conditions for the bird's-eye view transformation. The four corner points are transformed from the vehicle coordinate system to the camera coordinate system based on the camera extrinsic parameter inverse matrix, which describes the spatial rigid body transformation relationship between the vehicle and camera coordinate systems. After the coordinate transformation is completed, the transformed 3D points are projected onto the pixel coordinate system based on the camera intrinsic parameter matrix, which describes the imaging mapping relationship from 3D spatial points to a 2D pixel plane.

[0041] Based on the four projection points in the pixel coordinate system and the target plane points corresponding to the bird's-eye view area, a perspective transformation matrix is ​​calculated. This matrix is ​​used to establish a homography mapping relationship between the distortion-corrected image and the target bird's-eye view plane. After obtaining the perspective transformation matrix, the distortion-corrected image is geometrically remapped using this matrix, mapping the image content in the pixel coordinate system to the bird's-eye view coordinate space, thereby obtaining the bird's-eye view image. This ensures that fixed character identifiers maintain consistency in spatial scale and morphological expression, and provides a stable input foundation for subsequent character recognition and point cloud registration.

[0042] In one embodiment, such as Figure 3 As shown, the aggregation of spatially adjacent detection results to obtain aggregated character results includes the following steps S31-S33: In step S31, the character regions in the bird's-eye view image are identified to obtain the detection box, detection angle and character content of each character region; In step S32, the coordinates of the center point of each detection box are calculated, and a breadth-first search clustering is performed based on the distance between the center points to classify the detection boxes that are connected and whose distance is less than a preset threshold into the same category. In step S33, character content is merged according to the arrangement order of detection boxes of the same category in the bird's-eye view image, the average value of detection angles within the same category is calculated, and the envelope rectangle of each detection box within the same category is calculated to obtain aggregated character results including merged character content, aggregated detection angles, and aggregated regions.

[0043] In one embodiment, spatially adjacent detection results are aggregated to generate aggregated character results, reducing the impact of fragmented character recognition results on subsequent character content matching and pose estimation. Character recognition processing is performed on character regions in the bird's-eye view image to obtain at least one detection result. The detection result includes character content, detection boxes, and detection angles. The character content represents the semantic recognition result, the detection boxes represent the spatial position of the character region in the bird's-eye view image, and the detection angles represent the rotation state of the character in the bird's-eye view image. After obtaining the detection results, the center point coordinates of each detection box are calculated. Spatial adjacency relationships are constructed based on the distance between the center points. Detection boxes with a center point distance less than a preset threshold are connected. A breadth-first search algorithm is used to cluster the connected detection boxes, grouping those that meet the connectivity condition into the same category, thereby achieving the aggregation of spatially adjacent detection results. This process addresses the problem of single characters being segmented into multiple detection fragments, improving the consistency of character content representation.

[0044] After classifying the characters, the character content is merged according to the order of the detection boxes of the same category in the bird's-eye view image to form a continuous character representation. The average detection angle within the same category is calculated to obtain the aggregated detection angle. Simultaneously, the envelope rectangle is calculated based on the spatial boundary relationships of the detection boxes within the same category to obtain the aggregated region. The output aggregated character result includes the merged character content, the aggregated detection angle, and the aggregated region. The merged character content is used for subsequent map character point cloud indexing and matching, the aggregated detection angle is used for heading angle estimation, and the aggregated region is used for image character feature point cloud extraction.

[0045] In one embodiment, such as Figure 4 As shown, the step of determining the corresponding map character point cloud and map character direction, and extracting image character feature point cloud from the character region corresponding to the aggregated character result, includes the following steps S41-S43: In step S41, using the merged character content as an index, the point cloud data file and direction record corresponding to the merged character content are searched in the pre-built character point cloud map. The pre-built character point cloud map includes point cloud data files and direction records associated with the character content. In step S42, the point cloud data file is loaded to obtain a map character point cloud located in the global coordinate system, and the direction record is read to obtain the map character direction; In step S43, grayscale conversion, Gaussian blurring, adaptive thresholding, morphological opening operation, Canny edge detection, and contour extraction are performed within the aggregation region to obtain the character edge contour point cloud; In step S44, based on the pixel coordinates of the character edge contour point cloud in the bird's-eye view image, the pixel offset of the aggregation region, and the scale relationship between the bird's-eye view image and the vehicle coordinate system, the character edge contour point cloud is converted into an image character feature point cloud in the vehicle coordinate system.

[0046] In one embodiment, the process of determining the corresponding map character point cloud and map character direction, and extracting image character feature point cloud from the character region corresponding to the aggregated character result is specifically described, so as to realize the joint expression of character semantic information and point cloud geometric information, thereby providing a stable input for subsequent vehicle pose calculation.

[0047] Using the merged character content as an index, a query and match are performed in a pre-built character point cloud map to find the point cloud data file and direction record corresponding to the merged character content. The pre-built character point cloud map establishes a correspondence between character content and spatial point cloud structure and direction information, thereby achieving semantic tag-based map index retrieval. This indexing method can map the identified character content to a structured point cloud representation in a global coordinate system.

[0048] After matching is completed, the point cloud data file is loaded to obtain a map character point cloud located in the global coordinate system, and the direction record is read to obtain the map character direction, which is used to represent the spatial orientation information of the character in the global coordinate system. After obtaining the map character point cloud and map character direction, image processing is performed on the character region corresponding to the aggregated character result. Within the aggregated region, grayscale processing, Gaussian blur processing, adaptive thresholding processing, morphological opening operation processing, Canny edge detection processing, and contour extraction processing are performed sequentially to obtain the character edge contour point cloud.

[0049] Based on the pixel coordinates of the character edge contour point cloud in the bird's-eye view image, the pixel offset of the aggregation region, and the scale relationship between the bird's-eye view image and the vehicle coordinate system, the character edge contour point cloud is converted into an image character feature point cloud in the vehicle coordinate system. The pixel coordinates are used to characterize the two-dimensional positional relationship of the character edge contour point cloud in the bird's-eye view image. The pixel offset of the aggregation region is used to correct the spatial offset relationship of the character region in the bird's-eye view image. The scale relationship is used to complete the scale mapping between pixel space and the vehicle coordinate system, thereby achieving a consistent expression of the image character feature point cloud and the vehicle coordinate system.

[0050] In one embodiment, such as Figure 5 As shown, determining the vehicle heading angle search range and registering the image character feature point cloud with the map character point cloud within the vehicle heading angle search range includes the following steps S51-S53: In step S51, voxel downsampling is performed on the image character feature point cloud and the map character point cloud respectively to obtain downsampled image point cloud and downsampled map point cloud; In step S52, the angle difference between the map character direction and the aggregated detection angle is calculated, and the vehicle heading angle search range is determined with the angle difference as the center and a preset angle threshold as the angle half-width. In step S53, multiple candidate heading angles are generated within the vehicle heading angle search range. The downsampled image point cloud is rotated according to each candidate heading angle. The nearest neighbor distance between the rotated downsampled image point cloud and the downsampled map point cloud is calculated using a KD tree. The candidate heading angle with the smallest average nearest neighbor distance is selected as the optimal heading angle.

[0051] In one embodiment, the determination of the vehicle heading angle search range and the registration process between the image character feature point cloud and the map character point cloud are specifically described to achieve efficient point cloud matching based on character direction constraints. Voxel downsampling is performed on both the image character feature point cloud and the map character point cloud to reduce point cloud density and subsequent matching computation. By setting the voxel resolution, spatially adjacent point cloud data are aggregated into representative points, thereby obtaining downsampled image point clouds and downsampled map point clouds, enabling the point cloud data to maintain structural features while achieving higher computational efficiency.

[0052] After point cloud downsampling is completed, the angle difference between the map character direction and the aggregated detection angle is calculated. Using this angle difference as the center and a preset angle threshold as the half-width, the vehicle heading angle search range is determined. The map character direction represents the character's orientation information in the global coordinate system, while the aggregated detection angle represents the character's observation orientation information in the bird's-eye view image. The difference between these two factors constrains the initial search range of the vehicle heading angle, thereby reducing redundant searches across the entire angle range.

[0053] Multiple candidate heading angles are generated within the vehicle heading angle search range, and these candidate heading angles are discretized according to a preset angle step size. For each candidate heading angle, a rotation transformation is performed on the downsampled image point cloud to spatially align it with the downsampled map point cloud under different heading assumptions. The matching relationship between the rotated downsampled image point cloud and the downsampled map point cloud is calculated using a KD-tree structure for nearest neighbor search, and the nearest neighbor distance is used as the matching error index to obtain the average nearest neighbor distance corresponding to each candidate heading angle.

[0054] After evaluating all candidate heading angles, the candidate heading angle with the smallest average nearest neighbor distance is selected as the optimal heading angle. The point cloud transformation result corresponding to this optimal heading angle is used as the initial constraint in the subsequent pose estimation stage, thereby improving the registration convergence speed and reducing the probability of getting trapped in local optima.

[0055] In one embodiment, such as Figure 6 As shown, obtaining the repositioning pose of the vehicle in the global coordinate system includes the following steps S61-S64: In step S61, the image character feature point cloud is rotated according to the optimal heading angle, and the centroids of the rotated image character feature point cloud and the map character point cloud are calculated respectively. In step S62, the translation amount is determined based on the difference between the two centroids, and an initial transformation matrix is ​​constructed using the optimal heading angle and the translation amount. In step S63, using the initial transformation matrix as the initial value, under the constraints of the maximum number of iterations, the maximum corresponding distance, and the transformation convergence threshold, a two-dimensional constraint iterative nearest point registration is performed on the image character feature point cloud and the map character point cloud to obtain a fine registration transformation matrix. In step S64, the translation component and yaw angle in the fine registration transformation matrix are used as the planar pose components of the three-dimensional rigid body pose, and the repositioning pose of the vehicle in the global coordinate system is determined based on the preset height, roll angle and pitch angle.

[0056] In one embodiment, after obtaining the optimal heading angle, the image character feature point cloud is rotated according to the optimal heading angle to initially correspond with the map character point cloud in terms of direction. The centroid of the rotated image character feature point cloud is calculated, and the centroid of the map character point cloud is calculated, and the translation amount is determined based on the difference between the image centroid and the map centroid. The translation amount is used to characterize the offset of the rotated image character feature point cloud relative to the map character point cloud in planar position.

[0057] After determining the translation amount, an initial transformation matrix is ​​constructed using the optimal heading angle and the translation amount. This initial transformation matrix includes a rotation component determined by the optimal heading angle and a translation component determined by the translation amount, used to describe the rigid body transformation relationship during the initial alignment of the image character feature point cloud to the map character point cloud. Using this initial transformation matrix as the initial value, and under constraints of the maximum number of iterations, the maximum corresponding distance, and the transformation convergence threshold, a two-dimensional constrained iterative nearest-point registration is performed between the image character feature point cloud and the map character point cloud. This gradually reduces the distance between corresponding points in the image character feature point cloud and the map character point cloud, resulting in a fine registration transformation matrix.

[0058] The two-dimensional constraints are used to confine the registration process within the vehicle's driving plane, reducing the impact of height and non-planar attitude degrees of freedom on the registration results. After obtaining the fine registration transformation matrix, the translation and yaw components are extracted from the matrix. These components are then used as the planar pose components of the three-dimensional rigid body pose. Based on preset height, roll angle, and pitch angle, the vehicle's repositioning pose in the global coordinate system is determined. This repositioning pose characterizes the vehicle's position and attitude relative to the global coordinate system containing the map character point cloud and serves as the vehicle's initial positioning pose.

[0059] In one embodiment, Figure 7 This is a block diagram illustrating a vehicle relocation device based on image character recognition, according to an exemplary embodiment. Figure 7 As shown, the vehicle relocation device based on image character recognition includes an acquisition module 71, a recognition module 72, an extraction module 73, and a registration module 74.

[0060] The acquisition module 71 is used to acquire an original image containing fixed character identifiers when the vehicle positioning is not initialized or lost, to perform distortion correction on the original image, and to convert the distortion-corrected image into a bird's-eye view image. The recognition module 72 is used to perform character recognition on the bird's-eye view image to obtain at least one detection result. The detection result includes character content, detection box and detection angle. It also aggregates spatially adjacent detection results to obtain an aggregated character result including aggregated character content, aggregated region and aggregated detection angle. The extraction module 73 is used to determine the corresponding map character point cloud and map character direction in the pre-built character point cloud map based on the character content in the aggregated character result, and to extract the image character feature point cloud from the character region corresponding to the aggregated character result. The registration module 74 is used to calculate the angle difference between the map character direction and the aggregated detection angle in the aggregated character result, determine the vehicle heading angle search range based on the angle difference, register the image character feature point cloud with the map character point cloud within the vehicle heading angle search range, obtain the vehicle repositioning pose in the global coordinate system, and use the repositioning pose as the vehicle positioning initialization pose.

[0061] The acquisition module 71, the recognition module 72, the extraction module 73, and the registration module 74 included in the block diagram of the vehicle relocation device based on image character recognition are controlled to execute the vehicle relocation method based on image character recognition described in any of the above embodiments.

[0062] like Figure 8 As shown, the present invention provides an electronic device 800, which includes: a communication interface, a processor 801, and a memory 802; The memory 802 stores program instructions. When executed by the processor 801, which is connected to the memory 802 via the communication interface, the program instructions, when the vehicle positioning is not initialized or lost, acquire an original image containing fixed character identifiers, perform distortion correction on the original image, and convert the distortion-corrected image into a bird's-eye view image; perform character recognition on the bird's-eye view image to obtain at least one detection result, the detection result including character content, detection box, and detection angle; and aggregate spatially adjacent detection results to obtain an aggregate including aggregated character content, aggregated region, and aggregated detection angle. Character results; based on the character content in the aggregated character results, determine the corresponding map character point cloud and map character direction in the pre-built character point cloud map, and extract the image character feature point cloud from the character region corresponding to the aggregated character results; calculate the angle difference between the map character direction and the aggregated detection angle in the aggregated character results, determine the vehicle heading angle search range based on the angle difference, register the image character feature point cloud with the map character point cloud within the vehicle heading angle search range, obtain the vehicle's repositioning pose in the global coordinate system, and use the repositioning pose as the vehicle's initialization pose.

[0063] This invention provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are executed by a processor, when vehicle positioning is not initialized or lost, the following steps are taken: First, an original image containing fixed character identifiers is acquired. The original image is then distorted and converted into a bird's-eye view image. Next, character recognition is performed on the bird's-eye view image to obtain at least one detection result, which includes character content, a detection box, and a detection angle. Spatially adjacent detection results are aggregated to obtain an aggregated character result, which includes aggregated character content, an aggregated region, and an aggregated detection angle. Based on the character content in the aggregated character result, a corresponding map character point cloud and map character direction are determined in a pre-built character point cloud map. Image character feature point clouds are extracted from the character region corresponding to the aggregated character result. The angle difference between the map character direction and the aggregated detection angle in the aggregated character result is calculated. Based on the angle difference, a vehicle heading angle search range is determined. Within the vehicle heading angle search range, the image character feature point cloud and the map character point cloud are registered to obtain the vehicle's repositioning pose in the global coordinate system. This repositioning pose is then used as the vehicle positioning initialization pose.

[0064] It should be understood that the specific features, operations, and details described above regarding the method of the present invention can also be similarly applied to the apparatus and system of the present invention, or vice versa. Furthermore, each step of the method of the present invention described above can be performed by a corresponding component or unit of the apparatus or system of the present invention.

[0065] It should be understood that the various modules / units of the device of the present invention can be implemented wholly or partially through software, hardware, firmware, or a combination thereof. Each module / unit can be embedded in the processor of a computer device in hardware or firmware form or independent of the processor, or it can be stored in the memory of a computer device in software form for the processor to call to execute the operation of each module / unit. Each module / unit can be implemented as an independent component or module, or two or more modules / units can be implemented as a single component or module.

[0066] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores computer instructions executable by the processor, which, when executed by the processor, instruct the processor to perform steps of the methods of embodiments of the present invention. The computer device can be broadly categorized as a server, terminal, or any other electronic device with the necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc., connected via a system bus. The processor of the computer device can be used to provide the necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system, computer programs, etc. The internal memory can provide an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface and communication interface of the computer device can be used to connect and communicate with external devices via a network. When the computer program is executed by the processor, it performs the steps of the methods of the present invention.

[0067] This invention can be implemented as a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, causes the steps of the methods of embodiments of the invention to be performed. In one embodiment, the computer program is distributed across multiple network-coupled computer devices or processors, such that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be executed by one or more computer devices or processors, and one or more other method steps / operations may be executed by one or more other computer devices or processors. One or more computer devices or processors may execute a single method step / operation, or execute two or more method steps / operations.

[0068] It will be understood by those skilled in the art that the method steps of the present invention can be performed by a computer program instructing related hardware, such as a computer device or processor. The computer program may be stored in a non-transitory computer-readable storage medium, and its execution causes the steps of the present invention to be performed. Depending on the context, any references herein to memory, storage, databases, or other media may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0069] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle relocation method based on image character recognition, characterized in that, include: When the vehicle positioning is not initialized or lost, the original image containing fixed character identifiers is acquired, the original image is distorted, and the distorted image is converted into a bird's-eye view image. Character recognition is performed on the bird's-eye view image to obtain at least one detection result. The detection result includes character content, detection box, and detection angle. Spatially adjacent detection results are aggregated to obtain an aggregated character result including aggregated character content, aggregated region, and aggregated detection angle. Based on the character content in the aggregated character result, the corresponding map character point cloud and map character direction are determined in the pre-built character point cloud map, and the image character feature point cloud is extracted from the character region corresponding to the aggregated character result. Calculate the angle difference between the map character direction and the aggregated detection angle in the aggregated character result, determine the vehicle heading angle search range based on the angle difference, register the image character feature point cloud with the map character point cloud within the vehicle heading angle search range, obtain the vehicle's repositioning pose in the global coordinate system, and use the repositioning pose as the vehicle's initial positioning pose.

2. The vehicle relocation method based on image character recognition as described in claim 1, characterized in that, The process of converting the distortion-reduced image into a bird's-eye view image includes: Determine the four corner points of the bird's-eye view area in the vehicle coordinate system; The four corner points are transformed from the vehicle coordinate system to the camera coordinate system based on the inverse matrix of the camera extrinsic parameters, and the transformed three-dimensional points are projected onto the pixel coordinate system based on the camera intrinsic parameter matrix. The perspective transformation matrix is ​​calculated based on the four projection points in the pixel coordinate system and the target plane point corresponding to the bird's-eye view area. The distortion-free image is then processed using the perspective transformation matrix to obtain the bird's-eye view image.

3. The vehicle relocation method based on image character recognition as described in claim 2, characterized in that, The aggregation of spatially adjacent detection results to obtain aggregated character results includes: The character regions in the bird's-eye view image are identified to obtain the detection box, detection angle, and character content of each character region. Calculate the center point coordinates of each detection box, perform breadth-first search clustering based on the distance between the center points, and classify the detection boxes that are connected and whose distance is less than a preset threshold into the same category. The character content is merged according to the arrangement order of the detection boxes of the same category in the bird's-eye view image. The average value of the detection angles within the same category is calculated, and the envelope rectangle of each detection box within the same category is calculated to obtain the aggregated character result, which includes the merged character content, the aggregated detection angle, and the aggregated region.

4. The vehicle relocation method based on image character recognition as described in claim 3, characterized in that, The process of determining the corresponding map character point cloud and map character direction, and extracting image character feature point cloud from the character region corresponding to the aggregated character result, includes: Using the merged character content as an index, the point cloud data file and direction record corresponding to the merged character content are searched in the pre-built character point cloud map. The pre-built character point cloud map includes point cloud data files and direction records associated with the character content. Load the point cloud data file to obtain a map character point cloud located in the global coordinate system, and read the direction record to obtain the map character direction; Within the aggregation region, grayscale conversion, Gaussian blurring, adaptive thresholding, morphological opening operation, Canny edge detection, and contour extraction are performed to obtain the character edge contour point cloud; Based on the pixel coordinates of the character edge contour point cloud in the bird's-eye view image, the pixel offset of the aggregation region, and the scale relationship between the bird's-eye view image and the vehicle coordinate system, the character edge contour point cloud is converted into an image character feature point cloud in the vehicle coordinate system.

5. The vehicle relocation method based on image character recognition as described in claim 4, characterized in that, The step of determining the vehicle heading angle search range and registering the image character feature point cloud with the map character point cloud within the vehicle heading angle search range includes: Voxel downsampling is performed on the image character feature point cloud and the map character point cloud respectively to obtain downsampled image point cloud and downsampled map point cloud; Calculate the angle difference between the map character direction and the aggregated detection angle, and determine the vehicle heading angle search range with the angle difference as the center and a preset angle threshold as the angle half-width; Multiple candidate heading angles are generated within the vehicle heading angle search range. The downsampled image point cloud is rotated according to each candidate heading angle. The nearest neighbor distance between the rotated downsampled image point cloud and the downsampled map point cloud is calculated using a KD tree. The candidate heading angle with the smallest average nearest neighbor distance is selected as the optimal heading angle.

6. The vehicle relocation method based on image character recognition as described in claim 5, characterized in that, Obtaining the repositioning pose of the vehicle in the global coordinate system includes: The image character feature point cloud is rotated according to the optimal heading angle, and the centroids of the rotated image character feature point cloud and the map character point cloud are calculated respectively. The translation amount is determined based on the difference between the two centroids, and an initial transformation matrix is ​​constructed using the optimal heading angle and the translation amount. Using the initial transformation matrix as the initial value, under the constraints of the maximum number of iterations, the maximum corresponding distance, and the transformation convergence threshold, a two-dimensional constraint iterative nearest point registration is performed on the image character feature point cloud and the map character point cloud to obtain a fine registration transformation matrix; The translation and yaw angles in the fine registration transformation matrix are used as the planar pose components of the three-dimensional rigid body pose, and the repositioning pose of the vehicle in the global coordinate system is determined based on the preset height, roll angle and pitch angle.

7. A vehicle repositioning device based on image character recognition, characterized in that, include: The acquisition module is used to acquire an original image containing fixed character identifiers when the vehicle positioning is not initialized or lost, to perform distortion correction on the original image, and to convert the distortion-corrected image into a bird's-eye view image. The recognition module is used to perform character recognition on the bird's-eye view image to obtain at least one detection result. The detection result includes character content, detection box and detection angle. It also aggregates spatially adjacent detection results to obtain an aggregated character result including aggregated character content, aggregated region and aggregated detection angle. The extraction module is used to determine the corresponding map character point cloud and map character direction in the pre-built character point cloud map based on the character content in the aggregated character result, and to extract the image character feature point cloud from the character region corresponding to the aggregated character result. The registration module is used to calculate the angle difference between the direction of the map character and the aggregated detection angle in the aggregated character result, determine the vehicle heading angle search range based on the angle difference, register the image character feature point cloud with the map character point cloud within the vehicle heading angle search range, obtain the vehicle repositioning pose in the global coordinate system, and use the repositioning pose as the vehicle positioning initialization pose.

8. The vehicle relocation device based on image character recognition as described in claim 7, characterized in that: The acquisition module, the recognition module, the extraction module, and the registration module are controlled to execute the vehicle relocation method based on image character recognition as described in any one of claims 2 to 6.

9. An electronic device, characterized in that, include: Communication interface, processor, memory; The memory is used to store program instructions, which, when executed by the processor that is communicatively connected to the memory via the communication interface, cause the electronic device to implement the vehicle relocation method based on image character recognition as described in any one of claims 1 to 6.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by a computer, the computer implements the vehicle relocation method based on image character recognition as described in any one of claims 1 to 6.

Images