Positioning method and device for hectometer stake label

Abstract

The invention relates to the field of traffic, in particular to a hectometer stake sign positioning method and device. The method comprises the following steps: acquiring a target image which is shot by a camera and comprises a hectometer pile label, inputting the target image into a trained image recognition model, and recognizing label information on the hectometer pile label in the target image and corresponding key point pixel coordinates of the hectometer pile label in the target image; and finally, based on the pixel coordinates of the key points and camera parameters of a camera, performing coordinate conversion on the pixel coordinates of the key points to obtain latitude and longitude coordinates corresponding to the pixel coordinates of the key points, and establishing and recording an association relationship between the latitude and longitude coordinates and the label information. The real position of the hectometer pile sign can be accurately positioned, so that the traffic safety is further guaranteed, and the accident handling efficiency is improved.

Description

Technical Field

[0001] The invention relates to the field of transportation, and in particular to a positioning method and device for a 100-meter pile sign. Background Art

[0002] In modern traffic management systems, when a safety accident occurs on a highway, in order to handle the accident more quickly, a drone can be dispatched to the scene to determine the on-site situation, and then a response plan can be prepared in advance before rushing to the scene of the accident based on the on-site situation, so as to handle the accident more promptly and quickly. In addition, drones are also commonly used to identify pits, cracks, and other features on highways, which are called "road diseases". Maintenance personnel also need to obtain the longitude and latitude of the "road disease" and then convert the longitude and latitude into pile numbers so that they can go to the scene in time to repair the disease.

[0003] In this case, in order to dispatch drones to the accident site, it is very important to accurately locate the accident site. Currently, the accident site is mainly determined by the 100-meter pile signs on the highway. However, there is a deviation between the actual position of the 100-meter pile signs installed by workers and the position indicated by the 100-meter pile signs. This will cause the location of the accident site to deviate, making it impossible for drones to accurately reach the accident site in the first place, reducing the efficiency of handling accidents. When handling accidents, every minute and every second is crucial, and the "road diseases" on the highway should be handled as quickly as possible. Therefore, a positioning method for 100-meter pile signs is urgently needed to accurately locate the actual position of the 100-meter pile signs, so as to further ensure traffic safety and improve the efficiency of accident handling. Summary of the invention

[0004] The invention describes a positioning method and device for a 100-meter stake sign, which can accurately locate the real position of the 100-meter stake sign to further ensure traffic safety and improve accident handling efficiency.

[0005] According to a first aspect, the present invention provides a method for positioning a 100-meter stake sign, comprising:

[0006] Get the target image including the 100-meter stake sign captured by the camera;

[0007] Input the target image into a trained image recognition model to identify the sign information on the 100-meter pile sign in the target image and the key point pixel coordinates corresponding to the 100-meter pile sign in the target image;

[0008] The camera parameters of the camera are obtained, and based on the key point pixel coordinates and the camera parameters, the key point pixel coordinates are converted to obtain the longitude and latitude coordinates corresponding to the key point pixel coordinates, and the association between the longitude and latitude coordinates and the sign information is established and recorded.

[0009] According to a second aspect, the present invention provides a positioning device for a 100-meter stake sign, comprising:

[0010] An acquisition unit, used for acquiring a target image including a 100-meter stake sign taken by a camera;

[0011] A recognition unit, used for inputting the target image into a trained image recognition model, identifying the sign information on the 100-meter pile sign in the target image and the key point pixel coordinates corresponding to the 100-meter pile sign in the target image;

[0012] A positioning unit is used to obtain the camera parameters of the camera, perform coordinate transformation on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters, obtain the longitude and latitude coordinates corresponding to the key point pixel coordinates, and establish and record the association between the longitude and latitude coordinates and the sign information.

[0013] According to the positioning method and device of the hundred-meter pile sign provided by the present invention, by acquiring a target image containing the hundred-meter pile sign taken by a camera, and then inputting the target image into a trained image recognition model, the sign information on the hundred-meter pile sign in the target image and the key point pixel coordinates corresponding to the hundred-meter pile sign in the target image are identified, and finally based on the key point pixel coordinates and the camera parameters of the camera, the key point pixel coordinates are converted to obtain the longitude and latitude coordinates corresponding to the key point pixel coordinates, and the association relationship between the longitude and latitude coordinates and the sign information is established and recorded. The present application can accurately locate the real position of the hundred-meter pile sign to further ensure traffic safety and improve the efficiency of accident handling. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A schematic flow chart of a method for positioning a hundred-meter stake sign according to an embodiment is shown;

[0016] Figure 2 A schematic diagram showing the positions of key point pixels of a hundred-meter stake sign according to an embodiment is shown;

[0017] Figure 3 A schematic diagram showing the division of a hundred-meter stake sign according to an embodiment

[0018] Figure 4The figure is a schematic block diagram of a positioning device for a hundred-meter stake sign according to an embodiment. DETAILED DESCRIPTION

[0019] The solution provided by the present invention is described below in conjunction with the accompanying drawings.

[0020] Figure 1 FIG. 1 is a flow chart of a method for positioning a 100-meter stake sign according to an embodiment. It is understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 1 As shown, the method includes:

[0021] Step 101: Obtain a target image including a 100-meter pile sign captured by a camera.

[0022] In this embodiment, a video of the 100-meter post sign next to the highway can be taken along a real highway using a monocular camera (such as a mobile phone or a monocular camera), and then an image frame containing the 100-meter post sign in the video can be captured as the target image in step 101.

[0023] Step 102: input the target image into a trained image recognition model to identify the sign information on the 100-meter stake sign in the target image and the key point pixel coordinates corresponding to the 100-meter stake sign in the target image.

[0024] In this embodiment, the sign information on the 100-meter post sign includes 100 meters and kilometers, indicating the location indicated by the 100-meter post sign, for example Figure 2 The number 0 on the 100-meter post sign means that the 100-meter number is 0m, and the number 1524 below means that the 100-meter post sign is 1524km. The location represented by the 100-meter post sign is 1524km away from the entrance of the highway.

[0025] The key point pixel coordinates corresponding to the 100-meter pile sign in the target image refer to the key point pixel coordinates corresponding to the 100-meter pile sign in the target image. The key point pixel here refers to Figure 2 Pixel point P is the center point of the 100-meter stake sign shown.

[0026] Step 103, obtain the camera parameters of the camera, perform coordinate transformation on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters, obtain the longitude and latitude coordinates corresponding to the key point pixel coordinates, and establish and record the association between the longitude and latitude coordinates and the sign information.

[0027] In this embodiment, the camera parameters of the camera include at least the focal length f of the camera, the camera internal parameters and the coordinates of the center point of the camera in the three-dimensional world. The camera internal parameters include at least the unit physical length of the horizontal pixels and the vertical pixels of the image captured by the camera on the photosensitive plate of the target camera, and the coordinates of the center point of the camera in the three-dimensional world, i.e., the longitude and latitude coordinates corresponding to the target camera when the target image is captured, are obtained from the GPS data of the camera according to the timestamp corresponding to the time when the camera captures the target image.

[0028] Based on the key point pixel coordinates and the camera parameters, the key point pixel coordinates are transformed into real coordinates in the three-dimensional world by using the camera parameters, and then the real coordinates in the three-dimensional world corresponding to the hundred-meter stake sign are transformed into longitude and latitude coordinates according to the longitude and latitude coordinates of the target camera.

[0029] Among them, the GPS of the camera when the target image is taken can be directly found from the GPS data of the camera according to the timestamp.

[0030] So far, completed Figure 1 Description of the process shown.

[0031] pass Figure 1 The embodiment shown, this embodiment obtains a target image containing a hundred-meter pile sign taken by a camera, and then inputs the target image into a trained image recognition model, identifies the sign information on the hundred-meter pile sign in the target image and the key point pixel coordinates corresponding to the hundred-meter pile sign in the target image, and finally performs coordinate conversion on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters of the camera, obtains the longitude and latitude coordinates corresponding to the key point pixel coordinates, and establishes and records the association relationship between the longitude and latitude coordinates and the sign information. The present application can accurately locate the real position of the hundred-meter pile sign to further ensure traffic safety and improve the efficiency of accident handling.

[0032] For step 102,

[0033] As a preferred implementation, the image recognition model in step 102 may include a sign extraction module and a sign recognition module.

[0034] Among them, the sign extraction module is used to extract the hundred-meter pile sign image block corresponding to the hundred-meter pile sign from the target image, determine the key point pixels in the hundred-meter pile sign image block, and obtain the key point pixel coordinates corresponding to the hundred-meter pile sign. In this embodiment, to extract the hundred-meter pile sign image block from the target image, it is first necessary to identify the area where the hundred-meter pile sign image is located from the target image, determine the bounding box of the hundred-meter pile sign image, and then extract the hundred-meter pile sign image from the target image according to the bounding box of the hundred-meter pile sign image. In the specific implementation, the sign extraction module can be realized by pre-training the corresponding model, which will not be repeated here. After the description is completed Figure 1 After the illustrated process, the training process of the model used to implement the sign extraction module is described in detail.

[0035] The sign recognition module is used to recognize the image block of the 100-meter pile sign to obtain the sign information on the 100-meter pile sign.

[0036] As an embodiment, the sign recognition module can directly recognize the image block of the 100-meter pile sign based on the image recognition model to obtain the sign information on the 100-meter pile sign.

[0037] Preferably, the sign recognition module recognizes the 100-meter pile sign image block in the following manner:

[0038] Divide the 100-meter stake sign image block according to a preset partitioning rule to obtain a 100-meter area image block and a kilometer area image block;

[0039] Determine the classification corresponding to the 100-meter area image block through the classification model in the sign recognition module, and obtain the 100-meter number in the 100-meter pile sign based on the classification corresponding to the 100-meter area image block; different classifications in the classification model correspond to different 100-meter numbers;

[0040] The kilometer area image block is recognized by the character recognition submodule in the sign recognition module to obtain the kilometers in the hundred-meter pile sign; the sign information includes the hundred-meter number and the kilometer number in the hundred-meter pile sign.

[0041] The hundred-meter post sign contains the number of hundred meters and kilometers. The above-mentioned division of the hundred-meter post sign image block according to the preset partitioning rule means that according to the areas where the hundred-meter numbers and kilometers are respectively located, the area containing only the hundred-meter numbers is divided into the hundred-meter area image block, and the area containing only the kilometers is divided into the kilometer area image block. Figure 3 As shown in the figure, the 100-meter pile sign image block can be divided into Figure 3 The 100-meter area image blocks and kilometer area image blocks shown in FIG. Figure 3 The area a in the purple frame is a 100-meter area image block, and the area b in the orange frame is a kilometer area image block.

[0042] The reason why the present embodiment divides the image blocks of the hundred-meter pile sign is that the numerical value of the hundred-meter number in the image block of the hundred-meter pile sign is larger and easier to identify. The model trained by machine learning can recognize the hundred-meter number faster and basically without error. Preferably, the present embodiment also adopts a classification model with faster recognition for recognizing the hundred-meter number. Optionally, the classification model divides the image of the hundred-meter area into 0 to 9 categories according to the hundred-meter number. The hundred-meter number 0 corresponds to the 0th category, the hundred-meter number 1 corresponds to the 1st category, and so on. The hundred-meter number 9 corresponds to the 9th category. In the present embodiment, if the classification model divides the hundred-meter area image block into the 9th category, it indicates that the hundred-meter number corresponding to the hundred-meter area image block is 9. However, the kilometers in the image block of the hundred-meter pile sign are relatively small, especially the image block of the hundred-meter pile sign is a small image extracted from the target image. If the model trained by machine learning recognizes the kilometers, it may be easy to make mistakes. Therefore, it is more accurate to recognize the kilometers through a dedicated character recognition submodule.

[0043] It should be noted that, in this embodiment, the 100-meter pile sign image blocks may not be divided, and other recognition models with better computing power and higher accuracy may be selected to recognize the 100-meter pile sign image blocks. Also, after the 100-meter pile sign image blocks are divided, not only the classification model but also other lightweight recognition models may be used to recognize the 100-meter area image blocks, which is not limited in this application, and this embodiment only provides a preferred method. In addition, the dedicated character recognition submodule used to recognize the kilometer area image blocks is not limited in this application, and for example, OCR character recognition may be used.

[0044] Preferably, the image recognition model further includes an image processing module:

[0045] The image processing module is used to convert the image format of the hundred-meter pile sign image block into a preset format, filter the pixel blocks that meet the filtering rules in the hundred-meter pile sign image block of the preset format, and obtain the filtered hundred-meter pile sign image block; the filtering rules are used to enhance the difference between the pixels corresponding to the hundred-meter pile sign and other pixels in the hundred-meter pile sign image block.

[0046] In this embodiment, the preset format can be set to HSV format, and the filtering rules can be set to filter out pixel blocks whose V value or S value is lower than the preset value, wherein the V value is saturation and the S value is brightness. The purpose of setting the filtering rules in this embodiment is to help remove pixels that may be included in the image block of the 100-meter pile sign and do not belong to the image of the 100-meter pile sign through the filtering rules for segmentation, such as the asphalt background. Setting the filtering rules helps to remove random noise in the image block of the 100-meter pile sign.

[0047] For step 103,

[0048] As a preferred embodiment, in step 103, coordinate transformation is performed on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters, which specifically includes the following steps:

[0049] Establish the first coordinate system, the second coordinate system and the third coordinate system.

[0050] The first coordinate system has the pixel point at the upper left corner of the target image as the origin, the width of the target image as the u-axis, and the height of the target image as the v-axis.

[0051] The second coordinate system takes the center of the target image as the origin, the width of the target image as the x-axis, and the height of the target image as the y-axis.

[0052] The third coordinate system takes the center of the target image as its origin and the axis parallel to the x-axis as its x-axis. c axis, an axis parallel to the y axis as y c The axis and the axis coinciding with the optical axis of the camera are z c axis.

[0053] The first coordinates (u, v) of the key point pixel in the first coordinate system are obtained based on the key point pixel coordinates.

[0054] According to the preset coordinate conversion formula and the camera parameters, the first coordinate (u, v) is converted into the second coordinate (x, y) of the key point pixel in the second coordinate system, and the second coordinate (x, y) is converted into the third coordinate (x, y) in the third coordinate system. c ,y c , z c );

[0055] Based on the acquired latitude and longitude coordinates corresponding to the target image, the third coordinate (x c ,y c , z c ) is converted into the longitude and latitude coordinates corresponding to the pixel coordinates of the key point.

[0056] As an embodiment, obtaining the first coordinates (u, v) of the key point pixel in the first coordinate system based on the key point pixel coordinates specifically includes: scaling the key point pixel coordinates to coordinates corresponding to the size of the original image, and determining the coordinates corresponding to the size of the original image as the first coordinates (u, v).

[0057] In this example, the pixel coordinates of the key point can be directly obtained according to the pixel distribution on the target image. In order to avoid displacement of the pixel coordinates of the key point on the target image, when determining the first coordinate (u, v), the pixel coordinates of the key point need to be scaled to the coordinates corresponding to the size of the original image.

[0058] The above-mentioned converting the first coordinate (u, v) into the second coordinate (x, y) of the key point pixel in the second coordinate system includes:

[0059] The first coordinate (u, v) is converted to the second coordinate (x, y) of the key point pixel in the second coordinate system according to the following formula:

[0060]

[0061] Wherein, (u0, v0) is the coordinate of the center of the target image in the first coordinate system, dx is the unit physical length of the horizontal pixel on the photosensitive plate, and dy is the unit physical length of the vertical pixel on the photosensitive plate.

[0062] The above converts the second coordinate (x, y) into the third coordinate (x c ,y c , z c ),include:

[0063] The second coordinate (x, y) is converted into the third coordinate (x c ,y c , z c ), the formula is:

[0064]

[0065] Wherein, f is the focal length of the camera, the third coordinate (x c ,y c , z c ) c The determination is based on a scale plate carrying a scale placed when the camera captures the target image.

[0066] In specific implementation, when using a target camera to shoot a 100-meter pile sign, a scale plate can be set below the target camera to determine the depth of the 100-meter pile sign captured by the camera (i.e., z c ). The scale plate can be set to be located below the camera and parallel to the ground, with the scale origin on the scale plate located on the optical axis of the camera and the scale direction being aligned with the x-axis of the third coordinate system. c Axis parallel.

[0067] Then zc It can be calculated by the following formula:

[0068]

[0069] Among them, z1 is the scale value corresponding to the intersection between the center of the target image and the straight line connecting the 100-meter stake sign in the target image and the scale of the scale plate, h is the height between the camera and the ground, and h1 is the height between the camera and the scale plate.

[0070] For step 103,

[0071] Based on the fact that the target image is an image frame obtained from a video, the present application has another preferred embodiment, in which coordinate transformation is performed on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters, specifically including the following steps:

[0072] Acquire at least one image frame containing the 100-meter stake sign adjacent to the target image as a reference image;

[0073] Inputting the target image into a trained depth recognition model to identify the first three-dimensional coordinates corresponding to the pixel coordinates of the key point;

[0074] Inputting the reference image into the depth recognition model, identifying the second three-dimensional coordinates corresponding to the pixel coordinates of the key points of the 100-meter pile sign in the reference image;

[0075] Based on the first three-dimensional coordinates and the second three-dimensional coordinates, the camera position deviation when the camera shoots the target image and the reference image is calculated, and the error between the camera position deviation and the longitude and latitude deviation when the camera shoots the target image and the reference image in the camera parameters is calculated. If the error is less than a first preset error threshold, the longitude and latitude coordinates corresponding to the hundred-meter pile sign are calculated according to the first three-dimensional coordinates and the second three-dimensional coordinates. If the error is not less than the first preset error threshold, the steps of establishing the first coordinate system, the second coordinate system and the third coordinate system are executed.

[0076] When the target image is an image frame obtained from a video, the image frame adjacent to the target image generally also includes an image of a hundred-meter pile sign. Positioning the hundred-meter pile sign through multiple images is conducive to accurately positioning the position of the hundred-meter pile sign. Therefore, in this embodiment, the image frame including the hundred-meter pile sign adjacent to the target image is determined as a reference image, and then the first three-dimensional coordinates of a hundred-meter pile sign are determined based on the target image, and the second three-dimensional coordinates of at least one hundred-meter pile sign are determined based on the reference image. The camera position deviation when the camera shoots the target image and the reference image is calculated through the first three-dimensional coordinates and the second three-dimensional coordinates, and the first three-dimensional coordinates and the second three-dimensional coordinates calculated by the depth recognition model are determined to be accurate.

[0077] The first three-dimensional coordinate in this embodiment is in a coordinate system established based on the center point of the camera when shooting the target image as the origin, and the second three-dimensional coordinate is in a coordinate system established based on the center point of the camera when shooting the reference image as the origin. Since the camera is moving when shooting the video, the center point of the camera when shooting the target image and the center point of the camera when shooting the reference image are in different positions in the real world (i.e., different longitude and latitude), which will also cause the first three-dimensional coordinate and the second three-dimensional coordinate to be different. This embodiment is based on the first three-dimensional coordinate and the second three-dimensional coordinate identified by the depth recognition model, and the principle of geometry, and inversely calculates the distance moved by the camera when shooting the target image and the reference image (i.e., the camera position deviation), and then compares the calculated camera position deviation with the longitude and latitude deviation of the real camera when shooting the target image and the reference image, so as to determine whether the first three-dimensional coordinate and the second three-dimensional coordinate are accurate.

[0078] In this embodiment, if the error between the above-mentioned camera position deviation and the longitude and latitude deviation is less than the first preset error threshold, it is determined that the first three-dimensional coordinate and the second three-dimensional coordinate are accurate, and the first three-dimensional coordinate or the second three-dimensional coordinate can be directly converted into the longitude and latitude coordinates of the hundred-meter stake sign, or, the first three-dimensional coordinate and the second three-dimensional coordinate can be converted into longitude and latitude coordinates respectively, and then the coordinate values ​​of the converted longitude and latitude coordinates are added and averaged respectively, and the averaged coordinate values ​​are determined as the longitude and latitude coordinates of the hundred-meter stake sign.

[0079] If the error between the camera position deviation and the longitude and latitude deviation is not less than the first preset error threshold, it is determined that the first three-dimensional coordinate and the second three-dimensional coordinate are inaccurate, and the longitude and latitude coordinates of the 100-meter pile sign can be obtained by establishing the first coordinate system, the second coordinate system and the third coordinate system in another embodiment. The first preset error threshold can be determined according to the error probability of the depth recognition model.

[0080] As an embodiment, the step of the depth recognition model recognizing the first three-dimensional coordinate or the second three-dimensional coordinate includes:

[0081] translating each pixel in the image to be identified according to different translation lengths, and upsampling the multiple images obtained after the translation to obtain multiple feature maps corresponding to the image to be identified and a disparity map corresponding to each feature map; the image to be identified is the target image or the reference image, and the disparity map is used to represent the disparity between the pixel in the feature map and the pixel in the image to be identified;

[0082] Extracting pixel points of a specific area in each feature map, and forming pixel point pairs with the extracted adjacent pixel points; each pixel point in the image to be identified has a corresponding pixel point in the pixel points extracted from each feature map;

[0083] Selecting pixel points representing the same object according to the disparity map and accumulating them respectively, and adjusting the distances between the pixel points obtained after the accumulation based on the pixel pairs, so as to obtain a combined view corresponding to the image to be recognized;

[0084] The pixel points of the image to be identified are matched with the pixel points of the combined view, and based on the pixel difference between the matched pixel points, the three-dimensional coordinates corresponding to the pixel coordinates of the key points of the 100-meter pile image in the image to be identified are calculated.

[0085] In this embodiment, the depth recognition model can be obtained by training based on a training set of a small number of real views. Among them, the above steps are only the recognition algorithm logic inside the depth recognition model in this application. The combined view is a view generated by the depth recognition model based on the image to be recognized and has the same image content as the image to be recognized but a different perspective. In the above embodiments, the different translation lengths, specific areas in each feature map, and the pixels to be selected according to the disparity map are all parameters obtained by training the depth recognition model.

[0086] In this embodiment, the distance between the pixels obtained by accumulating the pixel pairs is adjusted based on the correlation feature between the adjacent pixels in the pixel pairs, so as to avoid distortion of the image content in the combined view.

[0087] In this embodiment, based on the pixel difference between the matching pixel points, the three-dimensional coordinates corresponding to the pixel coordinates of the key points in the 100-meter pile image in the image to be identified are calculated. The specific method of calculating the three-dimensional coordinates in this step can refer to the process of binocular image depth calculation in the relevant technology, which will not be repeated here.

[0088] Optionally, as a preferred embodiment, if the error between the third coordinate obtained according to the first coordinate system, the second coordinate system and the third coordinate system and the first three-dimensional coordinate is greater than a second preset error threshold, the target image including the hundred-meter stake sign is re-photographed by a camera, and the hundred-meter stake sign is positioned based on the re-photographed target image.

[0089] In this embodiment, if the error between the third coordinate obtained according to the first coordinate system, the second coordinate system and the third coordinate system and the first three-dimensional coordinate is greater than the second preset error threshold, it means that when calculating the third coordinate, the scale plate is used to calculate z c A data recording error may have occurred, and the exact coordinates of the 100-meter pile sign cannot be determined at this time. It is necessary to re-shoot the target image including the 100-meter pile sign with a camera, and then locate the 100-meter pile sign based on the re-shot target image. The second preset error threshold is the maximum error between the recognition result of the depth recognition model and the true result when the depth model is tested.

[0090] As an embodiment, the training process of the image recognition model is described below. The following training process is only an example provided by this application, and this application does not limit the training process of the image recognition model.

[0091] Firstly, we collect image frames containing 100-meter post signs captured by cameras in real road scenes and construct a 100-meter post sign image dataset. The 100-meter post sign image dataset includes samples in different seasons, different light intensities, different angles, and different times of the day. According to the environmental information corresponding to different samples and the sign information on the 100-meter post signs, we manually annotate the 100-meter post sign image dataset to obtain the category of the 100-meter post sign and the pixel coordinates of the bounding box.

[0092] The 100-meter pile sign image dataset obtained above is expanded by data enhancement, and the data enhancement means may include at least one of random scaling and cropping, random horizontal flipping and rotation, mixing, Gaussian noise, HSV transformation, and copying and pasting.

[0093] According to the pixel coordinates of the bounding boxes of the 100-meter pile signs, the manually annotated 100-meter pile sign bounding boxes are clustered (i.e., the bounding boxes with similar pixel coordinates are clustered into one category), and multiple prior boxes of different sizes are generated based on the clustered 100-meter pile sign bounding boxes.

[0094] Set up the initial image recognition model (for example, the YOLOv5 neural network model can be used), randomly shuffle the above data augmented 100-meter pile sign image dataset, and divide it into training set, validation set, and test set. Iterate the training with the training set and validation set to update the parameters of the initial image recognition model, retain the training weight of the round with the highest average accuracy, and then evaluate the performance of the updated image recognition model through the test set. If the accuracy of the image recognition model reaches above the preset accuracy threshold, and the recall rate reaches above the preset recall threshold, the image recognition model training is considered complete.

[0095] Among them, during the training process, the image recognition model can be used to detect the hundred-meter pile signs in the hundred-meter pile sign images. The image recognition model can predict the actual position of the signs in the hundred-meter pile sign images of the test set based on multiple prior frames of different sizes, and then calculate the normalized offset between the actual position of the sign and the prior frame according to the verification set, and then adjust the parameters of the image recognition model for iteration until the bounding box information of each hundred-meter pile sign finally output by the image recognition model passes the test of the test set.

[0096] According to an embodiment of another aspect, the present invention provides a positioning device for a 100-meter stake sign. Figure 4 FIG. 1 is a schematic block diagram of a positioning device for a 100-meter stake sign according to an embodiment. It is understood that the device can be implemented by any device, equipment, platform, and device cluster with computing and processing capabilities. Figure 4 As shown, the device comprises:

[0097] An acquisition unit 401 is used to acquire a target image including a 100-meter stake sign taken by a camera;

[0098] The recognition unit 402 is used to input the target image into the trained image recognition model to recognize the sign information on the 100-meter stake sign in the target image and the key point pixel coordinates corresponding to the 100-meter stake sign in the target image;

[0099] The positioning unit 403 is used to obtain the camera parameters of the camera, perform coordinate transformation on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters, obtain the longitude and latitude coordinates corresponding to the key point pixel coordinates, and establish and record the association between the longitude and latitude coordinates and the sign information.

[0100] As a preferred implementation, the image recognition model includes a sign extraction module and a sign recognition module;

[0101] The sign extraction module is used to extract the 100-meter pile sign image block corresponding to the 100-meter pile sign from the target image, determine the key point pixels in the 100-meter pile sign image block, and obtain the key point pixel coordinates corresponding to the 100-meter pile sign;

[0102] The sign recognition module is used to recognize the image block of the 100-meter pile sign to obtain the sign information on the 100-meter pile sign.

[0103] As a preferred embodiment, the sign recognition module recognizes the 100-meter pile sign image block, including:

[0104] Divide the 100-meter stake sign image block according to a preset partitioning rule to obtain a 100-meter area image block and a kilometer area image block;

[0105] Determine the classification corresponding to the 100-meter area image block through the classification model in the sign recognition module, and obtain the 100-meter number in the 100-meter pile sign based on the classification corresponding to the 100-meter area image block; different classifications in the classification model correspond to different 100-meter numbers;

[0106] The kilometer area image block is recognized by the character recognition submodule in the sign recognition module to obtain the kilometers in the hundred-meter pile sign; the sign information includes the hundred-meter number and the kilometer number in the hundred-meter pile sign.

[0107] As a preferred embodiment, the image recognition model further includes an image processing module:

[0108] The image processing module is used to convert the image format of the hundred-meter pile sign image block into a preset format, filter the pixel blocks that meet the filtering rules in the hundred-meter pile sign image block of the preset format, and obtain the filtered hundred-meter pile sign image block; the filtering rules are used to enhance the difference between the pixels corresponding to the hundred-meter pile sign and other pixels in the hundred-meter pile sign image block.

[0109] As a preferred embodiment, the positioning unit 403 performs coordinate transformation on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters, including:

[0110] Establish a first coordinate system, a second coordinate system, and a third coordinate system; the first coordinate system takes the pixel point at the upper left corner of the target image as the origin, the width of the target image as the u-axis, and the height of the target image as the v-axis; the second coordinate system takes the center of the target image as the origin, the width of the target image as the x-axis, and the height of the target image as the y-axis; the third coordinate system takes the center of the target image as the origin, the axis parallel to the x-axis as the x-axis, and the c axis, an axis parallel to the y axis as y cThe axis and the axis coinciding with the optical axis of the camera are z c axis;

[0111] Obtaining first coordinates (u, v) of the key point pixel in the first coordinate system based on the key point pixel coordinates;

[0112] According to the preset coordinate conversion formula and the camera parameters, the first coordinate (u, v) is converted into the second coordinate (x, y) of the key point pixel in the second coordinate system, and the second coordinate (x, y) is converted into the third coordinate (x, y) in the third coordinate system. c ,y c , z c );

[0113] Based on the acquired latitude and longitude coordinates corresponding to the target image, the third coordinate (x c ,y c , z c ) is converted into the longitude and latitude coordinates corresponding to the pixel coordinates of the key point.

[0114] As a preferred embodiment, the positioning unit 403 obtains the first coordinates (u, v) of the key point pixel in the first coordinate system based on the key point pixel coordinates, including:

[0115] The pixel coordinates of the key point are scaled to coordinates corresponding to the size of the original image, and the coordinates corresponding to the size of the original image are the first coordinates (u, v).

[0116] As a preferred embodiment, the camera parameters include unit physical lengths of horizontal pixels and vertical pixels on the photosensitive plate of the target camera;

[0117] The positioning unit 403 converts the first coordinate (u, v) into the second coordinate (x, y) of the key point pixel in the second coordinate system, including:

[0118] The first coordinate (u, v) is converted to the second coordinate (x, y) of the key point pixel in the second coordinate system according to the following formula:

[0119]

[0120] Wherein, (u0, v0) is the coordinate of the center of the target image in the first coordinate system, dx is the unit physical length of the horizontal pixel on the photosensitive plate, and dy is the unit physical length of the vertical pixel on the photosensitive plate.

[0121] As a preferred embodiment, the third coordinate (x c ,y c , zc ) c Determining based on a scale plate carrying a scale placed when the camera captures the target image;

[0122] The positioning unit 403 converts the second coordinate (x, y) into the third coordinate (x c ,y c , z c ),include:

[0123] The second coordinate (x, y) is converted into the third coordinate (x c ,y c , z c ), the formula is:

[0124]

[0125] Wherein, f is the focal length of the camera.

[0126] As a preferred embodiment, the scale plate is located below the camera and parallel to the ground, the scale origin on the scale plate is located on the optical axis of the camera, and the scale direction is aligned with the x-axis of the third coordinate system. c Axis parallel;

[0127] The z c Calculated by the following formula:

[0128]

[0129] Among them, z1 is the scale value corresponding to the intersection between the center of the target image and the straight line connecting the 100-meter stake sign in the target image and the scale of the scale plate, h is the height between the camera and the ground, and h1 is the height between the camera and the scale plate.

[0130] As a preferred embodiment, the target image is an image frame obtained from a video;

[0131] The step of performing coordinate transformation on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters includes:

[0132] Acquire at least one image frame containing the 100-meter stake sign adjacent to the target image as a reference image;

[0133] Inputting the target image into a trained depth recognition model to identify the first three-dimensional coordinates corresponding to the pixel coordinates of the key point;

[0134] Inputting the reference image into the depth recognition model, identifying the second three-dimensional coordinates corresponding to the pixel coordinates of the key points of the 100-meter pile sign in the reference image;

[0135] Based on the first three-dimensional coordinates and the second three-dimensional coordinates, the camera position deviation when the camera shoots the target image and the reference image is calculated, and the error between the camera position deviation and the longitude and latitude deviation when the camera shoots the target image and the reference image in the camera parameters is calculated. If the error is less than a first preset error threshold, the longitude and latitude coordinates corresponding to the hundred-meter pile sign are calculated according to the first three-dimensional coordinates and the second three-dimensional coordinates. If the error is not less than the first preset error threshold, the steps of establishing the first coordinate system, the second coordinate system and the third coordinate system are executed.

[0136] As a preferred embodiment, the step of the depth recognition model recognizing the first three-dimensional coordinate or the second three-dimensional coordinate includes:

[0137] translating each pixel in the image to be identified according to different translation lengths, and upsampling the multiple images obtained after the translation to obtain multiple feature maps corresponding to the image to be identified and a disparity map corresponding to each feature map; the image to be identified is the target image or the reference image, and the disparity map is used to represent the disparity between the pixel in the feature map and the pixel in the image to be identified;

[0138] Extracting pixel points of a specific area in each feature map, and forming pixel point pairs with the extracted adjacent pixel points; each pixel point in the image to be identified has a corresponding pixel point in the pixel points extracted from each feature map;

[0139] Selecting pixel points representing the same object according to the disparity map and accumulating them respectively, and adjusting the distances between the pixel points obtained after the accumulation based on the pixel pairs, so as to obtain a combined view corresponding to the image to be recognized;

[0140] The pixel points of the image to be identified are matched with the pixel points of the combined view, and based on the pixel difference between the matched pixel points, the three-dimensional coordinates corresponding to the pixel coordinates of the key points of the 100-meter pile image in the image to be identified are calculated.

[0141] As a preferred embodiment, if the error between the third coordinate obtained according to the first coordinate system, the second coordinate system and the third coordinate system and the first three-dimensional coordinate is greater than a second preset error threshold, the target image including the hundred-meter stake sign is re-photographed by a camera, and the hundred-meter stake sign is positioned based on the re-photographed target image.

[0142] The various embodiments of the present invention are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0143] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0144] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A positioning method for a 100-meter pile sign, characterized in that: include: Get the target image including the 100-meter stake sign captured by the camera; Input the target image into a trained image recognition model to identify the sign information on the 100-meter pile sign in the target image and the key point pixel coordinates corresponding to the 100-meter pile sign in the target image; The camera parameters of the camera are obtained, and based on the key point pixel coordinates and the camera parameters, the key point pixel coordinates are converted to obtain the longitude and latitude coordinates corresponding to the key point pixel coordinates, and the association between the longitude and latitude coordinates and the sign information is established and recorded.

2. The method according to claim 1, characterized in that The image recognition model includes a sign extraction module and a sign recognition module; The sign extraction module is used to extract the 100-meter pile sign image block corresponding to the 100-meter pile sign from the target image, determine the key point pixels in the 100-meter pile sign image block, and obtain the key point pixel coordinates corresponding to the 100-meter pile sign; The sign recognition module is used to recognize the image block of the 100-meter pile sign to obtain the sign information on the 100-meter pile sign.

3. The method according to claim 2, characterized in that The sign recognition module recognizes the 100-meter pile sign image block, including: Divide the 100-meter stake sign image block according to a preset partitioning rule to obtain a 100-meter area image block and a kilometer area image block; Determine the classification corresponding to the 100-meter area image block through the classification model in the sign recognition module, and obtain the 100-meter number in the 100-meter pile sign based on the classification corresponding to the 100-meter area image block; different classifications in the classification model correspond to different 100-meter numbers; The kilometer area image block is recognized by the character recognition submodule in the sign recognition module to obtain the kilometers in the hundred-meter pile sign; the sign information includes the hundred-meter number and the kilometer number in the hundred-meter pile sign.

4. The method according to claim 1, characterized in that: The step of performing coordinate transformation on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters includes: Establish a first coordinate system, a second coordinate system, and a third coordinate system; the first coordinate system takes the pixel point at the upper left corner of the target image as the origin, the width of the target image as the u-axis, and the height of the target image as the v-axis; the second coordinate system takes the center of the target image as the origin, the width of the target image as the x-axis, and the height of the target image as the y-axis; the third coordinate system takes the center of the target image as the origin, the axis parallel to the x-axis as the x-axis, and the c axis, an axis parallel to the y axis as y c The axis and the axis coinciding with the optical axis of the camera are z c axis; Obtaining first coordinates (u, v) of the key point pixel in the first coordinate system based on the key point pixel coordinates; According to the preset coordinate conversion formula and the camera parameters, the first coordinate (u, v) is converted into the second coordinate (x, y) of the key point pixel in the second coordinate system, and the second coordinate (x, y) is converted into the third coordinate (x, y) in the third coordinate system. c ,y c , z c ); Based on the acquired latitude and longitude coordinates corresponding to the target image, the third coordinate (x c ,y c , z c ) is converted into the longitude and latitude coordinates corresponding to the pixel coordinates of the key point.

5. The method according to claim 4, characterized in that The camera parameters include unit physical lengths of horizontal pixels and vertical pixels on the photosensitive plate of the target camera; The converting the first coordinate (u, v) into the second coordinate (x, y) of the key point pixel in the second coordinate system comprises: The first coordinate (u, v) is converted to the second coordinate (x, y) of the key point pixel in the second coordinate system according to the following formula: Wherein, (u0, v0) is the coordinate of the center of the target image in the first coordinate system, dx is the unit physical length of the horizontal pixel on the photosensitive plate, and dy is the unit physical length of the vertical pixel on the photosensitive plate.

6. The method according to claim 4, characterized in that The third coordinate (x c ,y c , z c ) c Determining based on a scale plate carrying a scale placed when the camera captures the target image; The second coordinate (x, y) is converted into a third coordinate (x c ,y c , z c ),include: The second coordinate (x, y) is converted into the third coordinate (x c ,y c , z c ), the formula is: Wherein, f is the focal length of the camera; The scale plate is located below the camera and parallel to the ground. The scale origin on the scale plate is located on the optical axis of the camera, and the scale direction is aligned with the x-axis of the third coordinate system. c Axis parallel; The z c Calculated by the following formula: Among them, z1 is the scale value corresponding to the intersection between the center of the target image and the straight line connecting the 100-meter stake sign in the target image and the scale of the scale plate, h is the height between the camera and the ground, and h1 is the height between the camera and the scale plate.

7. The method according to claim 4, characterized in that The target image is an image frame obtained from a video; The step of performing coordinate transformation on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters includes: Acquire at least one image frame containing the 100-meter stake sign adjacent to the target image as a reference image; Inputting the target image into a trained depth recognition model to identify the first three-dimensional coordinates corresponding to the pixel coordinates of the key point; Inputting the reference image into the depth recognition model, identifying the second three-dimensional coordinates corresponding to the pixel coordinates of the key points of the 100-meter pile sign in the reference image; Based on the first three-dimensional coordinates and the second three-dimensional coordinates, the camera position deviation when the camera shoots the target image and the reference image is calculated, and the error between the camera position deviation and the longitude and latitude deviation when the camera shoots the target image and the reference image in the camera parameters is calculated. If the error is less than a first preset error threshold, the longitude and latitude coordinates corresponding to the hundred-meter pile sign are calculated according to the first three-dimensional coordinates and the second three-dimensional coordinates. If the error is not less than the first preset error threshold, the steps of establishing the first coordinate system, the second coordinate system and the third coordinate system are executed.

8. The method according to claim 7, characterized in that The step of the depth recognition model recognizing the first three-dimensional coordinate or the second three-dimensional coordinate comprises: translating each pixel in the image to be identified according to different translation lengths, and upsampling the multiple images obtained after the translation to obtain multiple feature maps corresponding to the image to be identified and a disparity map corresponding to each feature map; the image to be identified is the target image or the reference image, and the disparity map is used to represent the disparity between the pixel in the feature map and the pixel in the image to be identified; Extracting pixel points of a specific area in each feature map, and forming pixel point pairs with the extracted adjacent pixel points; each pixel point in the image to be identified has a corresponding pixel point in the pixel points extracted from each feature map; Selecting pixel points representing the same object according to the disparity map and accumulating them respectively, and adjusting the distances between the pixel points obtained after the accumulation based on the pixel pairs, so as to obtain a combined view corresponding to the image to be recognized; The pixel points of the image to be identified are matched with the pixel points of the combined view, and based on the pixel difference between the matched pixel points, the three-dimensional coordinates corresponding to the pixel coordinates of the key points of the 100-meter pile image in the image to be identified are calculated.

9. The method according to claim 7, characterized in that: If the error between the third coordinate obtained according to the first coordinate system, the second coordinate system and the third coordinate system and the first three-dimensional coordinate is greater than a second preset error threshold, the target image including the hundred-meter stake sign is re-photographed by the camera, and the hundred-meter stake sign is positioned based on the re-photographed target image.

10. A positioning device for a 100-meter pile sign, characterized in that: include: An acquisition unit, used for acquiring a target image including a 100-meter stake sign taken by a camera; A recognition unit, used for inputting the target image into a trained image recognition model, identifying the sign information on the 100-meter pile sign in the target image and the key point pixel coordinates corresponding to the 100-meter pile sign in the target image; A positioning unit is used to obtain the camera parameters of the camera, perform coordinate transformation on the key point pixel coordinates based on the key point pixel coordinates and the camera parameters, obtain the longitude and latitude coordinates corresponding to the key point pixel coordinates, and establish and record the association between the longitude and latitude coordinates and the sign information.

Images