Indoor vehicle positioning method and device and storage medium

Abstract

The embodiment of the invention provides an indoor vehicle positioning method and device and a storage medium, and the method comprises the steps: obtaining a scene image corresponding to a target vehicle at a current moment when it is detected that the target vehicle drives into a positioning region of a target scene; edge point detection is carried out on the scene image to determine a target image area corresponding to a target calibration lattice, and the target calibration lattice is a calibration lattice assembled on the target vehicle. And extracting pixel coordinates corresponding to each pixel point in the target image area from the scene image. And according to a pre-calibrated internal reference matrix of the camera, the pixel coordinate corresponding to each pixel and the height from the target calibration grid to the ground, determining the camera coordinate of each pixel point projected to the camera coordinate system. And according to the pre-calibrated external parameter matrix of the camera, converting the camera coordinate corresponding to each pixel point into the world coordinate in the world coordinate system to determine the position information of the target vehicle in the positioning area, so that the vehicle position can be accurately positioned in the indoor scene.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to an indoor vehicle positioning method, device and storage medium. Background Art

[0002] In the motor vehicle driving test, mastering the content of the second subject assessment is an important learning content in driving training. However, the second subject training venues of some driving schools are placed indoors. Since the positioning signal of vehicles driving in indoor scenes is often weakened or interrupted, this will cause students and coaches to be unable to accurately obtain the vehicle's position and posture information during driving, and unable to give judgments during practice items, which brings troubles to driving practice.

[0003] In the traditional indoor vehicle positioning solution, an infrared emitting electronic tag is placed on the vehicle to be positioned, and multiple infrared sensors are installed in the indoor scene. Each infrared sensor can independently measure the distance information of the infrared signal emitted from the electronic tag, and determine the specific position of the vehicle to be positioned based on the distance information. However, this infrared positioning solution is not only too expensive, but also susceptible to environmental interference, resulting in reduced positioning precision and accuracy. Therefore, how to accurately locate the vehicle position in indoor scenes has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present invention provide an indoor vehicle positioning method, device and storage medium, which can accurately locate the position of the vehicle in an indoor scene.

[0005] In a first aspect, an embodiment of the present invention provides an indoor vehicle positioning method, the method comprising: when a target vehicle is detected to enter a positioning area of ​​a target scene, obtaining a scene image corresponding to the target vehicle at the current moment, the scene image being captured by a camera installed in the positioning area; performing edge point detection on the scene image according to feature information of a preset calibration grid to determine a target image area corresponding to a target calibration grid, the target calibration grid being a calibration grid mounted on the target vehicle; extracting pixel coordinates corresponding to each pixel point in the target image area from the scene image; determining the camera coordinates of each pixel point projected into a camera coordinate system according to a pre-calibrated intrinsic parameter matrix of the camera, the pixel coordinates corresponding to each pixel, and the height of the target calibration grid from the ground; converting the camera coordinates corresponding to each pixel point into world coordinates in a world coordinate system according to a pre-calibrated extrinsic parameter matrix of the camera; and determining the position information of the target vehicle within the positioning area according to the world coordinates corresponding to each pixel point.

[0006] Optionally, performing edge point detection on the scene image according to feature information of a preset calibration grid to determine a target image area corresponding to a target calibration grid includes: performing edge point detection on the scene image according to feature information of the calibration grid to determine image areas corresponding to multiple candidate calibration grids in the scene image; matching the image areas corresponding to the multiple candidate calibration grids with multiple reference calibration grids respectively to determine identifiers corresponding to the multiple candidate calibration grids; and screening out a target image area corresponding to the target calibration grid from the image areas corresponding to the multiple candidate calibration grids according to the identifiers corresponding to the multiple candidate calibration grids.

[0007] Optionally, performing edge point detection on the scene image according to feature information of the calibration grid to determine image areas corresponding to multiple candidate calibration grids in the scene image includes: performing edge point detection on the scene image according to feature information of the calibration grid to determine multiple candidate calibration grids contained in the scene image; extracting edge information corresponding to the multiple candidate calibration grids; and determining image areas of the multiple candidate calibration grids in the scene image using a minimum circumscribed rectangle or polygon fitting based on the edge information corresponding to the multiple candidate calibration grids.

[0008] Optionally, extracting pixel coordinates corresponding to each pixel point in the target image area from the scene image includes: performing corner point detection on the target image area to determine four corner points corresponding to the target calibration grid, wherein the corner points are feature points located at the four top corners of the target calibration grid; and reading the pixel coordinates corresponding to the four corner points.

[0009] Optionally, before determining the camera coordinates of each pixel point projected into the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera and the height of the target calibration grid from the ground, the method further includes: creating a checkerboard calibration plate for calibration, and laying the checkerboard calibration plate on the positioning area; using the camera to obtain a plurality of checkerboard images, the plurality of checkerboard images corresponding to different shooting angles; extracting the coordinates of corner points corresponding to each corner point in each checkerboard image; calculating the homography matrix according to the coordinates of corner points corresponding to each corner point in each checkerboard image; and determining the intrinsic parameter matrix corresponding to the camera according to the homography matrix.

[0010] Optionally, the method also includes: creating a calibration distribution map corresponding to the positioning area, and laying the calibration distribution map to the positioning area, the calibration distribution map including a plurality of calibration grids arranged at equal intervals; using the camera to shoot the calibration distribution map to obtain a plurality of calibration images, and the plurality of calibration images correspond to different shooting angles; extracting the corner point coordinates corresponding to each corner point in each calibration grid in the calibration image; determining the correspondence between the image coordinate system corresponding to the camera and the world coordinate system according to the corner point coordinates corresponding to each corner point in each calibration grid and the world coordinates corresponding to each calibration grid in the calibration distribution map; determining the extrinsic parameter matrix of the camera according to the intrinsic parameter matrix of the camera and the correspondence between the image coordinate system corresponding to the camera and the world coordinate system.

[0011] Optionally, each pixel point includes four corner points corresponding to the target calibration grid, and determining the camera coordinates of each pixel point projected into the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera, the pixel coordinates corresponding to each pixel and the height of the target calibration grid from the ground includes: performing cross multiplication processing on the pixel coordinates corresponding to the four corner points and the camera intrinsic parameter matrix to obtain first cross multiplication results corresponding to each of the four corner points in turn; determining the camera coordinates of the four corner points projected into the camera coordinate system according to the first cross multiplication results corresponding to each of the four corner points and the height of the target calibration grid from the ground.

[0012] Optionally, each pixel point includes four corner points corresponding to the target calibration grid, the extrinsic parameter matrix includes a rotation matrix and a translation matrix, and the camera coordinates corresponding to each pixel point are converted into world coordinates in a world coordinate system according to the pre-calibrated extrinsic parameter matrix of the camera, including: cross multiplying the camera coordinates corresponding to the four corner points with the rotation matrix to obtain second cross product results corresponding to each of the four corner points in turn; adding the second cross product results corresponding to each of the four corner points with the translation matrix to obtain the summed results corresponding to each of the four corner points; and determining the summed results corresponding to each of the four corner points as the world coordinates corresponding to the four corner points.

[0013] Optionally, each pixel point includes four corner points corresponding to the target calibration grid, and the method further includes: determining the world coordinates corresponding to the center point in the target calibration grid according to the world coordinates corresponding to the four corner points; determining the heading angle corresponding to the target vehicle according to the world coordinates corresponding to the center point and the coordinate origin of the world coordinate system where the target calibration grid is located.

[0014] Optionally, the method also includes: if it is detected that the target marking grid is located at the image edge of the scene image, then obtaining the position information corresponding to the target vehicle and the heading angle corresponding to the target vehicle at three consecutive historical moments before the current moment; judging whether there is a deviation in the position information corresponding to the target vehicle at the current moment based on the position information corresponding to the target vehicle at the three consecutive historical moments and the heading angle corresponding to the target vehicle; if there is a deviation, performing a weighted calculation on the position information corresponding to the target vehicle at the current moment and the heading angle corresponding to the target vehicle based on the position information corresponding to the target vehicle at three consecutive historical moments and the heading angle corresponding to the target vehicle, so as to obtain the corrected position information corresponding to the target vehicle at the current moment and the corrected heading angle corresponding to the target vehicle.

[0015] In a second aspect, an embodiment of the present invention provides an indoor vehicle positioning device, the device comprising:

[0016] An acquisition module, used for acquiring a scene image corresponding to the target vehicle at the current moment when a target vehicle is detected to enter a positioning area of ​​a target scene, wherein the scene image is captured by a camera installed in the positioning area;

[0017] A detection module, used for performing edge point detection on the scene image according to feature information of a preset calibration grid to determine a target image area corresponding to a target calibration grid, wherein the target calibration grid is a calibration grid mounted on the target vehicle;

[0018] An extraction module, used to extract pixel coordinates corresponding to each pixel point in the target image area from the scene image;

[0019] A first determination module is used to determine the camera coordinates of each pixel projected into the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera, the pixel coordinates corresponding to each pixel and the height of the target calibration grid from the ground;

[0020] A conversion module, used for converting the camera coordinates corresponding to each pixel point into world coordinates in a world coordinate system according to a pre-calibrated extrinsic parameter matrix of the camera;

[0021] The second determination module is used to determine the position information of the target vehicle in the positioning area according to the world coordinates corresponding to each pixel point.

[0022] Optionally, the detection module is specifically used to: perform edge point detection on the scene image according to feature information of the calibration grid to determine image areas corresponding to multiple candidate calibration grids in the scene image; respectively match the image areas corresponding to the multiple candidate calibration grids with multiple reference calibration grids to determine identifiers corresponding to the multiple candidate calibration grids; and filter out a target image area corresponding to a target calibration grid from the image areas corresponding to the multiple candidate calibration grids according to the identifiers corresponding to the multiple candidate calibration grids.

[0023] Optionally, the detection module is specifically used to: perform edge point detection on the scene image according to feature information of the calibration grid to determine multiple candidate calibration grids contained in the scene image; extract edge information corresponding to the multiple candidate calibration grids; and determine the image area of ​​the multiple candidate calibration grids in the scene image using a minimum circumscribed rectangle or polygon fitting according to the edge information corresponding to the multiple candidate calibration grids.

[0024] Optionally, the detection module is specifically used to: perform corner point detection on the target image area to determine four corner points corresponding to the target calibration grid, where the corner points are feature points located at the four vertices of the target calibration grid; and read the pixel coordinates corresponding to the four corner points.

[0025] Optionally, before determining the camera coordinates of each pixel point projected into the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera and the height of the target calibration grid from the ground, the device also includes a creation module, which is specifically used to: create a checkerboard calibration plate for calibration, and lay the checkerboard calibration plate on the positioning area; use the camera to capture a plurality of checkerboard images, and the plurality of checkerboard images correspond to different shooting angles; extract the coordinates of the corner points corresponding to each corner point in each checkerboard image; calculate the homography matrix according to the coordinates of the corner points corresponding to each corner point in each checkerboard image; and determine the intrinsic parameter matrix corresponding to the camera according to the homography matrix.

[0026] Optionally, the device also includes a calibration module, which is specifically used to: create a calibration distribution map corresponding to the positioning area, and lay the calibration distribution map to the positioning area, the calibration distribution map including a plurality of calibration grids arranged at equal intervals; use the camera to shoot the calibration distribution map to obtain a plurality of calibration images, and the plurality of calibration images correspond to different shooting angles; extract the corner point coordinates corresponding to each corner point in each calibration grid in the calibration image; determine the correspondence between the image coordinate system corresponding to the camera and the world coordinate system according to the corner point coordinates corresponding to each corner point in each calibration grid and the world coordinates corresponding to each calibration grid in the calibration distribution map; determine the external parameter matrix of the camera according to the intrinsic parameter matrix of the camera and the correspondence between the image coordinate system corresponding to the camera and the world coordinate system.

[0027] Optionally, each pixel point includes four corner points corresponding to the target calibration grid, and the first determination module is specifically used to: perform cross multiplication processing on the pixel coordinates corresponding to the four corner points and the camera intrinsic parameter matrix, and obtain first cross multiplication results corresponding to each of the four corner points in turn; determine the camera coordinates of the four corner points projected into the camera coordinate system according to the first cross multiplication results corresponding to each of the four corner points and the height of the target calibration grid from the ground.

[0028] Optionally, each pixel point includes four corner points corresponding to the target calibration grid, the external parameter matrix includes a rotation matrix and a translation matrix, and the conversion module is specifically used to: perform cross multiplication processing on the camera coordinates corresponding to the four corner points and the rotation matrix, and obtain the second cross product results corresponding to the four corner points in turn; perform summation processing on the second cross product results corresponding to the four corner points and the translation matrix, and obtain the summed results corresponding to the four corner points; and determine the summed results corresponding to the four corner points as the world coordinates corresponding to the four corner points.

[0029] Optionally, each pixel point includes four corner points corresponding to the target calibration grid, and the device also includes a third determination module, which is used to: determine the world coordinates corresponding to the center point in the target calibration grid according to the world coordinates corresponding to the four corner points; determine the heading angle corresponding to the target vehicle according to the world coordinates corresponding to the center point and the coordinate origin of the world coordinate system where the target calibration grid is located.

[0030] Optionally, the device also includes a correction module, which is specifically used to: if it is detected that the target marking grid is located at the image edge of the scene image, then obtain the position information corresponding to the target vehicle and the heading angle corresponding to the target vehicle at three consecutive historical moments before the current moment; based on the position information corresponding to the target vehicle at the three consecutive historical moments and the heading angle corresponding to the target vehicle, determine whether there is a deviation in the position information corresponding to the target vehicle at the current moment; if there is a deviation, perform weighted calculation on the position information corresponding to the target vehicle at the current moment and the heading angle corresponding to the target vehicle based on the position information corresponding to the target vehicle at three consecutive historical moments and the heading angle corresponding to the target vehicle, so as to obtain the corrected position information corresponding to the target vehicle at the current moment and the corrected heading angle corresponding to the target vehicle.

[0031] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the indoor vehicle positioning method as described in the first aspect.

[0032] In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the indoor vehicle positioning method as described in the first aspect.

[0033] In a fifth aspect, an embodiment of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it can implement the indoor vehicle positioning method as described in the first aspect.

[0034] The indoor vehicle positioning technical solution provided by the embodiment of the present invention, when the target vehicle is detected to enter the positioning area of ​​the target scene, the scene image corresponding to the target vehicle at the current moment is obtained. Among them, the scene image is obtained by shooting with a camera installed in the positioning area, and a target calibration grid is installed on the target vehicle to determine the position of the target vehicle in the scene image through the target calibration grid. Then, according to the feature information of the preset calibration grid, the edge point detection is performed on the scene image to determine the target image area corresponding to the target calibration grid. And from the scene image, the pixel coordinates corresponding to each pixel in the target image area are extracted. Then, according to the pre-calibrated camera intrinsic parameter matrix, the pixel coordinates corresponding to each pixel and the height of the target calibration grid from the ground, the camera coordinates of each pixel projected to the camera coordinate system are determined. Then, according to the pre-calibrated camera extrinsic parameter matrix, the camera coordinates corresponding to each pixel are converted into world coordinates in the world coordinate system. Finally, according to the world coordinates corresponding to each pixel, the position information of the target vehicle in the positioning area is determined.

[0035] In the above scheme, the pixel coordinates corresponding to each pixel point in the target calibration grid area are extracted from the scene image, and the camera coordinates of each pixel point projected into the camera coordinate system are determined according to the pre-calibrated intrinsic parameter matrix of the camera, the pixel coordinates corresponding to each pixel and the height of the target calibration grid from the ground. In addition, the camera coordinates corresponding to each pixel point are converted into world coordinates in the world coordinate system according to the pre-calibrated extrinsic parameter matrix of the camera. Then, the position information of the target vehicle in the positioning area is determined according to the world coordinates corresponding to each pixel point, so that the position information corresponding to the target vehicle can be quickly and accurately determined by using the target calibration grid installed on the target vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 A schematic diagram of the structure of an indoor vehicle positioning system provided by an embodiment of the present invention;

[0038] Figure 2 A flow chart of an indoor vehicle positioning method provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of an intrinsic parameter matrix for calibrating a camera provided by an embodiment of the present invention;

[0040] Figure 4A schematic diagram of an extrinsic parameter matrix for calibrating a camera provided by an embodiment of the present invention;

[0041] Figure 5 A flowchart of another indoor vehicle positioning method provided by an embodiment of the present invention;

[0042] Figure 6 A flowchart of another indoor vehicle positioning method provided by an embodiment of the present invention;

[0043] Figure 7 A schematic diagram of an application of correcting the position information and heading angle of a target vehicle provided by an embodiment of the present invention;

[0044] Figure 8 A schematic diagram of an application of an indoor vehicle positioning method provided by an embodiment of the present invention;

[0045] Fig. 9 A schematic diagram of the structure of an indoor vehicle positioning device provided by an embodiment of the present invention;

[0046] Fig.10 For Fig. 9 A schematic structural diagram of an electronic device corresponding to the indoor vehicle positioning device provided by the illustrated embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0049] When conducting motor vehicle driving training, some driving schools will place the training ground for Subject 2 indoors. However, the positioning signals (such as GPS signals, etc.) of vehicles traveling in indoor scenes often become weak or interrupted. This will cause students and coaches to be unable to accurately obtain the vehicle's position and posture information during driving, and unable to make judgments during practice items, which brings trouble to driving practice.

[0050] In order to locate the position information of the target vehicle in the indoor scene, Bluetooth base station, infrared, ultrasonic positioning and other methods can usually be used to determine the position information of the target vehicle. However, when using Bluetooth base station to determine the vehicle's position information, the vehicle positioning accuracy is low and the vehicle's position information cannot be accurately determined. The overall cost of using infrared or ultrasonic positioning is too high and is susceptible to environmental interference. Considering that the evaluation of Subject 2 requires extremely high precision requirements, the instructor needs to accurately understand the trainee's position in order to provide real-time guidance and evaluation and ensure the trainee's personal safety. At the same time, trainees also need to know their driving performance and make adjustments and improvements.

[0051] Therefore, in order to accurately locate the vehicle position in the driving scene in the driving training room during driving training, and the positioning cost is controllable. This embodiment provides an indoor vehicle positioning method, which can quickly and accurately identify the pixel coordinates corresponding to the pixel points of the target vehicle in the scene image by using the target calibration grid installed on the target vehicle, and project the pixel points and perform three-dimensional coordinate conversion processing to determine the world coordinates corresponding to the pixel points in the positioning area, and then accurately locate the position information of the target vehicle in the positioning area according to the world coordinates corresponding to the pixel points in the positioning area, which can not only improve the accuracy of the located target vehicle position information, but also reduce the positioning cost.

[0052] The technical solutions provided by various embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0053] Figure 1 FIG. 1 is a schematic diagram of the structure of an indoor vehicle positioning system provided by an embodiment of the present invention. Figure 1 As shown, the vehicle positioning system includes an indoor site 110 , a target vehicle 120 , and a server 130 .

[0054] The indoor venue 110 includes a positioning area 112 and a positioning area 113. In practical applications, corresponding positioning areas can be divided in the indoor venue 110 according to practical application requirements. In addition, multiple image acquisition devices 111 can be installed in the indoor venue 110 according to the requirements of positioning accuracy. The image acquisition devices 111 can be cameras, etc.

[0055] Among them, multiple image acquisition devices can be installed around the upper positions of the positioning areas in the indoor venue. During the installation process, it is necessary to fully consider the position, installation angle, and field of view of the camera, so that the image acquisition device 111 can capture the visual information of the target vehicle 120 in each positioning area in the indoor venue 110, and the field of view of each image acquisition device corresponds to a positioning area. Figure 1 The field of view of the image acquisition device 111 shown corresponds to the positioning area 113.

[0056] The image acquisition device 111 is used to acquire images of indoor venues at a fixed position and angle. When it is detected that the target vehicle 120 drives into the field of view of the image acquisition device 111, the image acquisition device 111 acquires a scene image including the target vehicle 120 and sends the acquired scene image to the server 130.

[0057] The server 130 is used to perform image processing on the scene image to determine the location information corresponding to the target vehicle 120 based on the scene image. The server 130 can be an independent physical server or a cloud server capable of cloud computing. The server 130 is connected in communication with the image acquisition device 111, can receive the scene image acquired by the image acquisition device 111, and determine the location information of the target vehicle 120 through the environmental information included in the scene image and the image information of the target vehicle 120.

[0058] In order to quickly identify the target vehicle 120 in the captured scene image and accurately locate the driving position and heading angle of the target vehicle 120 based on the scene image, at least one target calibration grid may be pre-installed on the target vehicle 120.

[0059] Specifically, when the server 130 detects that the target vehicle enters the positioning area of ​​the target scene, it controls the image acquisition device 111 to acquire the scene image corresponding to the target vehicle at the current moment, and receives the scene image corresponding to the target vehicle at the current moment sent by the image acquisition device 111, and performs edge point detection on the scene image according to the feature information of the preset calibration grid to determine the target image area corresponding to the target calibration grid, which is the calibration grid installed on the target vehicle. From the scene image, the pixel coordinates corresponding to each pixel point in the target image area are extracted; according to the pre-calibrated camera intrinsic parameter matrix, the pixel coordinates corresponding to each pixel and the height of the target calibration grid from the ground, the camera coordinates of each pixel point projected into the camera coordinate system are determined; according to the pre-calibrated camera extrinsic parameter matrix, the camera coordinates corresponding to each pixel point are converted into world coordinates in the world coordinate system; according to the world coordinates corresponding to each pixel point, the position information of the target vehicle in the positioning area is determined.

[0060] In addition, the server 130 can also communicate with the target vehicle 120. When the server 130 determines the location information (including the driving position and heading angle) of the target vehicle 120, it sends this information to the target vehicle 120, and the in-vehicle robot system further determines the vehicle driving information based on the sent driving position and heading angle. For example, in a driving training scenario, the in-vehicle robot system will conduct a simulated test based on the driving position and heading angle of the target vehicle 120 and the position of the driving school's storage line to determine whether it has violated the driving regulations.

[0061] The specific implementation process of indoor vehicle positioning is described in detail below in conjunction with the following embodiments.

[0062] Figure 2 Flow chart of an indoor vehicle positioning method provided by an embodiment of the present invention. Figure 2 As shown, the execution subject of the method may be an indoor vehicle positioning device. Specifically, the indoor vehicle positioning method includes the following steps:

[0063] 201. When a target vehicle is detected to enter a positioning area of ​​a target scene, a scene image corresponding to the target vehicle at the current moment is obtained, where the scene image is captured by a camera installed in the positioning area.

[0064] 202. Perform edge point detection on the scene image according to feature information of a preset calibration grid to determine a target image region corresponding to a target calibration grid, where the target calibration grid is a calibration grid mounted on a target vehicle.

[0065] 203. Extract pixel coordinates corresponding to each pixel point in the target image area from the scene image.

[0066] 204. Determine the camera coordinates of each pixel projected onto the camera coordinate system according to the pre-calibrated camera intrinsic parameter matrix, the pixel coordinates corresponding to each pixel, and the height of the target calibration grid from the ground.

[0067] 205. According to the pre-calibrated extrinsic parameter matrix of the camera, the camera coordinates corresponding to each pixel point are converted into world coordinates in the world coordinate system.

[0068] 206. Determine the position information of the target vehicle in the positioning area according to the world coordinates corresponding to each pixel point.

[0069] The indoor vehicle positioning solution provided by the embodiment of the present invention can be applied to accurately position each vehicle in any indoor scene, for example, to position target vehicles in indoor scenes such as indoor parking lots and indoor driving training venues, so as to accurately obtain the location information corresponding to the target vehicle.

[0070] When positioning a target vehicle in an indoor scene, a camera installed in a positioning area collects a scene image corresponding to the target vehicle, so as to locate the target vehicle based on the collected scene image containing the target vehicle.

[0071] Specifically, when it is detected that the target vehicle enters the positioning area of ​​the target scene, the scene image corresponding to the target vehicle at the current moment is obtained. The scene image is captured by a camera installed in the positioning area. A plurality of image acquisition devices are pre-installed at intervals. The image acquisition devices can be adjusted in the direction of the field of view according to the indoor light conditions and the network environment, so that the middle area of ​​the scene image acquired by the image acquisition device is aligned with the positioning area. When it is detected that the target vehicle enters the positioning area of ​​the target scene, that is, when the target vehicle enters the field of view of a certain image acquisition device, the image acquisition device acquires the scene image of the target vehicle.

[0072] In specific implementation, the target calibration grid can be pasted on the top and the tail of the target vehicle, or on other positions of the target vehicle, without limitation. Moreover, the target calibration grid pasted on each vehicle is different, because when multiple vehicles appear in the positioning area, the scene image captured by the camera includes not only the target calibration grid corresponding to the target vehicle but also the calibration grids corresponding to other vehicles. In order to quickly identify the target vehicle, the mark corresponding to the calibration grid can be used to quickly determine which vehicle is the target vehicle from multiple vehicles.

[0073] The target calibration grid may be an Aruco calibration code, a CharuCo calibration code, etc. Then the collected scene image includes the target calibration grid pasted on the target vehicle, so as to use the target calibration grid to quickly identify the position of the target vehicle in the scene image.

[0074] Specifically, edge point detection can be performed on the scene image according to the feature information of the preset calibration grid to determine the target image area corresponding to the target calibration grid. The collected scene image may include not only the target calibration grid corresponding to the target vehicle, but also the calibration grids corresponding to other vehicles. Then, when determining the target image area corresponding to the target calibration grid, the image area that may be the calibration grid in the scene image can be identified first, and then the image area corresponding to the target calibration grid can be screened out from the image area that may be the calibration grid.

[0075] Specifically, in an optional embodiment, edge point detection may be performed on the scene image based on feature information of the calibration grid to determine image regions corresponding to multiple candidate calibration grids in the scene image. Image regions corresponding to multiple candidate calibration grids are matched with multiple reference calibration grids to determine identifiers corresponding to multiple candidate calibration grids. Further, based on the identifiers corresponding to the multiple candidate calibration grids, a target image region corresponding to the target calibration grid is screened out from the image regions corresponding to the multiple candidate calibration grids.

[0076] Among them, the specific implementation process of performing edge point detection on the scene image according to the feature information of the calibration grid to determine the image areas corresponding to multiple candidate calibration grids in the scene image may include: performing edge point detection on the scene image according to the feature information of the calibration grid to determine the multiple candidate calibration grids contained in the scene image; extracting edge information corresponding to the multiple candidate calibration grids; and determining the image areas of the multiple candidate calibration grids in the scene image using a minimum circumscribed rectangle or polygon fitting based on the edge information corresponding to the multiple candidate calibration grids.

[0077] The calibration grid attached to each vehicle corresponds to a unique identifier, which may be an ID. The ID corresponding to the calibration grid attached to each vehicle is stored in a database, so that the vehicle information corresponding to each calibration grid in the scene image can be determined directly based on the calibration grid ID stored in the database.

[0078] That is to say, the scene image can be first detected for edge points based on the feature information of the preset calibration grid, and all image areas in the scene image that may correspond to the calibration grid can be identified. Then, the image areas corresponding to multiple candidate calibration grids can be further screened and identified based on the perimeter of the image area, the shapes that the rectangle corresponding to the preset calibration grid can change in the scene image, and the threshold value set for the side length of the preset calibration grid, so as to filter out similar image areas that may interfere with the calibration grid corresponding to the actual vehicle, and obtain the image areas corresponding to the multiple candidate calibration grids after screening. Then, the image areas corresponding to the multiple candidate calibration grids are extracted from the scene image, and the extracted image areas corresponding to the multiple candidate calibration grids are feature matched and compared with the standard reference calibration grid in the database to determine the image area corresponding to the target calibration grid.

[0079] After determining the target image region corresponding to the target calibration grid, the pixel coordinates corresponding to each pixel point in the target image region are extracted from the scene image, wherein the pixel coordinates refer to the position of a pixel point in the scene image, and the position here refers to the 2D position.

[0080] One possible way to extract the pixel coordinates corresponding to each pixel point in the target image area is to perform corner point detection on the target image area, determine the four corner points corresponding to the target calibration grid, where the corner points are feature points located at the four top corners of the target calibration grid, and read the pixel coordinates corresponding to the four corner points.

[0081] Among them, in order to speed up the image processing speed, when extracting the pixel coordinates corresponding to each pixel point in the target image area, only the pixel coordinates corresponding to the four corner points in the target image area can be extracted. This will not affect the final determined position information corresponding to the target vehicle, but also improve the positioning speed, so as to timely obtain the position information corresponding to the target vehicle at the current moment.

[0082] Next, each pixel point is projected to convert the two-dimensional pixel coordinates corresponding to each pixel point in the scene image into the three-dimensional world coordinates corresponding to the positioning area. The world coordinates refer to the coordinate information in the three-dimensional rectangular coordinate system established by taking a certain point in the positioning area in the real world as the origin and specifying the corresponding three directions corresponding to the X-axis, Y-axis, and Z-axis.

[0083] In specific implementation, if you want to locate the pixel position in the scene image to the position relative to the positioning area in the real world, you can pre-calibrate the positioning area and the camera's intrinsic parameter matrix and extrinsic parameter matrix to facilitate three-dimensional reconstruction of the positioning area, and then accurately determine the corresponding position information of the target vehicle in the positioning area. After the positioning area is determined in the indoor scene, the ground horizontal plane corresponding to the positioning area is used as the XY plane, the vertical upward is the positive direction of the Z axis, an origin is selected, and the coordinate axis directions corresponding to each coordinate axis are calibrated to establish a coordinate system corresponding to the positioning area.

[0084] The camera's intrinsic parameter matrix describes the camera's internal parameters, including focal length, principal point offset and other information. It is the key to converting image pixel coordinates to the camera coordinate system. The camera's intrinsic parameter matrix can be calibrated using a checkerboard. The checkerboard pattern is laid out in the positioning area. By continuously changing the position and angle of the checkerboard, multiple transformed checkerboard images are collected to generate the camera's intrinsic parameter matrix, which is used to correct distortion and establish the corresponding conversion relationship between image pixel coordinates and camera coordinates.

[0085] Among them, the camera's extrinsic matrix describes the camera's position and posture (rotation and translation) relative to the world coordinate system, and is used to convert 3D world coordinates to the camera coordinate system. You can use a chessboard or multiple fixed calibration grids arranged at equal intervals to lay out the positioning area, collect the pixel coordinates of the corner points corresponding to the multiple calibration grids through the camera, and the real world coordinates of the positioning area corresponding to the corner points, and use the pnp reprojection error optimization algorithm to estimate the camera's three-dimensional position and orientation relative to the positioning area, and then generate the camera's extrinsic matrix, so that the correspondence between the pixel points on the image and the world coordinates corresponding to the real positioning area is found.

[0086] After obtaining the pixel coordinates corresponding to each pixel point, the camera coordinates of each pixel point projected into the camera coordinate system are determined according to the pre-calibrated camera intrinsic parameter matrix, the pixel coordinates corresponding to each pixel point, and the height of the target calibration grid from the ground. And according to the pre-calibrated camera extrinsic parameter matrix, the camera coordinates corresponding to each pixel point are converted into world coordinates in the world coordinate system. Among them, the camera coordinate system refers to a three-dimensional rectangular coordinate system with the focal center of the camera as the origin and the optical axis as the Z axis.

[0087] When projecting each pixel point to the camera coordinate system, the height of the target calibration grid from the ground is combined to obtain the camera coordinate of each pixel point in the 3D camera coordinate system. Continuing with the above example, the target calibration grid is usually attached to the top or tail of the target vehicle, and the height of the target calibration grid from the ground can be determined according to the top height of the target vehicle or the tail height of the target vehicle.

[0088] According to the imaging principle, a pixel point on the target image area corresponds to a ray in the real world coordinates. However, if its depth information or height information is known, a unique point can be determined. Therefore, in an embodiment of the present invention, the three-dimensional coordinate information of the pixel point can be determined by directly measuring the roof height and the rear height of the target vehicle. The pixel points corresponding to the detected target calibration grids corresponding to the target vehicle are projected and calculated to obtain the world coordinates of these pixel points relative to the positioning area.

[0089] Finally, the position information of the target vehicle in the positioning area is determined based on the world coordinates corresponding to each pixel point. The actual position information corresponding to the target marking grid pasted on the target vehicle can be determined based on the world coordinates corresponding to each pixel point. The calculated actual position information corresponding to the target marking grid is equivalent to the calculated actual position information of the target vehicle.

[0090] The embodiment of the present invention extracts the pixel coordinates corresponding to each pixel point in the target calibration grid area from the scene image, and determines the camera coordinates of each pixel point projected into the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera, the pixel coordinates corresponding to each pixel and the height of the target calibration grid from the ground, and converts the camera coordinates corresponding to each pixel point into world coordinates in the world coordinate system according to the pre-calibrated extrinsic parameter matrix of the camera, and then determines the position information of the target vehicle in the positioning area according to the world coordinates corresponding to each pixel point, thereby realizing that the target calibration grid installed on the target vehicle can be used to quickly and accurately determine the position information corresponding to the target vehicle.

[0091] The above embodiment describes that after obtaining the pixel coordinates corresponding to each pixel of the target calibration grid, the camera's intrinsic parameter matrix and the camera's intrinsic parameter matrix are used to project each pixel point and perform coordinate conversion processing to obtain the world coordinates of each pixel point in the world coordinate system corresponding to the positioning area. In order to clearly understand the camera calibration process, this process is described in detail in conjunction with the following embodiments.

[0092] Among them, in an optional embodiment, the specific implementation method of calibrating the intrinsic parameter matrix of the camera can be: creating a checkerboard calibration plate for calibration, and laying the checkerboard calibration plate to the positioning area; using the camera to obtain multiple checkerboard images, the checkerboard calibration plate in the multiple checkerboard images is located in the positioning area at different positions and angles; extracting the corner point coordinates corresponding to each corner point in each checkerboard image; according to the corner point coordinates corresponding to each corner point in each checkerboard image, calculating the homography matrix; according to the homography matrix, determining the intrinsic parameter matrix corresponding to the camera.

[0093] In specific implementation, the camera can be used as follows Figure 3 The chessboard calibration method shown in the figure continuously changes the position and angle of the chessboard to take multiple chessboard images, that is, by taking multiple chessboard calibration plate images and extracting the feature points therein, the intrinsic parameter matrix of the camera can be solved by mathematical optimization method. The camera is calibrated through the intrinsic parameter matrix to determine the internal parameters of the camera to correct the distortion and establish the conversion relationship between the image pixel coordinates and the camera coordinates.

[0094] Among them, when calibrating the external parameter matrix of the camera, calibration can be performed using calibration grids that are arranged at equal intervals and laid in the positioning area. Specifically, in an optional embodiment, a calibration distribution map corresponding to the positioning area is created, and the calibration distribution map is laid in the positioning area, and the calibration distribution map includes multiple calibration grids arranged at equal intervals; the calibration distribution map is photographed with a camera to obtain multiple calibration images, and the multiple calibration images correspond to different shooting angles; the corner point coordinates corresponding to each corner point in each calibration grid in the calibration image are extracted; according to the corner point coordinates corresponding to each corner point in each calibration grid and the world coordinates corresponding to each calibration grid in the calibration distribution map, the corresponding relationship between the image coordinate system corresponding to the camera and the world coordinate system is determined; according to the corresponding relationship between the internal parameter matrix of the camera and the image coordinate system corresponding to the camera and the world coordinate system, the external parameter matrix of the camera is determined.

[0095] For example, Figure 4 As shown, the created calibration distribution map is laid out to the positioning area, and the calibration distribution map includes multiple calibration grids, each of which corresponds to an area of ​​the positioning area, where point A is the coordinate origin of the positioning area, and point B is a point in the positive direction of the X-axis of the world coordinate system corresponding to the positioning area. The calibration distribution map is photographed by a camera to identify the pixel coordinates corresponding to the four corner points of each calibration grid, and to determine the real coordinates corresponding to the four corner points in the positioning area. The pnp reprojection error optimization algorithm is used to estimate the three-dimensional positioning and orientation of the camera relative to the positioning area, and the corresponding relationship between the points on the image and the coordinates of the real positioning area is found, and then the external parameter matrix of the camera can be generated.

[0096] After the camera's intrinsic and extrinsic matrix is ​​calibrated, the camera coordinates of each pixel projected into the camera coordinate system are determined based on the pre-calibrated camera's intrinsic matrix, the pixel coordinates corresponding to each pixel, and the height of the target calibration grid from the ground. The four corner points of the target calibration grid can be used to locate the position information of the target vehicle in the positioning area.

[0097] Specifically, in an optional embodiment, the specific implementation method of determining the camera coordinates of each pixel point projected into the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera, the pixel coordinates corresponding to each pixel and the height of the target calibration grid from the ground can be: cross multiplying the pixel coordinates corresponding to the four corner points of the target calibration grid and the intrinsic parameter matrix of the camera, and obtaining the first cross product results corresponding to each of the four corner points in turn; determining the camera coordinates of the four corner points projected into the camera coordinate system according to the first cross product results corresponding to each of the four corner points and the height of the target calibration grid from the ground.

[0098] Then, according to the pre-calibrated extrinsic matrix of the camera, the camera coordinates corresponding to the four corner points are converted into world coordinates in the world coordinate system. The extrinsic matrix includes a rotation matrix and a translation matrix. Then in an optional embodiment, the conversion process can be: cross-product the camera coordinates corresponding to the four corner points with the rotation matrix to obtain the second cross-product results corresponding to the four corner points respectively; sum the second cross-product results corresponding to the four corner points with the translation matrix to obtain the summed results corresponding to the four corner points; and determine the summed results corresponding to the four corner points as the world coordinates corresponding to the four corner points.

[0099] By determining the world coordinates of the four corner points of the target calibration grid in the world coordinate system corresponding to the positioning area, the real position information of the target calibration grid in the positioning area can be determined, and then the real position information of the target vehicle in the positioning area can be determined based on the real position information of the target calibration grid in the positioning area.

[0100] In practical applications, in addition to determining the position information corresponding to the positioning area of ​​the target vehicle in the indoor scene, it is also necessary to determine the heading angle corresponding to the target vehicle at the current moment, so as to further guide the driver in the target vehicle to determine the next driving operation based on the position information and heading angle corresponding to the target vehicle at the current moment.

[0101] Combine the following Figure 5 The specific implementation process of determining the heading angle corresponding to the target vehicle at the current moment is illustrated by an example.

[0102] Figure 5 A flowchart of another indoor vehicle positioning method provided by an embodiment of the present invention; Figure 5As shown, based on the above embodiments, the method may further include the following steps:

[0103] 501. Determine the world coordinates corresponding to the center point in the target calibration grid according to the world coordinates corresponding to the four corner points.

[0104] 502. Determine the heading angle corresponding to the target vehicle according to the world coordinates corresponding to the center point and the coordinate origin of the world coordinate system where the target calibration grid is located.

[0105] The heading angle corresponding to the target vehicle at the current moment can be determined by determining the orientation of the four corner points of the target calibration grid relative to the coordinate axis of the positioning area. In specific implementation, the world coordinates corresponding to the center point in the target calibration draft can be determined based on the world coordinates corresponding to the four corner points of the target calibration grid, and the heading angle corresponding to the target vehicle can be determined based on the world coordinates corresponding to the center point and the coordinate origin of the world coordinate system where the target calibration grid is located.

[0106] In practical applications, at least two target calibration grids are usually installed on the target vehicle. Assuming that the target vehicle is equipped with two target calibration grids, the captured scene image may be divided into two cases, the first case in which the captured scene image contains two target calibration grids, and the second case in which the captured scene image contains only one target calibration grid. In an optional embodiment, in order to more accurately determine the heading angle corresponding to the target vehicle, when determining the heading angle, different methods can be used to determine the heading angle according to the number of target calibration grids included in the captured scene image.

[0107] For example, when a target calibration grid is detected, there are 4 corner points in total. Without considering the vertical offset of the target vehicle, the world coordinates corresponding to the center point of the target calibration grid can be obtained in the horizontal direction according to the 4 corner points of the detected target calibration grid, and the world coordinates corresponding to the upper left corner point of the target calibration grid are taken. The two coordinates can form a vector, and the angle between this vector and the basic vector of the origin of the positioning area coordinates is calculated. This angle is the direction of the vehicle head, that is, any coordinate on the vehicle body relative to the center point can be rotated by this angle to obtain the relative coordinates in this positioning area, and the center point coordinates are used as the vehicle position. When the front and rear target calibration grids installed on the target vehicle are detected, the angles between the center coordinates of the front and rear target calibration grids and the origin of the positioning area coordinates are calculated to determine the heading angle of the target vehicle, and the center point of the previous target calibration grid can be used as the target vehicle position.

[0108] In addition, in actual applications, if an indoor venue includes multiple positioning areas, when creating a coordinate system corresponding to the positioning area, the same coordinate system can be used for the multiple positioning areas, or a fixed standard coordinate system can be created in the indoor venue, and each positioning area uses a different coordinate system. After determining the position coordinates and heading angle of each vehicle in the positioning area coordinate system, they can be rotated and translated to convert them into the standard coordinate system corresponding to the indoor venue, and the position information of the target vehicle relative to the indoor venue is used as the position information corresponding to the target vehicle at the current moment, and the heading angle of the target vehicle relative to the indoor venue is used as the heading angle corresponding to the target vehicle at the current moment. The implementation of the present invention does not limit this and can be set according to actual application requirements.

[0109] According to an embodiment of the present invention, the world coordinates corresponding to the center point in the target calibration grid are determined based on the world coordinates corresponding to the four corner points of the target calibration grid, and the heading angle corresponding to the target vehicle is determined based on the world coordinates corresponding to the center point and the coordinate origin of the world coordinate system in which the target calibration grid is located. The orientation corresponding to the target vehicle at the current moment can be accurately determined.

[0110] The above embodiment describes the determination of the position information and heading angle corresponding to the target vehicle based on the collected pixel coordinates of the target marking grid. However, in the specific implementation, unstable situations may occur. For example, when the target vehicle is at the edge of the positioning area, the recognition is unstable, which may cause the vehicle body to shake and a small range of position deviation. In order to more accurately identify the target marking grid and further improve the accuracy of the located target vehicle position information, after determining the position information and heading angle corresponding to the target vehicle, the position information and heading angle may be further corrected.

[0111] Figure 6 A flowchart of another indoor vehicle positioning method provided by an embodiment of the present invention; Figure 6 As shown, based on the above embodiments, the method may further include the following steps:

[0112] 601. If it is detected that the target marking grid is located at the edge of the scene image, the position information corresponding to the target vehicle at three consecutive historical moments before the current moment and the heading angle corresponding to the target vehicle are obtained.

[0113] 602. Determine whether there is a deviation in the position information corresponding to the target vehicle at the current moment based on the position information corresponding to the target vehicle at three consecutive historical moments and the heading angle corresponding to the target vehicle.

[0114] 603. If there is a deviation, a weighted calculation is performed on the position information corresponding to the target vehicle at the current moment and the heading angle corresponding to the target vehicle according to the position information corresponding to the target vehicle at three consecutive historical moments, so as to obtain the corrected position information corresponding to the target vehicle at the current moment and the corrected heading angle corresponding to the target vehicle.

[0115] In order to make the position information and heading angle corresponding to the located target vehicle more accurate, after determining the position information and heading angle corresponding to the target vehicle at each moment, the position information and heading angle corresponding to the target vehicle at three consecutive historical moments can also be tracked.

[0116] In specific implementation, when the position information corresponding to the target vehicle at each moment is determined, the position information, heading angle, vehicle speed, number of target mark grids detected, and other information corresponding to the target vehicle at each moment can also be stored in the database. That is, the data at each moment is recorded during the driving process of the target vehicle, and the position information and heading angle corresponding to the target vehicle located at the current moment are detected based on the historical data corresponding to the three consecutive historical moments corresponding to the current moment to see if there is a deviation. In addition, when the target vehicle leaves the positioning area, that is, the target mark grid corresponding to the target vehicle is not detected for two consecutive moments, it is determined that the target vehicle has left the positioning area, and the data corresponding to the target vehicle is no longer tracked and recorded.

[0117] However, if the position information and heading angle corresponding to the target vehicle determined at each moment are tracked and corrected, it will affect the positioning efficiency of the target vehicle and the next driving operation. In addition, it was found during the implementation process that when both target marking grids of the target vehicle are identified, the stability of the current vehicle is better. The main jitter is sent when the number or type of target marking grids identified at the previous moment and this moment are different. For example, only the target marking grid in front of the target vehicle was detected at the previous moment, and only the target marking grid behind the target vehicle was detected at the current moment. Therefore, in an embodiment of the present invention, only in the case where the target marking grid is detected to be located at the image edge of the scene image, or when it is detected that the angle of the target vehicle changes greatly compared with the historical data corresponding to the previous moment when two marking grids or a single marking grid are detected, the position information and heading angle corresponding to the determined target vehicle can be corrected to avoid positioning deviation caused by vehicle jitter.

[0118] Then, in the specific implementation, when it is judged that there is a deviation in the position information corresponding to the target vehicle at the current moment based on the position information corresponding to the target vehicle at three consecutive historical moments and the heading angle corresponding to the target vehicle, the position information corresponding to the target vehicle at the current moment and the heading angle corresponding to the target vehicle can be weightedly calculated based on the position information corresponding to the target vehicle at three consecutive historical moments and the heading angle corresponding to the target vehicle to obtain the corrected position information corresponding to the target vehicle at the current moment and the corrected heading angle corresponding to the target vehicle.

[0119] In addition, in an optional embodiment, in order to quickly complete the position correction, when it is detected that the position information and heading angle of the target vehicle at the current moment have undergone significant changes compared with the position information and heading angle of the target vehicle at the previous moment, the heading angle of the current target vehicle can be first controlled within the change threshold range, and then weighted processing is performed with the historical moment information to obtain the corrected position information and the corrected heading angle.

[0120] When implementing it, Figure 7 As shown, the position information and heading angle information corresponding to the target vehicle at the current moment can be determined first. Then, the number of calibration grids corresponding to the target vehicle collected at the current moment is determined. If it is detected that a target calibration grid corresponding to the target vehicle is collected, it is determined whether the position and heading angle of the target vehicle at the current moment have changed significantly from the position and heading angle of the target vehicle determined at the previous moment. If there is a significant change, the heading angle of the current target vehicle is controlled within the change threshold range, and then weighted filtering is performed with the historical moment information to obtain the corrected position information and the corrected heading angle. Then, the corrected position information and the modified heading angle are stored in the database, and the corrected position information corresponding to the target vehicle at the current moment and the corrected heading angle corresponding to the target vehicle are sent to the target vehicle.

[0121] The embodiment of the present invention detects whether the target marking grid is located at the image edge of the scene image, and when it is detected that the target marking grid is located at the image edge of the scene image, it can judge whether there is a deviation in the position information corresponding to the target vehicle at the current moment based on the position information corresponding to the target vehicle at three consecutive historical moments and the heading angle corresponding to the target vehicle, and correct the position information and heading angle of the target vehicle at the current moment based on the position information corresponding to the target vehicle at three consecutive historical moments and the heading angle corresponding to the target vehicle, so as to ensure the accuracy of the driving position and orientation corresponding to the located target vehicle.

[0122] In order to facilitate understanding of the above scheme, examples are given in combination with specific application scenarios. For example, in an indoor driving training venue, a trainee needs to accurately know the distance between the specific position of the project and the storage line when practicing driving. Therefore, the indoor vehicle positioning method provided by the embodiment of the present invention can be used to locate the distance between the specific position of the project and the storage line when the trainee is practicing driving.

[0123] When applying specific Figure 8 As shown, this application embodiment provides a method for accurately locating the position and heading angle of a vehicle in an indoor venue by pasting a calibration grid on the vehicle. Specifically, the method may include the following steps:

[0124] Stick the marking grids with different IDs to the top and rear of the vehicle in the order of the marking grid IDs, for example, Figure 8 There are two positions A and B in the middle. When the trainee drives the target vehicle to the positioning area, that is, the target vehicle enters the field of view of the image acquisition device, the image acquisition device acquires the scene image containing the target vehicle at the current moment. Among them, in the indoor subject two training venue of the driving school, a plurality of image acquisition devices with intervals are pre-installed. At the same time, the image acquisition device can be set according to the indoor lighting conditions and the surrounding environment, and the field of view can be adjusted so that the middle area of ​​the image captured by the image acquisition device is aligned with the subject two project venue. When the target vehicle enters the field of view of a certain image acquisition device, the image acquisition device can acquire the image of the target vehicle.

[0125] When the target vehicle enters the positioning area, depending on the position of the vehicle, it is possible that one or two calibration grids will appear in the scene image at the same time. When multiple target vehicles appear in the positioning area at the same time, more calibration grids will appear in the collected scene image. Multiple target vehicles can be located simultaneously through the collected scene images.

[0126] According to the feature information of the preset calibration grid, edge point detection is performed on the scene image to determine the target image area corresponding to the target calibration grid, which is the calibration grid installed on the target vehicle. From the scene image, the pixel coordinates corresponding to the four corner points in the target image area are extracted.

[0127] Specifically, according to the image binarization threshold processing and corner point recognition algorithm, multiple image areas that may be calibration grids are obtained in the collected scene image. Then, according to the several shapes that the calibration grid rectangle may transform in the image and the calibration grid side length, the threshold is set to further screen the calibration grid image area, so as to filter out the image area similar to the calibration grid that may interfere with the calibration grid corresponding to the actual vehicle, and finally these image areas are feature matched and compared with the standard calibration grid in the library to determine the final extraction of all the image areas corresponding to the candidate calibration grids and their corresponding IDs and the pixel coordinates of the four corner points of the candidate calibration grids. Then, the candidate calibration grid ID is compared with the corresponding calibration grid ID bound to the vehicle stored in the database. If the vehicle corresponding to this calibration grid is found, it means that the target vehicle has been found.

[0128] Next, the camera coordinates of the four corner points projected into the camera coordinate system are determined in turn according to the pre-calibrated camera intrinsic parameter matrix, the pixel coordinates corresponding to the four corner points, and the height of the target calibration grid from the ground. According to the pre-calibrated camera extrinsic parameter matrix, the camera coordinates corresponding to the four corner points are converted into world coordinates in the world coordinate system.

[0129] Then, the position information of the target vehicle in the positioning area is determined according to the world coordinates corresponding to the four corner points. The world coordinates corresponding to the center point in the target calibration grid are determined according to the world coordinates corresponding to the center point and the coordinate origin of the world coordinate system where the target calibration grid is located, and the heading angle corresponding to the target vehicle is determined according to the world coordinates corresponding to the center point and the coordinate origin of the world coordinate system where the target calibration grid is located.

[0130] When arranging the image acquisition area and calibrating its external parameters, the origin of the positioning area coordinate system and the direction of the coordinate axis are recorded. Therefore, it is only necessary to perform a translation and rotation calculation on the positioning area coordinates and the angle to obtain the final vehicle position and heading angle in the driving school indoor scene.

[0131] Finally, a simulation test is conducted based on the vehicle position information, heading angle information, and the indoor site mapping and parking space information of the driving school. Figure 8 As shown in the figure, after knowing the real-time location information and heading angle information of the target vehicle, the target vehicle can be rendered in the indoor scene of the driving school based on the length, width and other information of the vehicle model. The fixed coordinates of the storage lines of all projects in the driving school are obtained from the surveying and mapping data, so they can also be displayed on the map. When the target vehicle moves in the positioning area, the server sends its location information in real time, and the vehicle's own mobile device renders the map for students and coaches to view. When the target vehicle enters area D, the simulation test of subject 2 is triggered. The trajectory of the target vehicle moving in the warehouse will be judged and deducted.

[0132] The detailed implementation process can be referred to the above related description, which will not be repeated here.

[0133] The indoor vehicle positioning device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0134] Fig. 9 A schematic diagram of the structure of an indoor vehicle positioning device provided by an embodiment of the present invention is shown in FIG. Fig. 9 As shown, the device includes: an acquisition module 11, a detection module 12, an extraction module 13, a first determination module 14, a conversion module 15, and a second determination module 16.

[0135] Optionally, the detection module 12 is specifically used to: perform edge point detection on the scene image according to feature information of the calibration grid to determine image areas corresponding to multiple candidate calibration grids in the scene image; respectively match the image areas corresponding to the multiple candidate calibration grids with multiple reference calibration grids to determine identifiers corresponding to the multiple candidate calibration grids; and filter out a target image area corresponding to a target calibration grid from the image areas corresponding to the multiple candidate calibration grids according to the identifiers corresponding to the multiple candidate calibration grids.

[0136] Optionally, the detection module 12 is specifically used to: perform edge point detection on the scene image according to feature information of the calibration grid to determine multiple candidate calibration grids contained in the scene image; extract edge information corresponding to the multiple candidate calibration grids; and determine the image area of ​​the multiple candidate calibration grids in the scene image using a minimum circumscribed rectangle or polygon fitting according to the edge information corresponding to the multiple candidate calibration grids.

[0137] Optionally, the detection module 12 is specifically used to: perform corner point detection on the target image area to determine four corner points corresponding to the target calibration grid, where the corner points are feature points located at the four vertices of the target calibration grid; and read the pixel coordinates corresponding to the four corner points.

[0138] Optionally, before determining the camera coordinates of each pixel point projected into the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera and the height of the target calibration grid from the ground, the device also includes a creation module, which is specifically used to: create a checkerboard calibration plate for calibration, and lay the checkerboard calibration plate on the positioning area; use the camera to capture a plurality of checkerboard images, and the plurality of checkerboard images correspond to different shooting angles; extract the coordinates of the corner points corresponding to each corner point in each checkerboard image; calculate the homography matrix according to the coordinates of the corner points corresponding to each corner point in each checkerboard image; and determine the intrinsic parameter matrix corresponding to the camera according to the homography matrix.

[0139] Optionally, the device also includes a calibration module, which is specifically used to: create a calibration distribution map corresponding to the positioning area, and lay the calibration distribution map to the positioning area, the calibration distribution map including a plurality of calibration grids arranged at equal intervals; use the camera to shoot the calibration distribution map to obtain a plurality of calibration images, and the plurality of calibration images correspond to different shooting angles; extract the corner point coordinates corresponding to each corner point in each calibration grid in the calibration image; determine the correspondence between the image coordinate system corresponding to the camera and the world coordinate system according to the corner point coordinates corresponding to each corner point in each calibration grid and the world coordinates corresponding to each calibration grid in the calibration distribution map; determine the external parameter matrix of the camera according to the intrinsic parameter matrix of the camera and the correspondence between the image coordinate system corresponding to the camera and the world coordinate system.

[0140] Optionally, each pixel point includes four corner points corresponding to the target calibration grid, and the first determination module 14 is specifically used to: perform cross multiplication processing on the pixel coordinates corresponding to the four corner points and the camera intrinsic parameter matrix, and obtain first cross multiplication results corresponding to each of the four corner points in turn; determine the camera coordinates of the four corner points projected into the camera coordinate system according to the first cross multiplication results corresponding to each of the four corner points and the height of the target calibration grid from the ground.

[0141] Optionally, each pixel point includes four corner points corresponding to the target calibration grid, the external parameter matrix includes a rotation matrix and a translation matrix, and the conversion module 15 is specifically used to: perform cross multiplication processing on the camera coordinates corresponding to the four corner points and the rotation matrix, and obtain the second cross product results corresponding to the four corner points in turn; add the second cross product results corresponding to the four corner points and the translation matrix to obtain the summed results corresponding to the four corner points; and determine the summed results corresponding to the four corner points as the world coordinates corresponding to the four corner points.

[0142] Optionally, each pixel point includes four corner points corresponding to the target calibration grid, and the device also includes a third determination module, which is used to: determine the world coordinates corresponding to the center point in the target calibration grid according to the world coordinates corresponding to the four corner points; determine the heading angle corresponding to the target vehicle according to the world coordinates corresponding to the center point and the coordinate origin of the world coordinate system where the target calibration grid is located.

[0143] Optionally, the device also includes a correction module, which is specifically used to: if it is detected that the target marking grid is located at the image edge of the scene image, then obtain the position information corresponding to the target vehicle and the heading angle corresponding to the target vehicle at three consecutive historical moments before the current moment; based on the position information corresponding to the target vehicle at the three consecutive historical moments and the heading angle corresponding to the target vehicle, determine whether there is a deviation in the position information corresponding to the target vehicle at the current moment; if there is a deviation, perform weighted calculation on the position information corresponding to the target vehicle at the current moment and the heading angle corresponding to the target vehicle based on the position information corresponding to the target vehicle at three consecutive historical moments and the heading angle corresponding to the target vehicle, so as to obtain the corrected position information corresponding to the target vehicle at the current moment and the corrected heading angle corresponding to the target vehicle.

[0144] Fig. 9 The device shown can perform the aforementioned Figures 1 to 8 The indoor vehicle positioning method provided in the illustrated embodiment, the detailed execution process and technical effects are described in the aforementioned embodiment, which will not be repeated here.

[0145] In one possible design, the above Fig. 9 The structure of the indoor vehicle positioning device shown in the figure can be implemented as an electronic device. The electronic device provided in this embodiment is as follows: Fig.10 As shown, the electronic device may include: a processor 21 and a memory 22. The memory 22 stores executable code. When the executable code is executed by the processor 21, the processor 21 can at least implement the above-mentioned Figures 1 to 9 The indoor vehicle positioning method provided in the illustrated embodiment.

[0146] In the embodiment of the present invention, the memory 22 is used to store computer programs and can be configured to store various other data to support operations on the device where it is located. The processor 21 can execute the computer program stored in the memory to implement the corresponding control logic.

[0147] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, wherein an executable code is stored on the non-transitory machine-readable storage medium. When the executable code is executed by a processor of an electronic device, the processor can at least implement the above-mentioned Figures 1 to 8 The indoor vehicle positioning method provided in the illustrated embodiment.

[0148] In addition, an embodiment of the present invention provides a computer program product, wherein an executable code is stored on the computer program product, and when the executable code is executed by a processor of an electronic device, the processor can at least implement the above-mentioned Figures 1 to 8The indoor vehicle positioning method provided in the illustrated embodiment.

[0149] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course, it can also be implemented by a combination of hardware and software. Based on such an understanding, the above technical solution can essentially or in other words, contribute to the prior art in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for indoor vehicle positioning, characterized in that: include: When a target vehicle is detected to enter a positioning area of ​​a target scene, a scene image corresponding to the target vehicle at the current moment is obtained, where the scene image is captured by a camera installed in the positioning area; According to feature information of a preset calibration grid, edge point detection is performed on the scene image to determine a target image area corresponding to a target calibration grid, wherein the target calibration grid is a calibration grid mounted on the target vehicle; Extracting pixel coordinates corresponding to each pixel point in the target image area from the scene image; Determine the camera coordinates of each pixel projected into the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera, the pixel coordinates corresponding to each pixel, and the height of the target calibration grid from the ground; According to the pre-calibrated extrinsic parameter matrix of the camera, the camera coordinates corresponding to each pixel point are converted into world coordinates in a world coordinate system; The position information of the target vehicle within the positioning area is determined according to the world coordinates corresponding to the pixel points.

2. The method according to claim 1, characterized in that The performing edge point detection on the scene image according to the feature information of the preset calibration grid to determine the target image area corresponding to the target calibration grid includes: Performing edge point detection on the scene image according to feature information of the calibration grid to determine image regions corresponding to a plurality of candidate calibration grids in the scene image; Matching the image areas corresponding to the multiple candidate calibration grids with multiple reference calibration grids respectively to determine the identifiers corresponding to the multiple candidate calibration grids; According to the identifiers corresponding to the multiple candidate marking grids, a target image region corresponding to the target marking grid is screened out from the image regions corresponding to the multiple candidate marking grids.

3. The method according to claim 2, characterized in that The performing edge point detection on the scene image according to the feature information of the calibration grid to determine the image areas corresponding to the multiple candidate calibration grids in the scene image includes: Performing edge point detection on the scene image according to feature information of the calibration grid to determine a plurality of candidate calibration grids contained in the scene image; Extracting edge information corresponding to the plurality of candidate calibration grids; According to the edge information corresponding to the plurality of candidate calibration grids, the image regions of the plurality of candidate calibration grids in the scene image are determined by using a minimum circumscribed rectangle or polygon fitting.

4. The method according to claim 1, characterized in that: The step of extracting pixel coordinates corresponding to each pixel point in the target image area from the scene image includes: Performing corner point detection on the target image area to determine four corner points corresponding to the target calibration grid, wherein the corner points are feature points located at the four vertices of the target calibration grid; Read the pixel coordinates corresponding to the four corner points.

5. The method according to claim 1, characterized in that: Before determining the camera coordinates of each pixel point projected onto the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera and the height of the target calibration grid from the ground, the method further includes: Creating a checkerboard calibration plate for calibration, and laying the checkerboard calibration plate on the positioning area; Using the camera to obtain a plurality of checkerboard images, the checkerboard calibration plates in the plurality of checkerboard images are located at different positions in the positioning area and at different angles in the positioning area; Extract the corner point coordinates corresponding to each corner point in each chessboard image; Calculating a homography matrix according to the corner point coordinates corresponding to each corner point in each chessboard image; According to the homography matrix, an intrinsic parameter matrix corresponding to the camera is determined.

6. The method according to claim 5, characterized in that The method further comprises: Creating a calibration distribution map corresponding to the positioning area, and laying the calibration distribution map to the positioning area, wherein the calibration distribution map includes a plurality of calibration grids arranged at equal intervals; Using the camera to shoot the calibration distribution map, a plurality of calibration images are obtained, and the plurality of calibration images correspond to different shooting angles; Extracting the corner point coordinates corresponding to each corner point in each calibration grid in the calibration image; Determine a correspondence between an image coordinate system corresponding to the camera and a world coordinate system according to the corner point coordinates corresponding to each corner point in each calibration grid and the world coordinates corresponding to each calibration grid in the calibration distribution map; The extrinsic parameter matrix of the camera is determined according to the intrinsic parameter matrix of the camera and the corresponding relationship between the image coordinate system corresponding to the camera and the world coordinate system.

7. The method according to claim 1, characterized in that Each pixel point includes four corner points corresponding to the target calibration grid, and determining the camera coordinates of each pixel point projected into the camera coordinate system according to the pre-calibrated intrinsic parameter matrix of the camera, the pixel coordinates corresponding to each pixel, and the height of the target calibration grid from the ground, includes: Performing cross multiplication processing on the pixel coordinates corresponding to the four corner points and the camera intrinsic parameter matrix, and obtaining first cross multiplication results corresponding to the four corner points respectively in turn; The camera coordinates of the four corner points projected onto the camera coordinate system are determined according to the first cross product results corresponding to the four corner points respectively and the height of the target calibration grid from the ground.

8. The method according to claim 1, characterized in that Each pixel point includes four corner points corresponding to the target calibration grid, the extrinsic parameter matrix includes a rotation matrix and a translation matrix, and the camera coordinates corresponding to each pixel point are converted into world coordinates in a world coordinate system according to the pre-calibrated extrinsic parameter matrix of the camera, including: Performing cross product processing on the camera coordinates corresponding to the four corner points and the rotation matrix, and obtaining second cross product results corresponding to the four corner points respectively in turn; Adding the second cross product results corresponding to the four corner points to the translation matrix to obtain the summed results corresponding to the four corner points; The summation results corresponding to the four corner points are determined as the world coordinates corresponding to the four corner points.

9. The method according to claim 1, characterized in that: Each pixel point includes four corner points corresponding to the target calibration grid, and the method further includes: Determine the world coordinates corresponding to the center point in the target calibration grid according to the world coordinates corresponding to the four corner points; The heading angle corresponding to the target vehicle is determined according to the world coordinates corresponding to the center point and the coordinate origin of the world coordinate system where the target calibration grid is located.

10. The method according to claim 9, characterized in that The method further comprises: If it is detected that the target marking grid is located at the edge of the scene image, obtaining the position information corresponding to the target vehicle and the heading angle corresponding to the target vehicle at three consecutive historical moments before the current moment; According to the position information corresponding to the target vehicle at the three consecutive historical moments and the heading angle corresponding to the target vehicle, determining whether there is a deviation in the position information corresponding to the target vehicle at the current moment; If there is a deviation, the position information corresponding to the target vehicle at the current moment and the heading angle corresponding to the target vehicle are weightedly calculated based on the position information corresponding to the target vehicle at three consecutive historical moments and the heading angle corresponding to the target vehicle, so as to obtain the corrected position information corresponding to the target vehicle at the current moment and the corrected heading angle corresponding to the target vehicle.

11. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the indoor vehicle positioning method as described in any one of claims 1 to 10.

12. A non-transitory machine-readable storage medium, characterized in that: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the indoor vehicle positioning method according to any one of claims 1 to 10.

Images