Auxiliary positioning method, system and computer medium for vehicle

Abstract

The present application relates to the technical field of vehicle positioning, and particularly relates to a vehicle auxiliary positioning method and system and a computer medium. The method determines pixel point coordinates representing a target vehicle from an acquired image to be processed, transforms the pixel point coordinates using a homography matrix to obtain a reference coordinate set, determines corner point coordinates from the reference coordinate set, and calculates a reference size according to the corner point coordinates. When the reference size is detected to be consistent with an actual size, a reference center point is calculated according to the reference coordinate set to obtain an auxiliary positioning coordinate. The vehicle is positioned through the image to be processed collected by a camera. The frequency of the image collected by the sensor is relatively high. When an error occurs in positioning due to an abnormality of a vehicle-mounted sensor, the vehicle can be positioned with assistance. The positioning is verified through the size of the vehicle to avoid errors in visual positioning, thereby improving the accuracy of vehicle positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle positioning, in particular to an auxiliary positioning method and system for a vehicle and a computer medium. BACKGROUND

[0002] At present, vehicle positioning usually adopts a multi-sensor fusion positioning mode, and is jointly positioned by a vehicle-mounted global navigation positioning system, a laser radar, an inertial navigation, a wheel speed sensor and other sensors, therefore, how to improve the accuracy of vehicle positioning in the case of abnormal state of a vehicle-mounted sensor becomes a problem to be solved. SUMMARY

[0003] Therefore, the embodiments of the present application provide an auxiliary positioning method and system for a vehicle and a computer medium to solve the problem of low accuracy of vehicle positioning in the case of abnormal state of a vehicle-mounted sensor.

[0004] In a first aspect, the embodiments of the present application provide an auxiliary positioning method for a vehicle, which comprises the following steps:

[0005] determining pixel point coordinates representing a target vehicle from an acquired image to be processed, transforming the pixel point coordinates using a homography matrix, and determining a change result to represent a reference coordinate set of the target vehicle, wherein the homography matrix is a transformation relationship between a preset map coordinate system and an image coordinate system of the image to be processed;

[0006] determining a reference coordinate in the reference coordinate set that meets a preset condition as a corner point coordinate of the target vehicle, calculating a reference size of the target vehicle according to the corner point coordinate, and acquiring an actual size of the target vehicle;

[0007] when detecting that the reference size is consistent with the actual size, calculating a reference center point of the target vehicle according to the reference coordinate set, and determining a reference coordinate corresponding to the reference center point as an auxiliary positioning coordinate.

[0008] In a second aspect, the embodiments of the present application provide an auxiliary positioning system for a vehicle, which comprises the following steps:

[0009] an image collector, a memory, a controller and a positioning terminal;

[0010] The image collector is connected with the controller, and is arranged on the top of a station platform in a fixed pose to continuously collect continuous images in a top-down view and send the continuous images to the controller.

[0011] The controller is connected with the memory, and identifies whether the continuous images contain a target vehicle when receiving the continuous images.

[0012] In the identification of the continuous image whether containing the target vehicle, the controller obtains the pixel point coordinates representing the target vehicle in the continuous image, uses a preset homography matrix to transform the pixel point coordinates, and obtains a reference coordinate set of the target vehicle;

[0013] The controller calculates the reference size of the target vehicle according to the reference coordinate set, and obtains the stored actual size of the target vehicle from the memory;

[0014] When the reference size is detected to be consistent with the actual size, the controller calculates the reference center point of the target vehicle according to the reference coordinate set, and determines the reference coordinate corresponding to the reference center point as an auxiliary positioning coordinate;

[0015] The controller is connected with the positioning terminal, and sends the auxiliary positioning coordinate to the positioning terminal for auxiliary positioning.

[0016] In a third aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the auxiliary positioning method of the first aspect.

[0017] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0018] The pixel point coordinates representing the target vehicle are determined from the obtained to-be-processed image, the homography matrix is used to transform the pixel point coordinates, the change result represents the reference coordinate set of the target vehicle, the homography matrix is the transformation relationship between the preset map coordinate system and the image coordinate system of the to-be-processed image, the reference coordinates meeting the preset condition in the reference coordinate set are determined as the corner point coordinates of the target vehicle, the reference size of the target vehicle is calculated according to the corner point coordinates, and the actual size of the target vehicle is obtained, when the reference size is detected to be consistent with the actual size, the reference center point of the target vehicle is calculated according to the reference coordinate set, and the reference coordinate corresponding to the reference center point is determined as the auxiliary positioning coordinate, the vehicle positioning is performed through the to-be-processed image collected by the camera, the frequency of collecting image data by the vehicle-mounted sensor is relatively high, when the vehicle-mounted sensor is abnormal and causes positioning error, the vehicle positioning can be assisted, and the positioning verification is performed through the vehicle size, so that the positioning error caused by the camera offset is avoided, and the accuracy of vehicle positioning is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0020] Figure 1 is a schematic diagram of an application environment of a vehicle auxiliary positioning method provided by the first embodiment of the present application;

[0021] Figure 2 is a flowchart of a vehicle auxiliary positioning method provided by the first embodiment of the present application;

[0022] Figure 3 is a flowchart of a vehicle auxiliary positioning method provided by the second embodiment of the present application;

[0023] Figure 4 is a system architecture diagram of a vehicle auxiliary positioning system provided by the third embodiment of the present application. DETAILED DESCRIPTION

[0024] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system architectures, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0025] It should be understood that the term "comprising" as used in the specification and the appended claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0026] It should also be understood that the term "and / or" as used herein refers to any combination of associated listed items, and all possible combinations, and includes these combinations.

[0027] As used in the description of the application and the appended claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]," depending on the context.

[0028] In addition, in the description of the application and the appended claims, the terms "first", "second", "third", etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0029] In the description of the application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0030] It should be understood that the size of the serial number of each step in the following embodiments does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0031] In order to illustrate the technical solutions of the application, the following specific embodiments are described.

[0032] The auxiliary positioning method of the vehicle provided by the embodiment of the application can be applied in the application environment of the vehicle management system as shown in Figure 1 The image acquisition device includes but is not limited to a camera, a camera head, a camera, other devices with a photographing function (a mobile phone, a tablet computer, etc.), and the like, and the vehicle management terminal corresponds to a device end including but not limited to a palm computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud terminal device, a personal digital assistant (PDA), and the like. The image acquisition device can be deployed on the top of the station to collect the images of the vehicles in the station area in a fixed pose.

[0033] Referring to Figure 2 , Fig. 1 is a flow diagram of a vehicle auxiliary positioning method provided by an embodiment of the present application. The auxiliary positioning method can be applied to a vehicle management terminal in Figure 1 , the vehicle management terminal is connected to an image acquisition device corresponding to a device end to obtain an image to be processed collected by the image acquisition device, and the vehicle management terminal has a storage function to provide an actual size of a target vehicle, which can be used to compare with a reference size calculated to determine the accuracy of the auxiliary positioning point. As shown in Figure 2 , the auxiliary positioning method can include the following steps:

[0034] Step S201, determine the pixel point coordinates representing the target vehicle from the obtained image to be processed, transform the pixel point coordinates using a homography matrix to determine the change result representing the reference coordinate set of the target vehicle.

[0035] Wherein, the image to be processed can be an image collected by an image acquisition device corresponding to a device end, the target vehicle can be a vehicle that needs to be positioned, the image acquisition device can be a camera, a camera, etc. The camera can be a binocular camera, a depth camera, etc.

[0036] The pixel point coordinates can be the coordinates of the pixel points belonging to the target vehicle in the image to be processed, the homography matrix is the transformation relationship between the preset map coordinate system and the image coordinate system of the image to be processed, and the reference coordinate can be the coordinate of the target vehicle in the map coordinate system.

[0037] Specifically, in this embodiment, the camera is fixed on the top of the platform to shoot the image of the internal area of the platform from a top-down or oblique top-down perspective. It should be noted that since the photographed area is a fixed area, the pose of the camera is fixed. Since the camera has a fixed pose, the internal and external parameters of the camera can be assumed to be known. Then, the first transformation matrix between the image coordinate system and the camera coordinate system of the camera can be determined by the internal parameters of the camera through camera calibration, and the second transformation matrix between the camera coordinate system and the preset map coordinate system can be determined by the external parameters of the camera. Multiplying the first transformation matrix and the second transformation matrix, the homography matrix can be obtained.

[0038] It should be noted that in the camera calibration process, the coordinates of the pixels in the image coordinate system and the points in the map coordinate system can be used to construct a coordinate pair, and the homography matrix can be directly solved according to the coordinate pair. The solving method can use a neural network to solve, so that the homography matrix can be obtained without knowing the internal and external parameters of the camera.

[0039] In this embodiment, after obtaining the to-be-processed image, the image can be recognized to confirm whether the to-be-processed image contains a vehicle. In this embodiment, a template matching manner is adopted to confirm, that is, a plurality of platform internal region images are pre-stored as template images. The plurality of template images are set to adapt to different external environments. The external environment can refer to weather, illumination and the like. Since the platform internal region image should be unchanged when no vehicle passes, the template matching manner can be adopted to quickly detect whether the to-be-processed image contains a vehicle. The template matching can refer to similarity calculation of the to-be-processed image and each template image. The similarity calculation manner can adopt cosine similarity, Euclidean distance and the like. In this embodiment, the cosine similarity is adopted for similarity calculation. The value range of the calculation result is [0, 1]. A corresponding similarity threshold is set, for example, 0.8. When the calculation result is greater than the similarity threshold, it is considered that the to-be-processed image does not contain a vehicle, but only contains noise information in the image acquisition process.

[0040] It should be noted that since only the external environment is different between the template images, the similarity between the template images is also high. At this time, to improve the calculation efficiency, one of the template images can be selected to perform similarity calculation with the to-be-processed image. When the calculation result is close to the similarity threshold, it is considered that the to-be-processed image does not contain a vehicle. Close can refer to a floating value. For example, the floating value can be 0.1, that is, the similarity threshold is floated downward by 0.1 to 0.7. To ensure the accuracy of the result, the to-be-processed image can be subjected to similarity calculation with each template image after the calculation result is close to the similarity threshold, and the calculation result is compared with the original similarity threshold 0.8, so as to reduce invalid calculation as much as possible, reduce the calculation load and improve the processing efficiency.

[0041] When the calculation result is less than or equal to the similarity threshold, it is considered that the to-be-processed image is greatly different from the template image, that is, the to-be-processed image contains a vehicle. At this time, it is determined that the detection result contains a target vehicle in the image recognition.

[0042] Optionally, the to-be-processed image is acquired by using a binocular camera. The to-be-processed image includes a left-eye to-be-processed image and a right-eye to-be-processed image.

[0043] Before determining the pixel point coordinates representing the target vehicle from the obtained to-be-processed image, the method further includes:

[0044] The left-eye to-be-processed image and the right-eye to-be-processed image are respectively input into a pre-trained instance segmentation model to obtain a left-eye segmentation image and a right-eye segmentation image.

[0045] The left-eye segmentation image is multiplied point by point with the left-eye to-be-processed image to obtain a left-eye vehicle contour image.

[0046] The right eye vehicle contour image is obtained by multiplying the right eye segmentation image and the right eye to-be-processed image point by point.

[0047] The binocular camera includes a left eye camera and a right eye camera, the left eye to-be-processed image can be an image to be processed collected by the left eye camera, the right eye to-be-processed image can be an image to be processed collected by the right eye camera, the instance segmentation model can be a target detection model capable of distinguishing instances, for example, a Mask-RCNN model can be used, the left eye segmentation image can be a target segmentation result of the left eye to-be-processed image, the right eye segmentation image can be a target segmentation result of the right eye to-be-processed image, the left eye vehicle contour image can be an image in which only target vehicle pixel points are contained, the right eye vehicle contour image can be an image in which only target vehicle pixel points are contained, and the left eye vehicle contour image and the right eye vehicle contour image are used to screen out objects to be recognized in the image to be processed.

[0048] Specifically, the left eye to-be-processed image and the right eye to-be-processed image correspond to each other, and are two images collected by the left eye camera and the right eye camera at the same time, the output of the instance segmentation model is an M-channel segmentation image, M can represent the number of vehicles contained in the image to be processed, and each vehicle corresponds to a segmentation image of one channel. In the segmentation image, the pixel points belonging to the vehicle are a first preset pixel value, and the other pixel points are a second preset pixel value. For example, in the segmentation image, the pixel points belonging to the vehicle have a pixel value of 1, and the other pixel points have a pixel value of 0.

[0049] If M is greater than 1, the implementer can determine that a vehicle corresponding to a segmentation image of one channel is a target vehicle. Since the pixel points belonging to the vehicle in the segmentation image have a pixel value of 1, and the other pixel points have a pixel value of 0, the segmentation image is multiplied with the image to be processed point by point, so that the pixel value of the pixel points belonging to the vehicle in the image to be processed is retained, and the pixel value of the other pixel points is set to 0, thereby obtaining a vehicle contour image.

[0050] The embodiment determines the pixel points of the target vehicle in the image to be processed by the instance segmentation manner, can isolate irrelevant working conditions, improve the accuracy of subsequent recognition, and further improve the accuracy of auxiliary positioning.

[0051] Optionally, after obtaining the right eye vehicle contour image, the method further includes:

[0052] The left eye vehicle contour image and the right eye vehicle contour image are input into a pre-trained character recognition model to obtain a left eye recognition result and a right eye recognition result.

[0053] When the left eye recognition result is consistent with the right eye recognition result, the left eye recognition result or the right eye recognition result is determined as the image recognition result. After obtaining the image recognition result, the step of determining the pixel point coordinates representing the target vehicle from the obtained to-be-processed image is performed.

[0054] The character recognition model can be an optical character recognition model (OCR), the left eye recognition result can be character information in a left eye vehicle contour image, and the right eye recognition result can be character information in a right eye vehicle contour image.

[0055] Specifically, the character recognition model can be used to extract a vehicle identification in a vehicle contour image. The vehicle identification can be a vehicle serial number, a vehicle license plate, or other information that can be used to identify a vehicle. To avoid recognition errors of the character recognition model, the left eye recognition result is compared with the right eye recognition result. When the left eye recognition result is consistent with the right eye recognition result, any one of the left eye recognition result and the right eye recognition result is determined as the image recognition result. If the left eye recognition result is inconsistent with the right eye recognition result, the instance segmentation and character recognition processing are performed again after waiting for a preset number of frames to avoid image recognition errors due to motion blur of the to-be-processed image collected during the entry of the vehicle. The preset number of frames can be set to 15 frames. If the left eye recognition result and the right eye recognition result are inconsistent after waiting for Q preset number of frames, Q can be set to 10, the abnormal information is generated and sent to the operation and maintenance personnel.

[0056] The left eye vehicle contour image and the right eye vehicle contour image are subjected to character recognition respectively, and the left eye recognition result and the right eye recognition result are verified with each other, so as to ensure the accuracy of the recognition result and avoid misjudgment in subsequent comparison with vehicle information, thereby improving the accuracy of the auxiliary positioning.

[0057] Optionally, after obtaining the image recognition result, the step of determining the pixel point coordinates representing the target vehicle from the obtained to-be-processed image includes:

[0058] N preset vehicle identifications are obtained, similarity calculation is performed between each vehicle identification and the image recognition result, and a similarity calculation result is obtained.

[0059] If the maximum similarity in the similarity calculation result is greater than a preset similarity threshold, it is determined that the image recognition result contains the target vehicle, and the step of determining the pixel point coordinates representing the target vehicle from the obtained to-be-processed image is performed.

[0060] Wherein, N is an integer greater than zero, the vehicle identifier can be pre-stored vehicle identifier information, for example, vehicle serial number, vehicle license plate and the like can be used to identify the vehicle information, the similarity calculation can use the Euclidean distance, and the similarity calculation result can be used to determine the vehicle identifier corresponding to the target vehicle.

[0061] Specifically, all vehicles passing through the platform store the corresponding vehicle identifier, and then the image recognition result of the target vehicle is compared with each stored vehicle identifier to obtain the actual vehicle identifier of the target vehicle.

[0062] Since there is a vehicle identifier consistent with the image recognition result, the similarity threshold can be set to 0.95 to ensure that the vehicle identifier corresponding to the target vehicle is obtained.

[0063] If the maximum similarity in the similarity calculation result is less than or equal to the similarity threshold, it means that the image recognition is abnormal at this time, and the identification can be performed again after waiting for a preset number of frames. The preset number of frames can be set to 15 frames. If Q preset frames are waited for, Q can be set to 10, and the maximum similarity in the similarity calculation result is still greater than the similarity threshold. Then, an abnormal information is generated and sent to the operation and maintenance personnel.

[0064] The embodiment compares the image recognition result with the stored vehicle identifier to obtain the actual vehicle identifier of the target vehicle, which is beneficial to the accuracy of the subsequent process of obtaining the actual size of the target vehicle according to the actual vehicle identifier, and thus improves the accuracy of the auxiliary positioning.

[0065] Optionally, the pixel point coordinates are transformed using the homography matrix to determine the change result representing the reference coordinate set of the target vehicle, comprising:

[0066] The feature points are extracted from the left-eye vehicle contour image and the right-eye vehicle contour image to obtain left-eye feature points and right-eye feature points, and when the left-eye feature points and the right-eye feature points meet a preset condition, the left-eye feature points and the right-eye feature points are determined as a feature point pair.

[0067] According to the depth of the pixel point coordinates calculated according to the feature point pair based on the principle of triangulation.

[0068] According to the feature point pair and the depth, the camera coordinates of the pixel point coordinates in the camera coordinate system are calculated based on the pinhole imaging model.

[0069] The camera coordinates are transformed into reference coordinates in the map coordinate system by the homography matrix to obtain the reference coordinate set.

[0070] The feature point extraction can be used to extract pixel points with preset features in the image, the feature point extraction mode can adopt a Harris corner detection, SIFT feature point extraction and the like, the preset condition can be a feature point matching condition, the feature point matching can be performed through a normalized cross-correlation function and a random sample consensus algorithm, the depth can be a distance from an actual position corresponding to the feature point to a camera imaging surface, and the pinhole imaging model can be a perspective projection model in physics.

[0071] Specifically, the focal length of the camera can be calculated based on the depth, and the pixel point coordinates can be converted into camera coordinates in a camera coordinate system according to the camera focal length and the coordinate information of the feature point pair. In this embodiment, the homography matrix can be a pose transformation matrix, which includes rotation parameters and translation parameters, i.e., the extrinsic parameters of the camera. The pose transformation matrix can convert the camera coordinates into reference coordinates in a map coordinate system. The above processing is performed for each feature point, and a reference coordinate set is obtained.

[0072] In this embodiment, the pose transformation matrix of the camera is used as the homography matrix, so that the homography matrix can be calculated only according to the camera pose, which reduces the difficulty of camera calibration, improves the efficiency of camera calibration, and further reduces the complexity of the preparation work before the auxiliary positioning, so that this embodiment can be applied to the auxiliary positioning of the vehicle more quickly.

[0073] The above step of obtaining the pixel point coordinates representing the target vehicle in the to-be-processed image, transforming the pixel point coordinates using the homography matrix, and determining the reference coordinate set representing the target vehicle according to the change result, converts the pixel point coordinates in the image coordinate system of the target vehicle into the reference coordinates in the map coordinate system, so as to obtain the actual position of the target vehicle, which facilitates the subsequent auxiliary positioning and improves the accuracy of the vehicle auxiliary positioning.

[0074] In step S202, the reference coordinates in the reference coordinate set that meet the preset condition are determined as the corner point coordinates of the target vehicle, the reference size of the target vehicle is calculated according to the corner point coordinates, and the actual size of the target vehicle is obtained.

[0075] The corner point coordinates can be the reference coordinates of the four corner points at the top of the target vehicle, the reference size can be the length and width of the target vehicle calculated from the reference coordinates, and the actual size can be the real length and real width of the target vehicle stored.

[0076] Specifically, the manner of obtaining the corner points can adopt a neural network model, corner point detection, etc. For example, in the embodiment, the extracted points are obtained by Harris corner point detection, i.e., the extracted points are initial corner points. At this time, the initial corner points can be used to represent points with gradient changes in the vicinity. The vicinity can refer to a four-neighbor domain, an eight-neighbor domain, etc.

[0077] The reference coordinates corresponding to the initial corner points are filtered. First, the reference coordinates with the largest z coordinates are retained according to the z coordinates of the reference coordinates. Then, the vehicle corner point characteristics are used for re-filtering. The vehicle corner point characteristics can refer to that the four corner points can constitute a rectangle.

[0078] The reference coordinates in the reference coordinate set that meet the preset condition are determined as the corner point coordinates of the target vehicle. According to the corner point coordinates, the reference size of the target vehicle is calculated, and the actual size of the target vehicle is obtained. The reference size of the target vehicle is calculated by the corner point coordinates, which is convenient for comparison with the actual size of the target vehicle obtained, thereby reducing the influence of parallax on positioning.

[0079] In step S203, when the reference size is consistent with the actual size, the reference center point of the target vehicle is calculated according to the reference coordinate set, and the reference coordinate corresponding to the reference center point is determined as the auxiliary positioning coordinate.

[0080] The reference center point can refer to the centroid point of all reference coordinates. The centroid point can be used to represent the positioning point of the target vehicle obtained by visual processing. The auxiliary positioning coordinate can refer to the actual coordinate corresponding to the positioning point obtained by visual processing.

[0081] Specifically, when the reference size is consistent with the actual size, it indicates that the parameters of the camera at this time are not abnormal, and are not affected by parallax.

[0082] Optionally, after determining the reference coordinate corresponding to the reference center point as the auxiliary positioning coordinate, the method further includes:

[0083] Obtaining the vehicle positioning coordinate sent by the in-vehicle sensor;

[0084] When the vehicle positioning coordinate is consistent with the auxiliary positioning coordinate, the position of the target vehicle in the preset topological electronic map is determined according to the vehicle positioning coordinate.

[0085] The vehicle positioning coordinate can refer to the coordinate in the map coordinate system obtained by positioning the target vehicle by the vehicle sensor. The topological electronic map can refer to a preset vehicle position representation map, which is used to visualize the real-time position of the target vehicle.

[0086] Specifically, when the vehicle positioning coordinate is consistent with the auxiliary positioning coordinate, it indicates that the visual positioning and the sensor positioning are both without deviation, and in this case, the vehicle positioning coordinate is directly taken as the real positioning coordinate.

[0087] When the vehicle positioning coordinate is inconsistent with the auxiliary positioning coordinate, it indicates that the visual positioning and the sensor positioning are deviated, and in this case, the next target vehicle can be waited for and positioned, and if the vehicle positioning coordinate is still inconsistent with the auxiliary positioning coordinate, it indicates that the camera is abnormal at this time, and there may be lens offset, jitter and the like, and if the positioning coordinate is consistent with the auxiliary positioning coordinate, it indicates that the sensor in the target vehicle is abnormal at this time, the auxiliary positioning coordinate is taken as the real positioning coordinate, and abnormal information is sent to prompt the operation and maintenance personnel to check and maintain the target vehicle.

[0088] The embodiment performs auxiliary positioning on the vehicle positioning coordinate acquired by the sensor according to the auxiliary positioning coordinate, so that the vehicle positioning abnormality caused by abnormal conditions can be avoided, and the accuracy of vehicle positioning is improved.

[0089] Optionally, the obtaining process of the homography matrix comprises:

[0090] Selecting a calibration feature point from the preset high-precision map data;

[0091] According to the calibration feature point, a calibration pixel point is matched from the acquired image to be calibrated;

[0092] According to the calibration pixel point, a camera calibration point corresponding to the calibration pixel point is obtained through an imaging model;

[0093] According to the calibration feature point and the camera calibration point, the homography matrix is calculated.

[0094] The high-precision map data can be coordinate information data in a map coordinate system, the calibration feature point can be a calibration point with obvious features, and the implementer can also set the calibration point artificially, for example, placing a calibration board in a station area, and the calibration pixel point can be a pixel point corresponding to the calibration feature point in the image to be calibrated, and the imaging model can be a pinhole imaging model, that is, the camera internal parameter data is known, and then the image coordinates of the calibration pixel point can be converted into the camera calibration point coordinates in the camera coordinate system.

[0095] The embodiment solves the homography matrix through pre-calibration, so that the parallax of the collected image to be processed is reduced as much as possible, and the accuracy of auxiliary positioning is improved.

[0096] The step of calculating the reference center point of the target vehicle according to the reference coordinate set when the reference size is consistent with the actual size, and determining the reference coordinate corresponding to the reference center point as the auxiliary positioning coordinate, verifies the accuracy of the auxiliary positioning through the reference size and the actual size, avoids the positioning error caused by the camera offset, and thus improves the accuracy of vehicle positioning.

[0097] The vehicle positioning is performed through the image to be processed collected by the camera, the frequency of collecting image data by the vehicle sensor is relatively high, when the vehicle sensor is abnormal and causes positioning error, the vehicle positioning can be assisted, and the positioning verification is performed through the vehicle size, the positioning error caused by the camera offset is avoided, and thus the accuracy of vehicle positioning is improved.

[0098] Referring to Figure 3 is a flowchart of a vehicle auxiliary positioning method provided in Embodiment Two of the application, in the auxiliary positioning method, the reference size can be determined according to the rectangle formed by the corner points, or can be determined according to the line segment fitted by the corner points.

[0099] When the reference size is determined according to the rectangle formed by the corner points, the determination process of the reference size is described with reference to Embodiment One, and is not described herein.

[0100] When the reference size is determined according to the line segment fitted by the corner points, the determination process of the reference size includes:

[0101] In step S301, a preset template is used to count the neighborhood coordinates of each reference coordinate, and the reference coordinates equal to the preset counting threshold in the counting result are determined as the corner point coordinates;

[0102] In step S302, the fitting line segment set is obtained through straight line fitting according to the corner point coordinates, and the fitting line segments with parallel fitting line segments in the fitting line segment set are determined as the boundary line segments;

[0103] In step S303, the distance between the boundary line segment and the parallel fitting line segment corresponding to the boundary line segment is calculated, and the calculation result is determined as the reference size of the target vehicle.

[0104] The preset template can be a corner point extraction template, the counting threshold can be used to determine whether the reference coordinate is a corner point coordinate, the fitting line segment can be a straight line fitting, and the fitting line segment is determined according to the fitted straight line, the straight line fitting can use a binary first-order equation as the expression of the straight line, at this time, it is defaulted that the z values in the reference coordinates are consistent, that is, the distances of the corner point coordinates of the vehicle from the horizontal axis and the vertical axis of the map coordinate system to form a plane are consistent, and the boundary line segment can be a line segment belonging to the boundary of the target vehicle.

[0105] Specifically, since the reference coordinates are three-dimensional coordinates, the size of the template can be 3*3*3, that is, the corner points are determined according to 26 neighborhood coordinates of the reference coordinates, for example, the elements in the template can all be set to 1, after the template is used to convolve the reference coordinates and the neighborhood coordinates of the reference coordinates, the convolution result is compared with a statistical threshold, the statistical threshold can be 4, in the case of not counting the reference coordinates and only counting the neighborhood coordinates, the statistical threshold can be 3, so that four corner point coordinates with the same z value are obtained;

[0106] Connecting any two of the four corner points and fitting a corresponding straight line can obtain six fitting line segments, wherein there are two groups of line segments, and the two fitting line segments in each group of line segments are parallel to each other, so that the fitting line segments contained in the two groups of line segments can be determined as boundary line segments.

[0107] The distance between the two parallel boundary line segments is calculated to obtain a first distance and a second distance, and the larger one of the first distance and the second distance is determined as a reference length, and the smaller one of the first distance and the second distance is determined as a reference width.

[0108] The embodiment determines the boundary line segments according to the parallel relationship between the line segments by the fitting line segments fitted by the corner points, so that the case that it is difficult to obtain a rectangle composed of corner points in the case that there is a small parallax is avoided, and the robustness of the auxiliary positioning is improved.

[0109] Figure 4 The system architecture diagram of the auxiliary positioning system of the vehicle provided by the embodiment three of the application is shown, and the auxiliary positioning system comprises:

[0110] An image collector, a memory, a controller and a positioning terminal;

[0111] The image collector is connected with the controller, and the image collector is arranged on the top of the station platform in a fixed pose to continuously collect continuous images in a top-down view and send the continuous images to the controller;

[0112] The controller is connected with the memory, and the controller identifies whether the continuous images contain the target vehicle when the continuous images are received;

[0113] When it is identified whether the continuous images contain the target vehicle, the controller obtains the pixel point coordinates representing the target vehicle in the continuous images, transforms the pixel point coordinates using a preset homography matrix to obtain a reference coordinate set of the target vehicle;

[0114] The controller calculates the reference size of the target vehicle according to the reference coordinate set, and obtains the stored actual size of the corresponding target vehicle from the memory;

[0115] When it is detected that the reference size is consistent with the actual size, the controller calculates a reference center point of the target vehicle according to the reference coordinate set, and determines a reference coordinate corresponding to the reference center point as an auxiliary positioning coordinate;

[0116] The controller is connected with the positioning terminal, and the auxiliary positioning coordinate is sent to the positioning terminal for auxiliary positioning.

[0117] The image collector can be a camera. Since the pose of the camera is fixed, the intrinsic and extrinsic parameters of the camera can be easily obtained through calibration and will not change. Therefore, the intrinsic and extrinsic parameters of the camera are known by default. Since the variable parameters in the homography matrix are the intrinsic and extrinsic parameters of the camera, the homography matrix is known by default.

[0118] Specifically, when identifying whether the continuous images contain the target vehicle, an instance segmentation model can be used to process the images to obtain the image position of the target vehicle in the images, so as to obtain a pixel point coordinate set of the target vehicle in the images. The pixel point coordinate set includes a plurality of pixel point coordinates of the target vehicle.

[0119] For any pixel point coordinate, the pixel point coordinate is expressed as a column vector, that is, a one-row and three-column vector. The elements in the vector are the horizontal coordinate in the pixel point coordinate, the column coordinate in the pixel point coordinate, and 1. The result obtained by multiplying the column vector by the homography matrix on the right is the reference coordinate of the corresponding pixel point coordinate.

[0120] The reference size can be determined by extracting the corner points from the reference coordinate set and then determining the distance between the corner points. In this embodiment, the corner points can be extracted by using Harris corner detection, SIFT feature point extraction, or the like.

[0121] In an embodiment, the corner points of the bounding box of the target vehicle obtained by instance segmentation can be used to determine the reference points closest to each bounding box corner point in the reference coordinate set as the corner points.

[0122] The reference center point can be obtained by using the centroid calculation method, that is, the average value of the horizontal coordinates of all reference coordinates in the reference coordinate set is used as the horizontal coordinate of the reference center point, and the average value of the vertical coordinates of all reference coordinates is used as the vertical coordinate of the reference center point.

[0123] It should be noted that the information interaction and execution process of the above system components are based on the same concept as the method embodiments. For specific functions and technical effects, please refer to the method embodiment part. Here, it will not be repeated.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above system components is exemplified, and in actual application, the functions corresponding to the above system components can be distributed by different computing modules or computing units according to needs, that is, the internal structure of the system is divided into different computing units or computing modules to complete all or part of the functions described above. The computing unit and the computing module corresponding to each system component in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit. In addition, the specific names of each system component are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of each system component in the system can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here. If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the foregoing embodiment methods, which can be completed by a computer program to instruct related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of the foregoing method embodiment can be realized. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0125] The present application realizes all or part of the processes in the foregoing embodiment methods, which can also be completed by a computer program product. When the computer program product runs on the computer device, it makes the computer device execute the steps in the foregoing method embodiment.

[0126] In the foregoing embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0127] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0128] In the embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the above-described system embodiments are merely illustrative. For example, the division of the above system components is only a logical function division. In actual implementation, another division manner can be used. For example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be through some interface, and the indirect coupling or communication connection between the units can be in electrical, mechanical or other forms.

[0129] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0130] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for assisted vehicle positioning, characterized in that, The method includes: A binocular camera is used to acquire images to be processed, which include left-eye and right-eye images to be processed. The left-eye image to be processed and the right-eye image to be processed are respectively input into the pre-trained instance segmentation model to obtain the left-eye segmentation image and the right-eye segmentation image; The left-eye segmented image is multiplied point by point with the left-eye image to be processed to obtain the left-eye vehicle outline image; The right-eye segmented image is multiplied point by point with the right-eye image to be processed to obtain the right-eye vehicle outline image. The left-eye vehicle outline image and the right-eye vehicle outline image are used to filter out the recognition object in the image to be processed. The left-eye vehicle outline image and the right-eye vehicle outline image are respectively input into the pre-trained character recognition model to obtain the left-eye recognition result and the right-eye recognition result. When the left-eye recognition result is consistent with the right-eye recognition result, the left-eye recognition result or the right-eye recognition result is determined to be the image recognition result; Obtain N preset vehicle identifiers, calculate the similarity between each vehicle identifier and the image recognition result, and obtain the similarity calculation result, where N is an integer greater than zero; If the maximum similarity in the similarity calculation result is greater than the preset similarity threshold, then the image recognition result is determined to contain the target vehicle. The pixel coordinates representing the target vehicle are determined from the acquired image to be processed. The pixel coordinates are transformed using a homography matrix, and the transformation result is determined to represent the reference coordinate set of the target vehicle. The homography matrix is ​​the transformation relationship between a preset map coordinate system and the image coordinate system of the image to be processed. The reference coordinates that meet the preset conditions in the reference coordinate set are determined as the corner coordinates of the target vehicle. Based on the corner coordinates, the reference size of the target vehicle is calculated, and the actual size of the target vehicle is obtained. When the reference size is detected to be consistent with the actual size, the reference center point of the target vehicle is calculated based on the reference coordinate set, and the reference coordinates corresponding to the reference center point are determined as auxiliary positioning coordinates.

2. The assisted positioning method according to claim 1, characterized in that, The process of transforming the pixel coordinates using a homography matrix to determine the reference coordinate set representing the target vehicle includes: Feature points are extracted from the left-eye vehicle contour image and the right-eye vehicle contour image respectively to obtain left-eye feature points and right-eye feature points. When the left-eye feature points and the right-eye feature points meet preset conditions, the left-eye feature points and the right-eye feature points are determined to be a feature point pair. For the pixel coordinates corresponding to any feature point in the feature point pair, the depth of the pixel coordinates is calculated based on the principle of triangulation. Based on the pinhole imaging model, the camera coordinates of the pixel point in the camera coordinate system are calculated according to the feature point pair and the depth. The camera coordinates are transformed into reference coordinates in the map coordinate system using the homography matrix to obtain the reference coordinate set.

3. The assisted positioning method according to claim 1, characterized in that, The step of determining the reference coordinates that meet the preset conditions in the reference coordinate set as the corner coordinates of the target vehicle, and calculating the reference dimensions of the target vehicle based on the corner coordinates, includes: The neighborhood coordinates of each reference coordinate are statistically analyzed using a preset template, and the reference coordinates that are equal to the preset statistical threshold in the statistical results are determined as the corner coordinates. Based on the corner coordinates, a set of fitted line segments is obtained by fitting a straight line, and the fitted line segments in the set of fitted line segments that contain parallel fitted line segments are determined as boundary line segments. Calculate the distance between the boundary line segment and the corresponding parallel fitted line segment, and determine the calculation result as the reference size of the target vehicle.

4. The assisted positioning method according to claim 1, characterized in that, After determining the reference coordinates corresponding to the reference center point as the auxiliary positioning coordinates, the method further includes: Obtain the vehicle positioning coordinates sent by the sensors inside the target vehicle; When the vehicle positioning coordinates are consistent with the auxiliary positioning coordinates, the position of the target vehicle in the preset topological electronic map is determined based on the vehicle positioning coordinates.

5. The auxiliary positioning method according to any one of claims 1 to 4, characterized in that, The process of obtaining the homography matrix includes: Select calibration feature points from the preset high-precision map data; Based on the calibration feature points, calibration pixels are obtained by matching from the acquired image to be calibrated; Based on the calibrated pixels, the camera calibration points corresponding to the calibrated pixels are obtained through the imaging model; The homography matrix is ​​calculated based on the calibration feature points and the camera calibration points.

6. A vehicle auxiliary positioning system, characterized in that, The auxiliary positioning system includes: Image acquisition unit, memory, controller, and positioning terminal; The image acquisition device is connected to the controller. The image acquisition device is deployed in a fixed position on the top of the station platform to continuously acquire continuous images from a top-down perspective and send the continuous images to the controller. The controller is connected to the memory, and when the controller receives the continuous images, it identifies whether the continuous images contain the target vehicle. When it is determined whether the continuous image contains a target vehicle, the controller obtains the pixel coordinates representing the target vehicle in the continuous image, transforms the pixel coordinates using a preset homography matrix, and obtains a reference coordinate set of the target vehicle. The homography matrix is ​​the transformation relationship between the preset map coordinate system and the image coordinate system of the image to be processed. The controller calculates the reference dimensions of the target vehicle based on the reference coordinate set, and retrieves the actual dimensions of the target vehicle stored in the memory. When the reference size is detected to be consistent with the actual size, the controller calculates the reference center point of the target vehicle based on the reference coordinate set and determines the reference coordinates corresponding to the reference center point as auxiliary positioning coordinates; The controller is connected to the positioning terminal and sends the auxiliary positioning coordinates to the positioning terminal for auxiliary positioning. The controller uses a binocular camera to acquire images to be processed, which include left-eye and right-eye images to be processed. The left-eye image to be processed and the right-eye image to be processed are respectively input into the pre-trained instance segmentation model to obtain the left-eye segmentation image and the right-eye segmentation image; The left-eye segmented image is multiplied point by point with the left-eye image to be processed to obtain the left-eye vehicle outline image; The right-eye segmented image is multiplied point by point with the right-eye image to be processed to obtain the right-eye vehicle outline image. The left-eye vehicle outline image and the right-eye vehicle outline image are used to filter out the recognition object in the image to be processed. The left-eye vehicle outline image and the right-eye vehicle outline image are respectively input into the pre-trained character recognition model to obtain the left-eye recognition result and the right-eye recognition result. When the left-eye recognition result is consistent with the right-eye recognition result, the left-eye recognition result or the right-eye recognition result is determined to be the image recognition result; Obtain N preset vehicle identifiers, calculate the similarity between each vehicle identifier and the image recognition result, and obtain the similarity calculation result, where N is an integer greater than zero; If the maximum similarity in the similarity calculation result is greater than the preset similarity threshold, then the image recognition result is determined to contain the target vehicle.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the assisted positioning method as described in any one of claims 1 to 5.

Images