Target object positioning method and apparatus, computer device, and storage medium

Abstract

The application relates to a target object positioning method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: obtaining a target image through a target camera installed on a target vehicle; obtaining a pixel mask of a target object through an identification model; performing lane line identification on the target image through a lane line detection model to obtain a lane line function corresponding to each lane line; inputting a target pixel longitudinal coordinate of the pixel mask of the target object into the lane line function to obtain a predicted transverse coordinate; determining a first lane line and a second lane line according to each predicted transverse coordinate, a minimum pixel transverse coordinate and a maximum pixel transverse coordinate, and determining a lane in which the target object is located; and determining position information of the target object according to a distance between the target object and the target vehicle and the lane in which the target object is located. The method can improve the positioning speed of the target object.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a target object localization and detection method, apparatus, computer equipment, storage medium and computer program product. Background Technology

[0002] With the development of information computer vision technology, automotive vision perception technology has emerged. This technology utilizes various advanced sensor technologies to perceive road traffic environment information and feeds real-time information such as target location and lane information acquired by the sensors back to a server for processing. Simultaneously, it identifies target objects such as vehicles or pedestrians based on road condition information, analyzes the object category of the target object, and locates its lane position. This allows for automatic early warning in emergency situations, ensuring smooth and safe vehicle movement.

[0003] Traditional LiDAR-based target object localization methods involve acquiring 3D point cloud data of road conditions, extracting features from this data (including different types of low-dimensional and high-dimensional features), and then transmitting these features to a target object detection network. After processing these features, the network obtains the lane lines and target object coordinates in the world coordinate system, thus enabling target object localization.

[0004] However, current target object localization methods based on lidar not only require high hardware costs, but also suffer from poor real-time localization performance due to the large amount of 3D point cloud data processing involved. Summary of the Invention

[0005] Therefore, it is necessary to provide a target object positioning method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the target object positioning speed in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a method for locating a target object. The method includes:

[0007] A target image is obtained by capturing images of the road using a target camera installed on the target vehicle. The target image includes image data corresponding to at least one lane.

[0008] The target image is used to perform object recognition by a recognition model to obtain the pixel mask of the target object;

[0009] The pixel mask coordinate information corresponding to the pixel mask of the target object in the target image is determined. The pixel mask coordinate information includes the maximum value of the pixel horizontal coordinate, the minimum value of the pixel horizontal coordinate, and the target pixel vertical coordinate. The target pixel vertical coordinate is determined based on the pixel vertical coordinate corresponding to the maximum value of the pixel horizontal coordinate and the pixel vertical coordinate corresponding to the minimum value of the pixel horizontal coordinate.

[0010] The target image is used to identify lane lines by a lane line detection model to obtain the lane line function corresponding to each lane line.

[0011] For any lane line function corresponding to the lane line, the target pixel ordinate of the target object is input into the lane line function to obtain the predicted abscissa corresponding to the target pixel ordinate.

[0012] Based on the predicted abscissas corresponding to the vertical coordinates of the target pixel in each of the lane line functions, as well as the minimum and maximum values ​​of the pixel abscissas, the first lane line and the second lane line are determined, and the lane in which the target object is located is determined based on the first lane line and the second lane line.

[0013] The distance between the target object and the target vehicle is obtained, and the location information of the target object is determined based on the distance and the lane in which the target object is located.

[0014] In one embodiment, the lane detection model includes a detection layer, a correction layer, and a fitting layer. The step of identifying lane lines in the target image using the lane detection model to obtain lane line functions corresponding to each lane line includes:

[0015] In the detection layer, lane line recognition is performed on the target image to obtain lane line pixels corresponding to each lane line;

[0016] In the correction layer, the inverse projection matrix is ​​determined according to the camera matrix of the target camera, and the lane line pixels corresponding to each lane line are corrected according to the inverse projection matrix to obtain the corrected lane line pixels corresponding to each lane line.

[0017] In the fitting layer, the modified lane line pixels corresponding to each lane line are fitted to obtain the lane line function corresponding to each lane line.

[0018] In one embodiment, the step of determining the inverse projection matrix based on the camera matrix of the target camera, and correcting the lane line pixels of each lane line according to the inverse projection matrix to obtain the corrected lane line pixels corresponding to each lane line includes:

[0019] Obtain the camera matrix of the target camera, and determine the inverse projection matrix based on the camera matrix;

[0020] Determine the mapping relationship between image pixels and image coordinates, and based on the mapping relationship and the lane line pixels of each lane line, determine the image coordinates corresponding to the lane line pixels of each lane line in the target image.

[0021] Based on the inverse projection matrix and the image coordinates of the lane line pixels in the target image, each lane line pixel is corrected to obtain the corrected lane line pixels corresponding to each lane line.

[0022] In one embodiment, determining a first lane line and a second lane line based on the predicted horizontal coordinates corresponding to the vertical coordinates of the target pixel in each of the lane line functions, as well as the minimum and maximum values ​​of the horizontal coordinates of the pixel, and determining the lane in which the target object is located based on the first lane line and the second lane line, includes:

[0023] In each of the lane line functions, the predicted abscissa corresponding to the vertical coordinate of the target pixel is compared with the minimum value of the pixel abscissa. Based on the comparison result, the lane line function corresponding to the minimum value of the pixel abscissa is determined, and the first lane line is determined based on the lane line function corresponding to the minimum value of the pixel abscissa.

[0024] In each of the lane line functions, the predicted abscissa corresponding to the vertical coordinate of the target pixel is compared with the maximum value of the pixel abscissa. Based on the comparison result, the lane line function corresponding to the maximum value of the pixel abscissa is determined, and the second lane line is determined based on the lane line function corresponding to the maximum value of the pixel abscissa.

[0025] The lane in which the target object is located is determined based on the first lane line and the second lane line.

[0026] In one embodiment, the step of acquiring a target image of the road using a target camera installed on the target vehicle includes:

[0027] An initial image is obtained by capturing images of the road using a target camera installed on the target vehicle.

[0028] The target region in the initial image is masked to obtain the target image, wherein the lane lines are not included in the target region.

[0029] In one embodiment, the method further includes:

[0030] Obtain a sample image dataset, which contains multiple sample image data, including image data corresponding to at least one lane line;

[0031] The target region in each sample image data is masked to obtain a target image dataset, wherein the lane line is not included in the target region;

[0032] The initial lane detection model is trained using the sample image data from the target image dataset to obtain the lane detection model.

[0033] In one embodiment, obtaining the distance from the target object to the target vehicle includes:

[0034] Based on the pixel mask of the target object, obtain the maximum and minimum values ​​of the pixel ordinate of the target object;

[0035] Based on the maximum and minimum pixel ordinate values ​​of the target object, the sensor parameters of the target camera are obtained;

[0036] The distance from the target object to the target vehicle is obtained based on the focal length of the target camera, the spatial height of the target object, and the sensor parameters of the target camera.

[0037] Secondly, this application also provides a target object positioning device. The device includes:

[0038] The acquisition module is used to acquire images of the road through a target camera installed on the target vehicle to obtain a target image, wherein the target image includes image data corresponding to at least one lane.

[0039] The first recognition module is used to perform object recognition on the target image through a recognition model to obtain the pixel mask of the target object;

[0040] The first determining module is used to determine the pixel mask coordinate information corresponding to the pixel mask of the target object in the target image. The pixel mask coordinate information includes the maximum value of the pixel horizontal coordinate, the minimum value of the pixel horizontal coordinate, and the target pixel vertical coordinate. The target pixel vertical coordinate is determined based on the pixel vertical coordinate corresponding to the maximum value of the pixel horizontal coordinate and the pixel vertical coordinate corresponding to the minimum value of the pixel horizontal coordinate.

[0041] The second recognition module is used to perform lane line recognition on the target image through a lane line detection model to obtain the lane line function corresponding to each lane line.

[0042] The prediction module is used to input the target pixel ordinate of the target object into the lane line function for any lane line to obtain the predicted abscissa corresponding to the target pixel ordinate;

[0043] The second determining module is used to determine the first lane line and the second lane line based on the predicted abscissa corresponding to the vertical coordinate of the target pixel in each of the lane line functions, as well as the minimum value and the maximum value of the pixel abscissa, and to determine the lane in which the target object is located based on the first lane line and the second lane line.

[0044] The third determining module is used to obtain the distance between the target object and the target vehicle, and determine the location information of the target object based on the distance and the lane in which the target object is located.

[0045] In one embodiment, the lane detection model includes a detection layer, a correction layer, and a fitting layer, and the second recognition module is specifically used for:

[0046] In the detection layer, lane line recognition is performed on the target image to obtain lane line pixels corresponding to each lane line;

[0047] In the correction layer, the inverse projection matrix is ​​determined according to the camera matrix of the target camera, and the lane line pixels corresponding to each lane line are corrected according to the inverse projection matrix to obtain the corrected lane line pixels corresponding to each lane line.

[0048] In the fitting layer, the modified lane line pixels corresponding to each lane line are fitted to obtain the lane line function corresponding to each lane line.

[0049] In one embodiment, the second identification module is specifically used for:

[0050] Obtain the camera matrix of the target camera, and determine the inverse projection matrix based on the camera matrix;

[0051] Determine the mapping relationship between image pixels and image coordinates, and based on the mapping relationship and the lane line pixels of each lane line, determine the image coordinates corresponding to the lane line pixels of each lane line in the target image.

[0052] Based on the inverse projection matrix and the image coordinates of the lane line pixels in the target image, each lane line pixel is corrected to obtain the corrected lane line pixels corresponding to each lane line.

[0053] In one embodiment, the second determining module is specifically used for:

[0054] In each of the lane line functions, the predicted abscissa corresponding to the vertical coordinate of the target pixel is compared with the minimum value of the pixel abscissa. Based on the comparison result, the lane line function corresponding to the minimum value of the pixel abscissa is determined, and the first lane line is determined based on the lane line function corresponding to the minimum value of the pixel abscissa.

[0055] In each of the lane line functions, the predicted abscissa corresponding to the vertical coordinate of the target pixel is compared with the maximum value of the pixel abscissa. Based on the comparison result, the lane line function corresponding to the maximum value of the pixel abscissa is determined, and the second lane line is determined based on the lane line function corresponding to the maximum value of the pixel abscissa.

[0056] The lane in which the target object is located is determined based on the first lane line and the second lane line.

[0057] In one embodiment, the acquisition module is specifically used for:

[0058] An initial image is obtained by capturing images of the road using a target camera installed on the target vehicle.

[0059] The target region in the initial image is masked to obtain the target image, wherein the lane lines are not included in the target region.

[0060] In one embodiment, the device further includes:

[0061] The acquisition module is used to acquire a sample image dataset, which contains multiple sample image data, including image data corresponding to at least one lane line.

[0062] The masking module is used to mask the target region in each sample image data to obtain a target image dataset, wherein the lane line is not included in the target region;

[0063] The training module is used to train the initial lane detection model based on the sample image data in the target image dataset to obtain the lane detection model.

[0064] In one embodiment, the third determining module is specifically used for:

[0065] Based on the pixel mask of the target object, obtain the maximum and minimum values ​​of the pixel ordinate of the target object;

[0066] Based on the maximum and minimum pixel ordinate values ​​of the target object, the sensor parameters of the target camera are obtained;

[0067] The distance from the target object to the target vehicle is obtained based on the focal length of the target camera, the spatial height of the target object, and the sensor parameters of the target camera.

[0068] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the target object localization method described in the first aspect.

[0069] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the target object location method described in the first aspect above.

[0070] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the target object location method described in the first aspect.

[0071] The aforementioned target object localization method, apparatus, computer equipment, storage medium, and computer program product acquire target images of the road using a target camera installed on a target vehicle. The target image includes image data corresponding to at least one lane line. An object recognition model is used to perform object recognition on the target image to obtain a pixel mask of the target object. The pixel mask coordinate information corresponding to the pixel mask of the target object in the target image is determined. The pixel mask coordinate information includes a maximum horizontal coordinate, a minimum horizontal coordinate, and a target pixel vertical coordinate. The target pixel vertical coordinate is determined based on the pixel vertical coordinate corresponding to the maximum horizontal coordinate and the pixel vertical coordinate corresponding to the minimum horizontal coordinate. A lane line detection model is used to locate the target object... Lane line recognition is performed on the target image to obtain lane line functions corresponding to each lane line. For any lane line function corresponding to a lane line, the ordinate of the target pixel of the target object is input into the lane line function to obtain the predicted abscissa of the target pixel. Based on the predicted abscissas of the target pixel in each lane line function, as well as the minimum and maximum values ​​of the pixel abscissa, a first lane line and a second lane line are determined. The lane in which the target object is located is determined based on the first lane line and the second lane line. The distance between the target object and the target vehicle is obtained. Based on the distance and the lane in which the target object is located, the position information of the target object is determined. Since the pixel mask coordinates in the target image are obtained based on the recognition model during the target object localization process, the lane in which the target object is located can be calculated through each lane line function. Furthermore, acquiring target images using a target camera is less expensive than acquiring 3D point cloud data using LiDAR, has lower hardware costs, and does not require extensive 3D point cloud data processing, reducing the computational complexity in the target object localization process and improving the target object localization speed. Attached Figure Description

[0072] Figure 1 This is a flowchart illustrating a target object location method in one embodiment;

[0073] Figure 2 This is a flowchart illustrating the process of obtaining the lane line function corresponding to each lane line in one embodiment.

[0074] Figure 3 This is a flowchart illustrating the process of obtaining the corrected lane line pixels corresponding to each lane line in one embodiment.

[0075] Figure 4 This is a flowchart illustrating the process of determining the lane where a target object is located in one embodiment;

[0076] Figure 5This is a schematic diagram illustrating the determination of the lane where the target object is located in one embodiment;

[0077] Figure 6 This is a schematic diagram of the process of acquiring a target image in one embodiment;

[0078] Figure 7 This is a flowchart illustrating the process of obtaining the lane detection model in one embodiment;

[0079] Figure 8 This is a schematic diagram illustrating the process of determining the distance from a target object to a target vehicle in one embodiment;

[0080] Figure 9 This is a structural block diagram of a target object positioning device in one embodiment;

[0081] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0083] In one embodiment, such as Figure 1 As shown, a method for locating a target object is provided. This embodiment illustrates the application of this method to a vehicle terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0084] Step 102: The target camera installed on the target vehicle captures images of the road to obtain the target image.

[0085] The target image includes image data corresponding to at least one lane; the target vehicle is a vehicle equipped with a target camera and a vehicle terminal; the target camera may be, but is not limited to, a monocular camera or a multi-view camera, and any camera device capable of image acquisition can be used in this application. The embodiments of this application do not limit the target camera.

[0086] Step 104: Perform object recognition on the target image using the recognition model to obtain the pixel mask of the target object.

[0087] The recognition model is used to identify the object category of the target object and extract the pixel mask of the target object.

[0088] In this embodiment of the application, the vehicle terminal obtains a recognition model based on the vehicle's visual perception, and the vehicle terminal uses the recognition model to recognize the target image, identify the target object in the target image, and extract the pixel mask of the target object through the recognition model.

[0089] The target image can contain multiple target objects; these objects can be things or pedestrians appearing on the road, such as cars, trucks, bicycles, or pedestrians. This embodiment does not limit the object category. The recognition model can be a model trained by the server on an initial semantic segmentation model based on a large number of sample images containing various target objects; alternatively, the recognition model can be a model based on DDRnet (Real-Time Segmentation Model Network for Road Scenes). The vehicle terminal can download the recognition model from the server via network communication. This embodiment does not limit the recognition model or the method of obtaining it.

[0090] Step 106: Determine the pixel mask coordinates of the target object in the target image.

[0091] The pixel mask coordinate information includes the maximum and minimum values ​​of the pixel horizontal coordinates and the target pixel vertical coordinate. The target pixel vertical coordinate is determined based on the pixel vertical coordinates corresponding to the maximum and minimum values ​​of the pixel horizontal coordinates. The maximum value of the pixel horizontal coordinate is the pixel horizontal coordinate of the target object that is farthest from the horizontal coordinate of the origin of the target image coordinates, and the minimum value of the pixel horizontal coordinate is the pixel horizontal coordinate of the target object that is closest to the horizontal coordinate of the origin of the target image coordinates. The origin of the target image coordinates can be the image pixel located at the lower left corner of the target image. The method for determining the origin of the target image coordinates is not limited in this embodiment.

[0092] In this embodiment of the application, the vehicle terminal extracts the pixel mask of the target object through the recognition model, and obtains the pixel mask coordinate information of the target object in the target image through the image processing algorithm based on the pixel mask of the target object.

[0093] Then, the vehicle terminal determines the target pixel's ordinate based on the ordinate of the pixel corresponding to the maximum and minimum x-coordinates of the pixel mask in the target image. For example, the vehicle terminal averages the ordinates of the pixel corresponding to the maximum and minimum x-coordinates of the pixel mask in the target image to obtain the target pixel's ordinate; alternatively, the vehicle terminal can directly use either the ordinate of the pixel corresponding to the maximum or minimum x-coordinate of the pixel mask in the target image as the target pixel's ordinate.

[0094] For the target pixel ordinate, any pixel ordinate between the pixel ordinate corresponding to the maximum value of the pixel x-coordinate and the pixel ordinate corresponding to the minimum value of the pixel x-coordinate can be used as the target pixel ordinate. This application does not specifically limit this.

[0095] In this application, any algorithm that can determine the pixel mask coordinates of the target object in the target image based on the pixel mask and the target image can be applied to the image processing algorithm. The embodiments of this application do not limit the algorithm.

[0096] Step 108: Use the lane line detection model to identify lane lines in the target image and obtain the lane line function corresponding to each lane line.

[0097] In this embodiment, the vehicle terminal acquires a lane detection model and inputs the target image into the lane detection model. Then, the lane detection model performs lane line recognition on the target image to obtain the lane line pixels corresponding to each lane line. The vehicle terminal determines the lane line function corresponding to each lane line based on the lane line pixels.

[0098] The lane detection model can be a LaneNet-based model (an open-source lane detection toolkit) trained by the server on a large number of sample images containing lane lines; alternatively, it can be a LaneNet-based model pre-trained on various platforms by those skilled in the art. The vehicle terminal can download the lane detection model from the server via network communication. This application does not limit the lane detection model or the method of obtaining it. The lane function can be a linear slope-intercept function of the form y = kx + b or other forms of multivariate polynomial functions; this application does not specifically limit the type of lane function.

[0099] Step 110: For any lane line function, input the target pixel ordinate of the target object into the lane line function to obtain the predicted abscissa corresponding to the target pixel ordinate.

[0100] In this embodiment, the vehicle terminal substitutes the target pixel ordinate of the target object into each lane line function to obtain the predicted abscissa of each lane line function corresponding to the target pixel ordinate.

[0101] Step 112: Based on the predicted abscissas corresponding to the vertical coordinates of the target pixel in each lane line function, as well as the minimum and maximum values ​​of the pixel abscissas, determine the first lane line and the second lane line, and determine the lane where the target object is located based on the first lane line and the second lane line.

[0102] In this embodiment, the vehicle terminal compares the predicted abscissas corresponding to the vertical coordinates of the target pixel with the minimum abscissa value in each lane line function, obtaining a comparison result. Based on the order of the predicted abscissas and the minimum abscissa value in the comparison result, the first lane line is determined. Then, the vehicle terminal compares the predicted abscissas corresponding to the vertical coordinates of the target pixel with the maximum abscissa value, obtaining a comparison result. Based on the order of the predicted abscissas and the maximum abscissa value in the comparison result, the second lane line is determined.

[0103] The vehicle terminal determines the lane where the target object is located based on the first lane line and the second lane line. If the first lane line and the second lane line are far apart, the vehicle terminal determines that the target object is located in the lane encompassed by the first lane line and the second lane line.

[0104] For example, refer to Figure 5 As shown, when the first lane line is lane two and the second lane line is lane four, the vehicle terminal determines that the target object is located in lane two or three.

[0105] It is understandable that the target object may exist in multiple lanes because it may change lanes during its journey.

[0106] Step 114: Obtain the distance between the target object and the target vehicle, and determine the location information of the target object based on the distance and the lane in which the target object is located.

[0107] In this embodiment, the vehicle terminal calculates the distance between the target object and the target vehicle based on the target camera parameters and the height of the target object. Then, the vehicle terminal determines the location information of the target object based on the distance between the target object and the target vehicle and the lane in which the target object is located, thereby achieving the localization of the target object.

[0108] Optionally, the vehicle terminal can determine the lane where the target vehicle is located based on the target image. For example, the vehicle terminal can determine the lane where the target vehicle is located in the target image based on the installation position of the target camera and the center position of the target image. The vehicle terminal can also invoke an image processing algorithm to obtain the pixel mask coordinates of the target vehicle's pixel mask in the target image, and determine the target vehicle's pixel coordinates and lane line function based on the pixel coordinates corresponding to the maximum and minimum values ​​of the target vehicle's pixel horizontal coordinates, thereby determining the lane where the target vehicle is located. The specific methods for determining the lane have been described in detail in the foregoing embodiments and will not be repeated here.

[0109] Regarding the method for determining the lane where the target vehicle is located, any algorithm capable of obtaining the lane where the target vehicle is located can be applied in this application, and the embodiments of this application do not limit this.

[0110] In the aforementioned target object localization method, a target camera installed on the target vehicle acquires images of the road to obtain a target image, which includes image data corresponding to at least one lane line. An object recognition model is used to perform object recognition on the target image to obtain the pixel mask of the target object. The pixel mask coordinate information corresponding to the pixel mask of the target object in the target image is determined. The pixel mask coordinate information includes the maximum and minimum values ​​of the pixel horizontal coordinate and the target pixel vertical coordinate. The target pixel vertical coordinate is determined based on the pixel vertical coordinate corresponding to the maximum and minimum values ​​of the pixel horizontal coordinate. A lane line detection model is then used... Lane line recognition is performed on the target image to obtain the lane line function corresponding to each lane line. For any lane line function, the target pixel's ordinate is input into the lane line function to obtain the predicted abscissa corresponding to the target pixel's ordinate. Based on the predicted abscissas corresponding to the target pixel's ordinate in each lane line function, as well as the minimum and maximum values ​​of the pixel's abscissa, the first and second lane lines are determined, and the lane in which the target object is located is determined based on the first and second lane lines. The distance between the target object and the target vehicle is obtained, and the target object's position information is determined based on the distance and the lane in which the target object is located. Since the pixel mask coordinates in the target image are obtained based on the recognition model during the target object localization process, the lane in which the target object is located can be calculated using each lane line function. Furthermore, acquiring target images using a target camera is less expensive than acquiring 3D point cloud data using LiDAR, has lower hardware costs, and eliminates the need for extensive 3D point cloud data processing, reducing the computational complexity in the target object localization process and improving the target object localization speed.

[0111] In one embodiment, such as Figure 2As shown, the lane detection model includes a detection layer, a correction layer, and a fitting layer. Step 108 includes:

[0112] Step 202: In the detection layer, lane line recognition is performed on the target image to obtain the lane line pixels corresponding to each lane line.

[0113] In this embodiment, the vehicle terminal performs lane line recognition on the target image through the detection layer of the lane line detection model, and obtains the set of instantiated pixels of each lane line corresponding to each lane line, i.e., lane line pixels, based on the shape of each lane line.

[0114] In this embodiment, the lane line pixels corresponding to each lane line can be a discrete set of two-dimensional pixels. The specific shape of the lane line pixels is not limited.

[0115] Step 204: In the correction layer, the inverse projection matrix is ​​determined based on the camera matrix of the target camera. The lane line pixels corresponding to each lane line are corrected based on the inverse projection matrix to obtain the corrected lane line pixels corresponding to each lane line.

[0116] Since the target image is an image containing lane lines captured by the target camera during the movement of the target vehicle, the lane lines in the target image are generally curved or inclined conical shapes from near to far. In the real world, the two lane lines of a lane are mostly parallel curves or straight lines. Therefore, this embodiment of the application uses a correction layer of the lane line detection model to restore the lane lines in the target image to their true shape from a top-down view perspective.

[0117] In this embodiment, the vehicle terminal calculates the inverse projection matrix based on the camera matrix, and obtains the correction layer of the lane detection model based on the inverse projection matrix. The vehicle terminal then performs correction processing on the lane line pixels corresponding to each lane line based on the camera matrix of the target camera, transforming the lane line pixels in the target image into the true shape of the lane lines from a top-down view, thus obtaining the corrected lane line pixels for each lane line.

[0118] Step 206: In the fitting layer, the modified lane line pixels corresponding to each lane line are fitted to obtain the lane line function corresponding to each lane line.

[0119] In this embodiment, the vehicle terminal uses the fitting layer of the lane line detection model to fit the corrected lane line pixels corresponding to each lane line in the top-view perspective, thereby obtaining the corrected lane lines corresponding to each lane line in the top-view perspective. In another implementation, the vehicle terminal can also use the least squares method to perform curve fitting on the corrected lane line pixels corresponding to each lane line in the top-view perspective, thereby obtaining the corrected lane lines corresponding to each lane line.

[0120] Among them, the modified lane lines corresponding to each lane line are used to represent the true shape of the lane lines.

[0121] The vehicle terminal calculates the lane line function corresponding to each lane line by using the pixel coordinates of each corrected lane line corresponding to each lane line, and assigns the data identifiers in a preset order, for example, numbering each lane line function.

[0122] For example, when there are four lane lines, the lane lines are numbered as Lane 1, Lane 2, Lane 3, and Lane 4, and the lane line functions are y1, y2, y3, and y4, respectively. Since each lane consists of two lane lines, there are three lanes in total when there are four lane lines, and the lanes are numbered as Lane 1, Lane 2, and Lane 3, respectively.

[0123] The data identifier can be a numeric identifier, a character identifier, etc., and this application embodiment does not make specific limitations on the data identifier.

[0124] Optionally, after obtaining the corrected lane lines corresponding to each lane line, the vehicle terminal obtains the pixel spacing between any two lane lines. If the pixel spacing between the two lane lines is less than or equal to a pixel spacing threshold, the vehicle terminal can use a preset fitting algorithm to fit the two lane lines to obtain a target lane line. Alternatively, the vehicle terminal can delete either of the two lane lines to obtain a target lane line. The vehicle terminal then replaces the two lane lines with the target lane line and calculates the lane line function of the target lane line.

[0125] In this embodiment, a lane detection model is used to identify lane lines in the target image. The lane line pixels are corrected to obtain the lane line shape in the real world. Then, the corrected lane line pixels are fitted to a lane line function to obtain the lane line function in the real world. Furthermore, in existing technologies, H-net (pixel fitting network) is typically used to fit the lane line pixels to obtain the lane line shape in the image. However, H-net results in lane lines with a roughly conical shape, which is not conducive to determining the lane where the target object is located. Therefore, based on the camera matrix, an inverse projection matrix is ​​obtained, and then the lane line pixels corresponding to each lane line are corrected to transform the lane line pixels in the target image into the true shape of the lane lines from a top-down view. Since the camera matrix is ​​a pre-calibrated and precise camera matrix, the accuracy of the lane line pixel correction process is improved compared to the lane line shape generated by H-net. This facilitates subsequent determination of the lane where the target object is located based on the lane line function and the pixel mask of the target object.

[0126] In one embodiment, such as Figure 3 As shown, step 204 includes:

[0127] Step 302: Obtain the camera matrix of the target camera, and determine the inverse projection matrix based on the camera matrix.

[0128] In this embodiment of the application, the vehicle terminal can obtain the camera matrix of the target camera by having a technician perform camera calibration on the target camera; the vehicle terminal can also pre-store the camera matrix of the target camera.

[0129] The vehicle terminal calculates the inverse projection matrix based on the intrinsic and extrinsic parameters of the camera matrix. The inverse projection matrix is ​​used to obtain the corrected pixel image coordinates of the top-view perspective corresponding to the image coordinates of each pixel.

[0130] The camera matrix can include intrinsic and extrinsic parameter matrices for the target camera. The camera matrix can be used to convert between image coordinates in the target image and 3D camera coordinates in the camera coordinate system, where image coordinates are the pixel coordinates of a pixel in the image. The inverse projection matrix is ​​used to convert the 3D camera coordinates in the camera coordinate system into image coordinates from a top-down view. Any algorithm capable of calculating the inverse projection matrix can be applied in this application, and the embodiments of this application do not limit this application.

[0131] Step 304: Determine the mapping relationship between image pixels and image coordinates, and based on the mapping relationship and the lane line pixels of each lane line, determine the corresponding image coordinates of the lane line pixels of each lane line in the target image.

[0132] In this embodiment of the application, the vehicle terminal can call an image processing algorithm to obtain the mapping relationship between image pixels and image coordinates. Based on the mapping relationship, the lane line pixels of each lane line in the target image are converted into the image coordinates corresponding to the lane line pixels of each lane line.

[0133] In this application, any image processing algorithm that has the function of extracting image pixel coordinates can be used, such as a pixel coordinate acquisition algorithm. This application does not limit the specific algorithm used in this application.

[0134] Step 306: Based on the inverse projection matrix and the image coordinates of the lane line pixels in the target image, correct each lane line pixel to obtain the corrected lane line pixels for each lane line.

[0135] In this embodiment, the vehicle terminal uses an inverse projection matrix to convert the image coordinates of the lane line pixels in the target image into corrected lane line pixel image coordinates from a top-down view. The vehicle terminal then uses the top-down view image coordinates of the lane line pixels for each lane line as the image coordinates of the corrected lane line pixels for that lane line.

[0136] In this embodiment, the inverse projection matrix obtained by calculating the camera matrix can convert the image coordinates of lane line pixels in the target image into image coordinates from a top-down view, thereby restoring the true shape of the lane line pixels and facilitating subsequent calculations of the functions of each lane line.

[0137] In one embodiment, such as Figure 4 As shown, step 112 includes:

[0138] Step 402: Compare the predicted x-coordinates corresponding to the vertical coordinates of the target pixel in each lane line function with the minimum value of the pixel x-coordinate. Based on the comparison results, determine the lane line function corresponding to the minimum value of the pixel x-coordinate, and determine the first lane based on the lane line function corresponding to the minimum value of the pixel x-coordinate.

[0139] In this embodiment, the vehicle terminal compares the predicted abscissas corresponding to the vertical coordinates of the target pixel in each lane line function with the minimum abscissa of the target object, obtaining a comparison result. Then, based on the comparison result, the vehicle terminal obtains the predicted abscissa that is less than and closest to the minimum abscissa, and determines the lane line function corresponding to that predicted abscissa. The vehicle terminal uses the lane line corresponding to that lane line function as the first lane line.

[0140] Exemplarily, when there are four lane line functions, the predicted abscissas corresponding to the ordinate y of the target pixel are x1, x2, x3, and x4 respectively, and the minimum abscissa of the target object's pixel is xmin (i.e., the minimum abscissa of the pixel). The vehicle terminal compares each predicted abscissa x1, x2, x3, x4 with the minimum abscissa xmin of the target object's pixel to obtain a comparison result. For example, x1 < xmin < x2 < x3 < x4. Since x1 is the predicted abscissa that is less than xmin and closest to xmin, the vehicle terminal determines the first lane line as the first lane line corresponding to the predicted abscissa x1.

[0141] In another implementation, the vehicle terminal determines between which two predicted abscissas the minimum abscissa of the pixel lies according to the comparison result, determines the two lane line functions corresponding to the two predicted abscissas, and determines the first lane line according to the two lane line functions corresponding to the two predicted abscissas, where the first lane line includes two lane lines.

[0142] Step 404: Compare each predicted abscissa corresponding to the ordinate of the target pixel in each lane line function with the maximum abscissa of the pixel. According to the comparison result, determine the lane line function corresponding to the maximum abscissa of the pixel, and determine the second lane line according to the lane line function corresponding to the maximum abscissa of the pixel.

[0143] In the embodiment of the present application, the vehicle terminal compares each predicted abscissa corresponding to the ordinate of the target pixel in each lane line function with the maximum abscissa of the pixel to obtain a comparison result. Then, the vehicle terminal obtains the predicted abscissa that is greater than the maximum abscissa of the pixel and closest to the maximum abscissa of the pixel according to the comparison result, and determines the lane line function corresponding to the predicted abscissa. The vehicle terminal uses the lane line corresponding to the lane line function as the second lane line.

[0144] Exemplarily, when there are four lane line functions, the predicted abscissas corresponding to the ordinate y of the target pixel are x1, x2, x3, and x4 respectively, and the maximum abscissa of the target object's pixel is xmax (i.e., the maximum abscissa of the pixel). The vehicle terminal compares each predicted abscissa x1, x2, x3, x4 with the maximum abscissa xmax of the target object's pixel to obtain a comparison result. For example, x1 > x2 > xmax > x3 > x4. Since x2 is the predicted abscissa that is greater than xmax and closest to xmax, the vehicle terminal determines the second lane line as the second lane line corresponding to the predicted abscissa x2.

[0145] In another implementation, the vehicle terminal determines, based on the comparison results, which two predicted abscissas the maximum value of the pixel falls between, and then determines the two lane line functions corresponding to the two predicted abscissas. Based on the two lane line functions corresponding to the two predicted abscissas, the vehicle terminal determines a second lane line, which comprises two lane lines.

[0146] Step 406: Determine the lane where the target object is located based on the first lane line and the second lane line.

[0147] In the embodiments of this application, reference is made to Figure 5 As shown, the vehicle terminal determines the lane where the target object is located based on the first lane line and the second lane line. For example, the vehicle terminal determines that the target object is located in lane number one based on the first lane line being lane number one and the second lane line being lane number two.

[0148] Understandably, when the first lane line and the second lane line are far apart, the vehicle terminal determines that the target object is located in a lane encompassed by both the first and second lane lines. For example, if the first lane line is lane number one and the second lane line is lane number three, then the target object is located in lanes one and two.

[0149] In this embodiment, by comparing the predicted horizontal coordinates, the minimum and maximum values ​​of the target object's pixel horizontal coordinates, the first and second lanes corresponding to the target object can be determined. Based on the first and second lanes, the lane in which the target object is located can be obtained. Since only the horizontal coordinates need to be compared, without the need for extensive data computation, this not only improves the real-time performance of target object positioning but also saves computing resources on the vehicle terminal.

[0150] In one embodiment, such as Figure 6 As shown, step 102 includes:

[0151] Step 602: The target camera installed on the target vehicle captures images of the road to obtain an initial image.

[0152] The initial image includes image data corresponding to at least one lane.

[0153] In this embodiment of the application, the vehicle terminal acquires images of the road through a target camera installed on the target vehicle, and obtains an initial image containing at least one lane.

[0154] Step 604: Mask the target region in the initial image to obtain the target image.

[0155] The target image contains road information regions and background information regions. The target region is the background information region in the target image that does not include lane lines.

[0156] In this embodiment of the application, the vehicle terminal uses an image masking processing algorithm to mask the target area in the initial image to obtain the target image.

[0157] Specifically, the vehicle terminal can perform image masking processing on target areas in the initial image that do not include lane lines to obtain the target image. For example, since the initial image contains road and background information, and the vanishing points of the lane lines in the initial image are almost all concentrated above 50% of the initial image's vertical coordinate, the vehicle terminal adjusts the RGB (Red, Green, Blue) color values ​​of the pixels in the top 50% region of the initial image to 0 or 255, thereby masking the target area in the initial image and obtaining the target image.

[0158] In this embodiment, by masking the initial image, background information regions unrelated to lane lines and target objects can be filtered out, retaining only the key road information regions. This reduces background interference in the initial image, thus reducing the inference time of the lane line detection model for the target image. The lane line recognition model can process the target image quickly, improving the recognition speed of the lane line detection model and the real-time performance of target object localization.

[0159] In one embodiment, such as Figure 7 As shown, the above method also includes:

[0160] Step 702: Obtain the sample image dataset.

[0161] The sample image dataset contains multiple sample image data, including image data corresponding to at least one lane line.

[0162] In this embodiment of the application, the server obtains a sample image dataset containing multiple sample image data.

[0163] The sample image data includes lane line data identifiers. For any given sample image data, different image scales are obtained by using an image scaling algorithm and a scaling factor. For example, the server uses an image scaling algorithm to obtain sample image data scaled by 0.8 times and sample image data scaled by 1.25 times.

[0164] The image scaling algorithm and scaling factor can be set by technicians in actual applications, and this application embodiment does not limit them.

[0165] Step 704: Mask the target region in each sample image data to obtain the target image dataset.

[0166] Lane lines are not included in the target area.

[0167] In this embodiment of the application, the server can perform image masking processing on the target area that does not include lane lines in each sample image data to obtain multiple target image data, and construct a target image dataset from the multiple target image data.

[0168] Step 706: Train the initial lane detection model using the sample image data from the target image dataset to obtain the lane detection model.

[0169] The initial lane detection model can be a lane detection model based on Lanenet.

[0170] In this embodiment, the server trains the initial lane detection model based on sample image data from the target image dataset to obtain the lane detection model. Then, the vehicle terminal can download the lane detection model from the server via network communication to obtain the lane detection model.

[0171] The model training method for the initial lane line detection model can be adjusted by technicians in actual applications, and this application embodiment does not impose specific limitations on it.

[0172] In this embodiment, the server performs masking processing on each sample image data, filtering out background information regions unrelated to lane lines and target objects. This accelerates model convergence during the training of the initial lane line detection model. Furthermore, by scaling each sample image, different scales of the same sample image are preserved. Through multi-scale sample image data, more lane line feature information can be extracted, thereby improving lane line detection accuracy. Multi-scale sample image data also facilitates the integration of various target camera sizes, reducing the decrease in sensitivity of the lane line detection model caused by changes in target camera size. The trained lane line detection model can adapt to target images input from most target cameras, improving its applicability.

[0173] In one embodiment, such as Figure 8 As shown, step 114 includes:

[0174] Step 802: Based on the pixel mask of the target object, obtain the maximum and minimum values ​​of the pixel ordinates of the target object.

[0175] In this embodiment, the vehicle terminal obtains the maximum and minimum values ​​of the vertical coordinates of the target object's pixels using an image processing algorithm based on the pixel mask of the target object.

[0176] Step 804: Based on the maximum and minimum pixel ordinate values ​​of the target object, obtain the sensor parameters of the target camera.

[0177] In this embodiment, the vehicle terminal subtracts the maximum and minimum pixel ordinates of the target object to obtain the height of the target object in the image sensor of the target camera, and uses the height of the target object in the image sensor of the target camera as the sensor parameter of the target camera.

[0178] Step 806: Based on the focal length of the target camera, the spatial height of the target object, and the sensor parameters of the target camera, obtain the distance from the target object to the target vehicle.

[0179] In this embodiment of the application, the vehicle terminal can obtain the focal length of the target camera through the camera matrix of the target camera; the vehicle terminal can also pre-store the focal length data of the target camera.

[0180] Then, the vehicle terminal identifies the target image using a recognition model to obtain the object category of the target object. The vehicle terminal determines the spatial height of the target object based on a pre-stored mapping relationship between the object category and the corresponding spatial height; alternatively, the vehicle terminal can directly calculate the spatial height of the target object using the recognition model and spatial height calculation algorithm. This application does not specifically limit the method for obtaining the spatial height of the target object.

[0181] The vehicle terminal calculates the distance from the target object to the target vehicle based on the focal length of the target camera, the spatial height of the target object, and the sensor parameters of the target camera. The calculation method for the distance from the target object to the target vehicle can refer to the following formula, and the specific calculation process is as follows:

[0182]

[0183] Where d represents the distance from the target object to the target vehicle; f represents the focal length of the target camera; h represents the sensor parameters of the target camera; and H represents the spatial height of the target object.

[0184] In this embodiment, the distance from the target object to the target vehicle can be calculated by using the parameters of the target camera and the object category of the target object, without the need for range measurement processing by lidar and 3D point cloud data processing, thus further improving the speed of target object localization.

[0185] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0186] Based on the same inventive concept, this application also provides a target object positioning device for implementing the target object positioning method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more target object positioning device embodiments provided below can be found in the limitations of the target object positioning method described above, and will not be repeated here.

[0187] In one embodiment, such as Figure 9 As shown, a target object positioning device 900 is provided, including: a data acquisition module 902, a first identification module 904, a first determination module 906, a second identification module 908, a prediction module 910, a second determination module 912, and a third determination module 914, wherein:

[0188] The acquisition module 902 is used to acquire images of the road through a target camera installed on the target vehicle to obtain a target image, wherein the target image includes image data corresponding to at least one lane.

[0189] The first recognition module 904 is used to perform object recognition on the target image through a recognition model to obtain the pixel mask of the target object.

[0190] The first determining module 906 is used to determine the pixel mask coordinate information corresponding to the pixel mask of the target object in the target image. The pixel mask coordinate information includes the maximum value of the pixel horizontal coordinate, the minimum value of the pixel horizontal coordinate, and the target pixel vertical coordinate. The target pixel vertical coordinate is determined based on the pixel vertical coordinate corresponding to the maximum value of the pixel horizontal coordinate and the pixel vertical coordinate corresponding to the minimum value of the pixel horizontal coordinate.

[0191] The second recognition module 908 is used to perform lane line recognition on the target image through a lane line detection model to obtain the lane line function corresponding to each lane line.

[0192] The prediction module 910 is used to input the target pixel ordinate of the target object into the lane line function for any lane line to obtain the predicted abscissa corresponding to the target pixel ordinate.

[0193] The second determining module 912 is used to determine the first lane line and the second lane line based on the predicted horizontal coordinates corresponding to the vertical coordinates of the target pixel in each of the lane line functions, as well as the minimum and maximum values ​​of the horizontal coordinates of the pixel, and to determine the lane in which the target object is located based on the first lane line and the second lane line.

[0194] The third determining module 914 is used to obtain the distance between the target object and the target vehicle, and determine the location information of the target object based on the distance and the lane where the target object is located.

[0195] The target object localization device provided in this disclosure can calculate the lane where the target object is located by using lane line functions after identifying the pixel mask coordinates in the target image based on the recognition model during the target object localization process. Furthermore, acquiring target images via a target camera is less costly than acquiring 3D point cloud data using LiDAR, and it eliminates the need for extensive 3D point cloud data processing, reducing the computational complexity of the target object localization process and improving the target object localization speed.

[0196] In one embodiment, the lane detection model includes a detection layer, a correction layer, and a fitting layer, and the second recognition module 908 is specifically used for:

[0197] In the detection layer, lane line recognition is performed on the target image to obtain lane line pixels corresponding to each lane line;

[0198] In the correction layer, the inverse projection matrix is ​​determined according to the camera matrix of the target camera, and the lane line pixels corresponding to each lane line are corrected according to the inverse projection matrix to obtain the corrected lane line pixels corresponding to each lane line.

[0199] In the fitting layer, the modified lane line pixels corresponding to each lane line are fitted to obtain the lane line function corresponding to each lane line.

[0200] In one embodiment, the second identification module 908 is specifically used for:

[0201] Obtain the camera matrix of the target camera, and determine the inverse projection matrix based on the camera matrix;

[0202] Determine the mapping relationship between image pixels and image coordinates, and based on the mapping relationship and the lane line pixels of each lane line, determine the image coordinates corresponding to the lane line pixels of each lane line in the target image.

[0203] Based on the inverse projection matrix and the image coordinates of the lane line pixels in the target image, each lane line pixel is corrected to obtain the corrected lane line pixels corresponding to each lane line.

[0204] In one embodiment, the second determining module 912 is specifically used for:

[0205] In each of the lane line functions, the predicted abscissa corresponding to the vertical coordinate of the target pixel is compared with the minimum value of the pixel abscissa. Based on the comparison result, the lane line function corresponding to the minimum value of the pixel abscissa is determined, and the first lane line is determined based on the lane line function corresponding to the minimum value of the pixel abscissa.

[0206] In each of the lane line functions, the predicted abscissa corresponding to the vertical coordinate of the target pixel is compared with the maximum value of the pixel abscissa. Based on the comparison result, the lane line function corresponding to the maximum value of the pixel abscissa is determined, and the second lane line is determined based on the lane line function corresponding to the maximum value of the pixel abscissa.

[0207] The lane in which the target object is located is determined based on the first lane line and the second lane line.

[0208] In one embodiment, the acquisition module 902 is specifically used for:

[0209] An initial image is obtained by capturing images of the road using a target camera installed on the target vehicle.

[0210] The target region in the initial image is masked to obtain the target image, wherein the lane lines are not included in the target region.

[0211] In one embodiment, the apparatus further includes:

[0212] The acquisition module is used to acquire a sample image dataset, which contains multiple sample image data, including image data corresponding to at least one lane line.

[0213] The masking module is used to mask the target region in each sample image data to obtain a target image dataset, wherein the lane line is not included in the target region;

[0214] The training module is used to train the initial lane detection model based on the sample image data in the target image dataset to obtain the lane detection model.

[0215] In one embodiment, the third determining module 914 is specifically used for:

[0216] Based on the pixel mask of the target object, obtain the maximum and minimum values ​​of the pixel ordinate of the target object;

[0217] Based on the maximum and minimum pixel ordinate values ​​of the target object, the sensor parameters of the target camera are obtained;

[0218] The distance from the target object to the target vehicle is obtained based on the focal length of the target camera, the spatial height of the target object, and the sensor parameters of the target camera.

[0219] Each module in the aforementioned target object positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0220] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a target object localization method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0221] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0222] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0223] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0224] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0225] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0226] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0227] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0228] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for locating a target object, characterized in that, The method includes: A target image is obtained by capturing images of the road using a target camera installed on the target vehicle. The target image includes image data corresponding to at least one lane. The target image is used to perform object recognition by a recognition model to obtain the pixel mask of the target object; The pixel mask coordinate information corresponding to the pixel mask of the target object in the target image is determined. The pixel mask coordinate information includes the maximum value of the pixel horizontal coordinate, the minimum value of the pixel horizontal coordinate, and the target pixel vertical coordinate. The target pixel vertical coordinate is determined based on the pixel vertical coordinate corresponding to the maximum value of the pixel horizontal coordinate and the pixel vertical coordinate corresponding to the minimum value of the pixel horizontal coordinate. The target image is used to identify lane lines by a lane line detection model to obtain the lane line function corresponding to each lane line. For any lane line function corresponding to the lane line, the target pixel ordinate of the target object is input into the lane line function to obtain the predicted abscissa corresponding to the target pixel ordinate. In each of the lane line functions, the predicted abscissa corresponding to the vertical coordinate of the target pixel is compared with the minimum value of the pixel abscissa. Based on the comparison result, the lane line function corresponding to the minimum value of the pixel abscissa is determined, and the first lane line is determined based on the lane line function corresponding to the minimum value of the pixel abscissa. In each of the lane line functions, the predicted abscissa corresponding to the vertical coordinate of the target pixel is compared with the maximum value of the pixel abscissa. Based on the comparison result, the lane line function corresponding to the maximum value of the pixel abscissa is determined, and the second lane line is determined based on the lane line function corresponding to the maximum value of the pixel abscissa. The lane in which the target object is located is determined based on the first lane line and the second lane line; The distance between the target object and the target vehicle is obtained, and the location information of the target object is determined based on the distance and the lane in which the target object is located.

2. The method according to claim 1, characterized in that, The lane detection model includes a detection layer, a correction layer, and a fitting layer. The process of identifying lane lines in the target image using the lane detection model to obtain the lane line function corresponding to each lane line includes: In the detection layer, lane line recognition is performed on the target image to obtain lane line pixels corresponding to each lane line; In the correction layer, the inverse projection matrix is ​​determined according to the camera matrix of the target camera, and the lane line pixels corresponding to each lane line are corrected according to the inverse projection matrix to obtain the corrected lane line pixels corresponding to each lane line. In the fitting layer, the modified lane line pixels corresponding to each lane line are fitted to obtain the lane line function corresponding to each lane line.

3. The method according to claim 2, characterized in that, The step of determining the inverse projection matrix based on the camera matrix of the target camera, and correcting the lane line pixels of each lane line according to the inverse projection matrix to obtain the corrected lane line pixels corresponding to each lane line includes: Obtain the camera matrix of the target camera, and determine the inverse projection matrix based on the camera matrix; Determine the mapping relationship between image pixels and image coordinates, and based on the mapping relationship and the lane line pixels of each lane line, determine the image coordinates corresponding to the lane line pixels of each lane line in the target image. Based on the inverse projection matrix and the image coordinates of the lane line pixels in the target image, each lane line pixel is corrected to obtain the corrected lane line pixels corresponding to each lane line.

4. The method according to claim 1, characterized in that, The process of acquiring target images of the road using a target camera installed on the target vehicle includes: An initial image is obtained by capturing images of the road using a target camera installed on the target vehicle. The target region in the initial image is masked to obtain the target image, wherein the lane lines are not included in the target region.

5. The method according to claim 1, characterized in that, The method further includes: Obtain a sample image dataset, which contains multiple sample image data, including image data corresponding to at least one lane line; The target region in each sample image data is masked to obtain a target image dataset, wherein the lane line is not included in the target region; The initial lane detection model is trained using the sample image data from the target image dataset to obtain the lane detection model.

6. The method according to claim 1, characterized in that, The step of obtaining the distance from the target object to the target vehicle includes: Based on the pixel mask of the target object, obtain the maximum and minimum values ​​of the pixel ordinate of the target object; Based on the maximum and minimum pixel ordinate values ​​of the target object, the sensor parameters of the target camera are obtained; The distance from the target object to the target vehicle is obtained based on the focal length of the target camera, the spatial height of the target object, and the sensor parameters of the target camera.

7. A target object positioning device, characterized in that, The device includes: The acquisition module is used to acquire images of the road through a target camera installed on the target vehicle to obtain a target image, wherein the target image includes image data corresponding to at least one lane. The first recognition module is used to perform object recognition on the target image through a recognition model to obtain the pixel mask of the target object; The first determining module is used to determine the pixel mask coordinate information corresponding to the pixel mask of the target object in the target image. The pixel mask coordinate information includes the maximum value of the pixel horizontal coordinate, the minimum value of the pixel horizontal coordinate, and the target pixel vertical coordinate. The target pixel vertical coordinate is determined based on the pixel vertical coordinate corresponding to the maximum value of the pixel horizontal coordinate and the pixel vertical coordinate corresponding to the minimum value of the pixel horizontal coordinate. The second recognition module is used to perform lane line recognition on the target image through a lane line detection model to obtain the lane line function corresponding to each lane line. The prediction module is used to input the target pixel ordinate of the target object into the lane line function for any lane line to obtain the predicted abscissa corresponding to the target pixel ordinate; The second determining module is used to compare the predicted abscissas corresponding to the vertical coordinates of the target pixel in each of the lane line functions with the minimum value of the pixel abscissa, and determine the lane line function corresponding to the minimum value of the pixel abscissa based on the comparison result, and determine a first lane line based on the lane line function corresponding to the minimum value of the pixel abscissa; compare the predicted abscissas corresponding to the vertical coordinates of the target pixel in each of the lane line functions with the maximum value of the pixel abscissa, and determine the lane line function corresponding to the maximum value of the pixel abscissa based on the comparison result, and determine a second lane line based on the lane line function corresponding to the maximum value of the pixel abscissa; and determine the lane in which the target object is located based on the first lane line and the second lane line. The third determining module is used to obtain the distance between the target object and the target vehicle, and determine the location information of the target object based on the distance and the lane in which the target object is located.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images