Method, system and equipment for positioning and rectifying deviation of cart based on lane lines and spikes and medium

Abstract

In order to solve the problems of insufficient precision, low reliability, inconvenience in installation and maintenance and high cost in the prior art, the invention provides a lane line and spike-based cart positioning correction method, system, equipment and medium. The method comprises the following steps that a gantry crane cart operation site is preprocessed, lane lines are drawn, and spikes are installed in the position parallel to the lane lines; rFID tags are implanted into the spikes; cameras are installed at the positions, close to legs, of the sea side and the land side of a cart, a support is installed on the ground, a camera assembly vertically facing the ground is installed on the support, and the camera assembly collects spike and lane line image data; calibrating and correcting the camera component by using a computer vision method, and outputting internal parameters, external parameters and a transmission correction mapping matrix of the camera; and calculating the offset of the cart and the correction value of each foot in real time based on the internal parameters, the external parameters and the transmission correction mapping matrix of the camera, the YOLOv8 spike detection model and the YOLOv8 lane line segmentation model.

Description

TECHNICAL FIELD

[0001] The present application relates to the positioning and deviation correction of gantry crane, and in particular to a gantry positioning and deviation correction method, system, device and medium based on lane lines and spikes. BACKGROUND

[0002] Automatic wharf operation is a mode of operation in which technical equipment and intelligent systems replace manual work to automatically complete the entire process of unloading, transferring and storing goods (mainly containers) on the wharf for loading on ships. The core is to reduce manual intervention and improve efficiency and accuracy through "machine autonomous operation + system unified scheduling".

[0003] In the existing automatic wharf operation scenario, the positioning of the gantry crane direction and the deviation correction during operation are basic functions, but both the deviation correction laser and the GPS face the problems of insufficient accuracy, low reliability, inconvenience of installation and maintenance, and high cost. SUMMARY

[0004] To solve the problems of insufficient accuracy, low reliability, inconvenience of installation and maintenance, and high cost of both the deviation correction laser and the GPS in the prior art, the present application provides a gantry positioning and deviation correction method, system, device and medium based on lane lines and spikes.

[0005] The technical solution adopted by the present application is as follows: The gantry positioning and deviation correction method based on lane lines and spikes provided by the present application performs the following steps: Step S100, preprocessing the gantry crane operation site, drawing lane lines according to the drawings, and installing spikes at intervals of 3.2m parallel to the lane lines 1m apart; RFID tags are implanted in the spikes to record unique identification information to facilitate subsequent binding operations; Cameras are installed on the sea side and the land side of the gantry near the leg positions, and supports are installed at a distance of 1.4m from the ground, wherein 1.4m is the distance from the camera to the ground; A camera assembly is installed on the support and faces vertically towards the ground, the camera assembly collects spike and lane line image data and outputs them; The camera assembly is calibrated and corrected using a computer vision method, and the camera intrinsic parameters, extrinsic parameters and transmission correction mapping matrix are outputted; The intrinsic parameters include focal length and distortion coefficient, and the extrinsic parameters include the rotation matrix and translation vector of the camera in the world; The YOLOv8 spike detection model and the YOLOv8 lane line segmentation model are initialized, and the YOLOv8 spike detection model and the YOLOv8 lane line segmentation model are trained using the spike and lane line image data collected by the camera assembly; Step S200, based on the camera internal parameter, external parameter and transmission correction mapping matrix output by step S100, the trained YOLOv8 spike detection model and the trained YOLOv8 lane line segmentation model, the offset of the large vehicle and the correction value of each foot are calculated and output in real time.

[0006] Further, in step S100, the computer vision method is used for calibration and correction processing of the installed camera assembly. The computer vision method is an OpenCV-based calibration board calibration method. The specific method is: using black and white checkerboard and multi-angle sampling of the camera to be corrected, and combining image correction tool analysis data.

[0007] Further, step S200 specifically performs the following sub-steps: Step S210, the camera internal parameter and external parameter output by step S100 are subjected to mapping matrix calculation processing, and the initUndistortRectifyMap function in OpenCV is used for operation, and the mapping matrix x_map_dst and y_map_dst of distortion and perspective transformation are output; Step S220, set ROI and scaling parameters; Wherein, ROI is the useful field of view of the camera covering 2-3 spike range and lane line; Loop calling the image data output by step S100, and storing the image data in a mat type array; Step S230, the mat type array storing the image data in step S220 is subjected to correction and acceleration processing, first uploaded to gpumat, then the mapping matrix x_map_dst and y_map_dst output by step S210 are used to call remap function to do homography transformation, at the same time, cuda module is used to accelerate remap, and openmp is used to realize multi-thread operation, and the corrected image data is output; Step S240, the corrected image data output by step S230 and the camera-to-ground distance are subjected to proportional calculation processing, combined with the camera internal parameter output by step S100, the conversion ratio ratio of pixel coordinates and world coordinates is calculated and output; Step S250, based on the corrected image data output by step S230 and the conversion ratio ratio output by step S240, the YOLOv8 spike detection model trained in step S200 is used to detect the position of the spike and bind with the spike, and the real-time position of the large vehicle is calculated and output; Step S260, based on the corrected image data output by step S230 and the conversion ratio ratio output by step S240, the YOLOv8 lane line segmentation model trained in step S200 is used to detect the lane line and output the sea-land side position and angle of the lane line; Step S270, according to the output of the cart real-time position in step S250 and the land side position and angle of the lane line output in step S260, the offset of the cart and the correction value of each foot are calculated and output; Step S280, steps S210 to S270 are repeatedly executed, and the offset of the cart and the correction value of each foot are continuously output by cyclic processing for each frame of image.

[0008] Further, the formula for calculating the conversion ratio ratio of pixel coordinates and world coordinates is: ratio=1400mm÷f; Wherein, f is the focal length, unit: pixel.

[0009] Further, step S250 specifically executes the following sub-steps: Step S251, calling the YOLOv8 spike detection model after initial training, the corrected image data output in step S230 is subjected to spike detection processing, and the pixel coordinates and size of each spike in the corrected image data are obtained; According to the ROI, 2-3 spikes meeting the requirements are selected; Wherein, if the corrected image data is the first frame, the Kalman filter is initialized according to the position of each spike, otherwise the correctness of the spike position is judged according to the output of the Kalman filter, if correct, the state of the Kalman filter is updated according to the position of each spike, if incorrect, the detection position of the spike is replaced by the prediction value of the Kalman filter; According to the spike pixel coordinates and the conversion ratio ratio, the spike world coordinates are obtained; According to the difference between the corrected image data of the current frame and the corrected image data of the last frame, the direction of the cart movement is calculated; When a new spike RFID is detected, the new spike world coordinates are bound with the RFID according to the direction of the cart movement, and the spike position-RFID binding information is output; Step S252: calculating and outputting the real-time position of the cart according to the spike position-RFID binding information output in step S251; Wherein, the real-time position of the cart is the position based on the world coordinate system.

[0010] Further, step S260 specifically executes the following sub-steps: Step S261, calling the trained YOLOv8 lane line segmentation model, the corrected image data output in step S230 is subjected to lane line segmentation processing, and the lane line mask is obtained; Step S262, according to the area of the lane line mask, one optimal lane line mask meeting the conditions is selected according to the ROI; Step S263, using a least square fitting algorithm to fit a straight line to the mask of the lane line screened out in step S262 to obtain a straight line equation; According to the straight line equation, an intercept and a straight line angle are obtained; According to the intercept and the conversion ratio ratio, a land side position of the lane line is obtained; The land side position and the angle of the lane line are output.

[0011] Further, step S262 specifically performs the following sub-steps: Step S2611, calculating the proportion of white pixels in each mask within the ROI range set in step S220 for all lane line masks output in step S261.

[0012] Masks with a proportion greater than or equal to a preset threshold are retained, and masks with a proportion less than the preset threshold are removed; Step S2612, calculating the area of each mask for the masks retained in step S2611; A lower limit threshold of the area is set, and masks with an area greater than or equal to the lower limit threshold are retained, and masks with an area less than the threshold are removed; Step S2613, comparing the areas of the remaining masks for the masks retained in step S2612, and selecting the mask with the largest area; the mask with the largest area is the optimal lane line mask.

[0013] Based on the same inventive concept, the present application also provides a large vehicle positioning and deviation correction system based on lane lines and studs, which executes the aforementioned large vehicle positioning and deviation correction method based on lane lines and studs, comprising: A data acquisition module inputs camera intrinsic parameters, extrinsic parameters and real-time image data, performs collection and preliminary arrangement, and outputs camera intrinsic parameters, extrinsic parameters and image data; A mapping matrix calculation module inputs the camera intrinsic parameters and extrinsic parameters output by the data acquisition module, calls the initUndistortRectifyMap function of OpenCV to perform mapping matrix calculation, and outputs the mapping matrices x_map_dst and y_map_dst of distortion and perspective transformation; An image correction acceleration module inputs image data, mapping matrices, a preset ROI and scaling parameters, the ROI is a useful field of view covering 2-3 studs range and lane lines, uploads the image data to a mat type array after storing it in the gpumat, calls the remap function in combination with the mapping matrices to perform homography transformation, and outputs corrected image data through cuda module acceleration and openmp multi-thread operation; A coordinate ratio calculation module inputs corrected image data, camera-to-ground distance and camera intrinsic parameters, performs ratio calculation, and outputs a conversion ratio ratio of pixel coordinates and world coordinates; The stud positioning module inputs the corrected image data, the ratio and the trained YOLOv8 stud detection model, detects the stud position and binds, and outputs the real-time position of the cart; The lane line detection module inputs the corrected image data, the ratio and the trained YOLOv8 lane line segmentation model, detects the lane line, and outputs the land-sea side position and angle thereof. The offset correction calculation module inputs the real-time position of the cart, the land-sea side position and angle of the lane line, calculates and continuously outputs the offset amount of the cart and the correction value of each foot.

[0014] Based on the same inventive concept, the application also provides a computer device, comprising a memory and a processor, which are communicatively connected with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the aforementioned positioning and deviation correction method for the cart based on lane lines and studs.

[0015] Based on the same inventive concept, the application also provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the aforementioned positioning and deviation correction method for the cart based on lane lines and studs.

[0016] The application has the following beneficial effects: The positioning and deviation correction method, system, device and medium for the cart based on lane lines and studs disclosed in the application can realize real-time identification of pre-buried studs and lane line marks through a camera installed on a gantry crane, calculate the accurate position and heading deviation of the cart by combining image processing and pattern recognition algorithms, and correct the cart in real time according to the deviation, so that the application has the characteristics of low cost, convenient maintenance and high deviation correction accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0018] Figure 1 The flowchart of the positioning and deviation correction method for the cart based on lane lines and studs; Figure 2 The block diagram of the positioning and deviation correction system for the cart based on lane lines and studs. DETAILED DESCRIPTION

[0019] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0020] The following disclosure provides many different embodiments or examples for implementing different structures of the present application. In order to simplify the disclosure of the present application, the components and settings of specific examples are described below. Of course, they are only examples and the purpose is not to limit the present application.

[0021] The embodiments of the application will be described in detail below with reference to the accompanying drawings.

[0022] As shown in the accompanying Figure 1 The lane line and stud-based large vehicle positioning deviation correction method provided by the present embodiment performs the following steps: Step S100, pretreat the gantry crane large vehicle running site, draw lane lines according to drawings, and install studs at intervals of 3.2m parallel to the lane lines 1m apart; RFID tags are implanted in the studs to record unique identification information and lay the foundation for subsequent binding operations; Install cameras on the sea side and land side of the large vehicle near the leg position, and install a bracket at a distance of 1.4m from the ground, wherein 1.4m is the distance from the camera to the ground; Install a camera assembly vertically facing the ground on the bracket, which collects stud and lane line image data and outputs them; Calibrate and correct the camera assembly using computer vision methods, and output the camera intrinsic parameters, extrinsic parameters, and transmission correction mapping matrix; The intrinsic parameters include focal length and distortion coefficient; the extrinsic parameters include the rotation matrix and translation vector of the camera in the world; Initialize the YOLOv8 stud detection model and the YOLOv8 lane line segmentation model, and use the stud and lane line image data collected by the camera assembly to train the YOLOv8 stud detection model and the YOLOv8 lane line segmentation model; Step S200, based on the camera intrinsic parameters, extrinsic parameters, and transmission correction mapping matrix output by step S100, the trained YOLOv8 stud detection model, and the trained YOLOv8 lane line segmentation model, real-time calculation and continuous output of the offset of the large vehicle and the correction value of each foot.

[0023] Further, in step S100, the computer vision method is used for calibration correction processing of the installed camera assembly. The computer vision method is an OpenCV-based calibration board calibration method. The specific method is as follows: a black and white checkerboard is used for multi-angle sampling of the camera to be corrected, and image correction tools are used to analyze the data.

[0024] Further, step S200 specifically performs the following sub-steps: Step S210, mapping matrix calculation processing is performed on the camera intrinsic and extrinsic parameters output by step S100, and the initUndistortRectifyMap function in OpenCV is used for operation, and the mapping matrix x_map_dst and y_map_dst of distortion and perspective transformation are output; Step S220, setting ROI and scaling parameters; Wherein, ROI is the useful field of view of the camera covering 2-3 spike ranges and lane lines; Looping the image data output by step S100, the image data is stored in a mat type array; Step S230, correcting and accelerating the mat type array storing image data in step S220, uploading to gpumat first, then using the mapping matrix x_map_dst and y_map_dst output by step S210 to call the remap function to do homography transformation, using the cuda module to accelerate remap, and using openmp to realize multi-thread operation, outputting the corrected image data; Step S240, proportional calculation processing is performed on the corrected image data output by step S230 and the camera-to-ground distance, and the camera intrinsic parameters output by step S100 are combined to calculate and output the conversion ratio ratio of pixel coordinates and world coordinates; Step S250, based on the corrected image data output by step S230 and the conversion ratio ratio output by step S240, the YOLOv8 spike detection model trained in step S200 is used to detect the position of the spike and bind with the spike, and the real-time position of the large vehicle is calculated and output; Step S260, based on the corrected image data output by step S230 and the conversion ratio ratio output by step S240, the YOLOv8 lane line segmentation model trained in step S200 is used to detect the lane line and output the sea-land side position and angle of the lane line; Step S270, according to the real-time position of the large vehicle output by step S250 and the sea-land side position and angle of the lane line output by step S260, the offset of the large vehicle and the correction value of each foot are calculated and output; Step S280, repeat steps S210 to S270, loop processing for each frame of image, continuously output the offset of the cart and the correction value of each foot.

[0025] Further, the formula for calculating the conversion ratio ratio of pixel coordinates and world coordinates is: ratio = 1400 mm ÷ f; Where f is the focal length, in pixels.

[0026] Further, step S250 specifically performs the following sub-steps: Step S251, call the YOLOv8 spike detection model after initial training to do spike detection processing on the corrected image data output by step S230, to get each spike pixel coordinate and size in the corrected image data; According to the ROI, 2-3 spikes meeting the requirements are selected; Wherein, if the corrected image data is the first frame, the Kalman filter is initialized according to the position of each spike, otherwise the correctness of the spike position is judged according to the output of the Kalman filter, if correct, the state of the Kalman filter is updated according to the position of each spike, if incorrect, such as spike missed detection or false detection, the predicted value of the Kalman filter is used to replace the detection position of the spike; According to the spike pixel coordinates and the conversion ratio ratio, the spike world coordinates (unit: mm) are obtained; According to the difference between the current frame of corrected image data and the last frame of corrected image data, the direction of the cart movement is calculated; When a new RFID of the spike is detected, the new spike world coordinates are bound with the RFID according to the direction of the cart movement, and the spike position-RFID binding information is output; Step S252: calculate and output the real-time position of the cart based on the spike position-RFID binding information output by step S251; Wherein, the real-time position of the cart is the position based on the world coordinate system.

[0027] Further, step S260 specifically performs the following sub-steps: Step S261, call the trained YOLOv8 lane line segmentation model to do lane line segmentation processing on the corrected image data output by step S230, to get the lane line mask; Step S262, according to the area of the lane line mask, select one optimal lane line mask meeting the conditions according to the ROI; Step S263, use the least square fitting algorithm to fit a straight line to get the straight line equation for the lane line mask selected by step S262; According to the linear equation, the intercept and the linear angle are obtained; According to the intercept and the conversion ratio, the sea-land side positions of the lane line are obtained; The sea-land side positions and the angle of the lane line are output.

[0028] Further, step S262 specifically performs the following sub-steps: Step S2611 (preliminary filtering of the mask based on the ROI range), the proportion of the white pixels (lane line area) in each mask in the ROI range set in step S220 (i.e., "the number of white pixels in the ROI ÷ the total number of white pixels in the mask") is calculated for all lane line masks (each mask is a binary image, and the white pixels represent the lane line area) output in step S261.

[0029] The masks with a proportion greater than or equal to a preset threshold (such as 80%, which can be adjusted according to the scene) are retained, and the masks with a proportion less than the preset threshold (such as ground stains, other irrelevant markings) are removed; Step S2612 (screening of the effective mask based on the area), the area of each mask (i.e., the total number of white pixels in the mask, reflecting the visible range of the lane line in the image) is calculated for the masks retained in step S2611; The area lower limit threshold (such as ≥ 50 pixels, which is set according to the camera resolution and the actual width of the lane line) is set, the masks with an area greater than or equal to the lower limit threshold are retained, and the masks with an area less than the threshold (such as noise, short line segments that are false detections, which have no actual reference value) are removed; Step S2613 (determination of the optimal mask), the areas of the masks retained in step S2612 are compared, and the mask with the largest area is selected; the mask with the largest area is the optimal lane line mask (because it usually corresponds to the most complete and clearest lane line part in the image, and can more accurately reflect the real position and angle of the lane line).

[0030] As shown in the accompanying Figure 2 Based on the same inventive concept, the embodiment also provides a large vehicle positioning and deviation correction system based on lane lines and studs, which executes the aforementioned large vehicle positioning and deviation correction method based on lane lines and studs, and includes: A data acquisition module 1 inputs camera intrinsic parameters, extrinsic parameters, and real-time image data, performs acquisition and preliminary arrangement, and outputs the camera intrinsic parameters, extrinsic parameters, and image data; A mapping matrix calculation module 2 inputs the camera intrinsic parameters and extrinsic parameters output by the data acquisition module 1, calls the initUndistortRectifyMap function of OpenCV to perform mapping matrix calculation, and outputs the mapping matrices x_map_dst and y_map_dst of distortion and perspective transformation; The image correction acceleration module 3 inputs image data, a mapping matrix, a preset ROI, and a scaling parameter, the ROI is a useful field of view covering 2-3 stud ranges and lane lines in the transverse direction, stores the image data in a mat type array, uploads the image data to a gpumat, calls a remap function to perform homography transformation in combination with the mapping matrix, performs acceleration through a cuda module and multi-threaded operation through openmp, and outputs corrected image data; The coordinate scaling calculation module 4 inputs corrected image data, a camera-to-ground distance, and camera intrinsic parameters, performs scaling calculation, and outputs a conversion ratio ratio of pixel coordinates and world coordinates; The stud positioning module 5 inputs corrected image data, the ratio, and a trained YOLOv8 stud detection model, detects and binds stud positions, and outputs real-time positions of the large vehicle; The lane line detection module 6 inputs corrected image data, the ratio, and a trained YOLOv8 lane line segmentation model, detects lane lines, and outputs sea-land side positions and angles of the lane lines; The offset correction calculation module 7 inputs real-time positions of the large vehicle, sea-land side positions and angles of the lane lines, calculates and continuously outputs offset amounts of the large vehicle and correction values of each foot.

[0031] Based on the same inventive concept, the embodiment also provides a computer device, which comprises a memory and a processor, the memory and the processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the foregoing large-vehicle positioning and deviation correction method based on lane lines and studs.

[0032] Based on the same inventive concept, the embodiment also provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the foregoing large-vehicle positioning and deviation correction method based on lane lines and studs.

Claims

1. A method for locating and correcting large vehicles based on lane lines and road studs, characterized in that, The following steps are performed: Step S100, pretreat the gantry crane running site, draw lane lines according to drawings, and install spikes at intervals of 3.2 m parallel to the lane lines 1 m apart; RFID tags are implanted in the spikes to record unique identification information and lay the foundation for subsequent binding operations; Cameras are installed on the sea side and land side of the crane near the leg positions, and supports are installed 1.4 m from the ground, wherein 1.4 m is the distance from the camera to the ground; A camera assembly vertically facing the ground is installed on the support, which collects spike and lane line image data and outputs; The camera assembly is calibrated and corrected using a computer vision method, and camera intrinsic parameters, extrinsic parameters, and transmission correction mapping matrices are output; The intrinsic parameters include focal length and distortion coefficient; the extrinsic parameters include the rotation matrix and translation vector of the camera in the world; The YOLOv8 spike detection model and the YOLOv8 lane line segmentation model are initialized, and the spike and lane line image data collected by the camera assembly are used to train the YOLOv8 spike detection model and the YOLOv8 lane line segmentation model; Step S200, based on the camera intrinsic parameters, extrinsic parameters, and transmission correction mapping matrices output by step S100, the trained YOLOv8 spike detection model, and the trained YOLOv8 lane line segmentation model, the offset of the crane and the correction value of each foot are calculated and continuously output in real time.

2. The lane line and stud based large vehicle positioning and deviation correction method according to claim 1, wherein, In step S100, the installed camera assembly is calibrated and corrected using a computer vision method, which is an OpenCV-based calibration board calibration method. The specific method is as follows: a black and white chessboard is used for multi-angle sampling with the camera to be corrected, and image correction tools are used to analyze the data.

3. The lane line and stud based large vehicle positioning and deviation correction method according to claim 2, wherein, Step S200 specifically performs the following sub-steps: Step S210, calculate the mapping matrix of the camera intrinsic parameters and extrinsic parameters output by step S100, and input the initUndistortRectifyMap function in OpenCV to perform operations, and output the mapping matrices x_map_dst and y_map_dst of distortion and perspective transformation; Step S220, set ROI and scaling parameters; The ROI is the useful field of view of the camera covering 2-3 spikes and lane lines in the horizontal direction; Loop the image data output by step S100 and store it in a mat type array; Step S230, correct and accelerate the mat type array storing image data in step S220, first upload to gpumat, then use the mapping matrices x_map_dst and y_map_dst output by step S210 to call the remap function for homography transformation, use the cuda module to accelerate remap, and use openmp to realize multi-threaded operation, and output the corrected image data; Step S240, calculate the proportion of the corrected image data output by step S230 and the distance from the camera to the ground, and combine the camera intrinsic parameters output by step S100 to calculate and output the conversion ratio ratio of pixel coordinates and world coordinates; Step S250, based on the corrected image data output in step S230 and the conversion ratio ratio output in step S240, the YOLOv8 spike detection model trained in step S200 is used to detect the position of the spike and bind with the spike, calculate and output the real-time position of the cart; Step S260, based on the corrected image data output in step S230 and the conversion ratio ratio output in step S240, the YOLOv8 lane line segmentation model trained in step S200 is used to detect the lane line and output the land side position and angle of the lane line; Step S270, according to the real-time position of the cart output in step S250 and the land side position and angle of the lane line output in step S260, the offset of the cart and the correction value of each foot are calculated and output; Step S280, repeat steps S210 to S270, loop processing for each frame of image, continuously output the offset of the cart and the correction value of each foot.

4. The lane line and stud based large vehicle positioning and deviation correction method according to claim 3, wherein, The formula for calculating the conversion ratio ratio of pixel coordinates and world coordinates is: ratio=1400mm÷f; Where f is the focal length, unit is pixel.

5. The lane line and stud based large vehicle positioning and deviation correction method according to claim 3, wherein, Step S250 specifically executes the following sub-steps: Step S251, call the YOLOv8 spike detection model after initial training to do spike detection processing on the corrected image data output in step S230, get the pixel coordinates and size of each spike in the corrected image data; According to the ROI, 2-3 spikes meeting the requirements are selected; Wherein, if the corrected image data is the first frame, the Kalman filter is initialized according to the position of each spike, otherwise the correctness of the spike position is judged according to the output of the Kalman filter, if correct, the state of the Kalman filter is updated according to the position of each spike, if not correct, the predicted value of the Kalman filter is used to replace the detected position of the spike; According to the spike pixel coordinates and the conversion ratio ratio, the spike world coordinates are obtained; According to the difference between the current frame of corrected image data and the last frame of corrected image data, the direction of the cart movement is calculated; When a new RFID of the spike is detected, the new spike world coordinates are bound with the RFID according to the direction of the cart movement, and the spike position-RFID binding information is output; Step S252: calculate and output the real-time position of the cart based on the spike position-RFID binding information output in step S251; Wherein, the real-time position of the cart is the position based on the world coordinate system.

6. The lane line and stud based large vehicle positioning and correction method according to any one of claims 3-5, wherein, Step S260 specifically executes the following sub-steps: Step S261, call the trained YOLOv8 lane line segmentation model to do lane line segmentation processing on the corrected image data output in step S230, get the mask of the lane line; Step S262, according to the area of the lane line mask, one optimal lane line mask meeting the conditions is selected according to the ROI; Step S263, the least square fitting algorithm is used to fit a straight line to get the straight line equation according to the mask of the lane line selected in step S262; According to the intercept and the conversion ratio ratio, the land side position of the lane line is obtained; ​ Output the land and sea side position and angle of the lane line.

7. The lane line and stud based large vehicle positioning and deviation correction method according to claim 6, wherein, Step S262 specifically performs the following sub-steps: Step S2611, calculate the proportion of white pixels in the ROI range set in step S220 in each mask for all lane line masks output in step S261; Retain the masks whose proportion is greater than or equal to the preset threshold; Eliminate the masks whose proportion is less than the preset threshold; Step S2612, calculate the area of each mask for the masks retained in step S2611; Set a lower limit threshold for the area, and retain the masks whose area is greater than or equal to the lower limit threshold; Eliminate the masks whose area is less than the threshold; Step S2613, compare the areas of the remaining masks for the masks retained in step S2612, and select the mask with the largest area; the mask with the largest area is the optimal lane line mask.

8. A lane line and stud based positioning and deviation correction system for a vehicle, comprising: The lane line and stud-based large vehicle positioning and deviation correction method according to any one of claims 3-7 is executed, comprising: a data acquisition module that inputs camera intrinsic parameters, extrinsic parameters, and real-time image data, performs collection and preliminary arrangement, and outputs the camera intrinsic parameters, extrinsic parameters, and image data; a mapping matrix calculation module that inputs the camera intrinsic parameters and extrinsic parameters output by the data acquisition module, calls the initUndistortRectifyMap function of OpenCV to perform mapping matrix calculation, and outputs the mapping matrices x_map_dst and y_map_dst for distortion and perspective transformation; an image rectification acceleration module that inputs image data, mapping matrices, a preset ROI, and scaling parameters, the ROI being a useful field of view covering 2-3 studs and lane lines in the horizontal direction, stores the image data in a mat type array, uploads the image data to a gpumat, calls the remap function in combination with the mapping matrices to perform homography transformation, and outputs rectified image data through cuda module acceleration and openmp multi-threading operation; a coordinate scale calculation module that inputs rectified image data, a camera-to-ground distance, and camera intrinsic parameters, performs scale calculation, and outputs a conversion ratio ratio between pixel coordinates and world coordinates; a stud positioning module that inputs rectified image data, ratio, and a trained YOLOv8 stud detection model, detects stud positions and binds them, and outputs real-time positions of the large vehicle; a lane line detection module that inputs rectified image data, ratio, and a trained YOLOv8 lane line segmentation model, detects lane lines, and outputs their land and sea side positions and angles; an offset correction calculation module that inputs real-time positions of the large vehicle, land and sea side positions and angles of the lane lines, and continuously outputs offset amounts of the large vehicle and correction values for each foot.

9. A computer device, comprising: comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the lane line and stud-based large vehicle positioning and deviation correction method according to any one of claims 3-7.

10. A computer-readable storage medium, characterized in that, A computer readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute the lane line and stud-based large vehicle positioning and deviation correction method according to any one of claims 3-7.

Images