Large semi-trailer reversing positioning method and system based on subjective parking space identification

Abstract

The invention discloses a large semi-trailer reversing positioning method and system based on subjective parking space identification, and belongs to the technical field of automatic driving vehicle localization. The method comprises the steps that firstly, a camera is calibrated, camera parameters are obtained, a bird's-eye view is generated through perspective transformation, then, a red ground marking line is recognized, edge points are extracted, and a parking space is obtained; and fitting a straight line by using a least square method, determining an offset distance and a deflection angle between a center line and the vehicle so as to calculate a spatial position between the vehicle and the parking space, and meanwhile, calculating the position of the vehicle relative to the bottom edge of the parking space by combining a matts color identifier beside the parking space, thereby comprehensively realizing vehicle backing positioning. According to the large-scale semi-trailer vehicle positioning method, in the vehicle reversing process, the positioning precision of the large-scale semi-trailer vehicle is high, and a reliable data basis is provided for the large-scale semi-trailer vehicle reversing auxiliary technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic driving vehicle positioning, and in particular relates to a method and system for reversing positioning of a large semi-trailer vehicle based on subjective parking space marking. Background Art

[0002] With the rapid development of intelligent transportation systems, vehicle parking positioning technology has gradually shifted from the traditional experience-based and sensor-based methods to a more intelligent and automated direction. Assisted driving technology has played a key role in this transformation, and its main functions include lane keeping, traffic sign recognition, and vehicle parking positioning. In the study of vehicle parking positioning, domestic and foreign scholars have mainly focused on two major scenarios: outdoor and indoor, and have achieved remarkable results. In outdoor parking scenarios, researchers have achieved high positioning accuracy by integrating multiple high-precision maps, lidar, and GNSS (Global Navigation Satellite System) signals. However, these methods still face the problem of reliability verification when GPS signals are lost or interfered. At the same time, multi-sensor fusion technology has also been widely used in complex environments. For example, combining vision, distance measurement, and high-precision map data fusion can achieve adaptability to complex road conditions and dynamic obstacles. However, due to the algorithm complexity and high hardware cost brought by sensor diversity, real-time and universality are limited. In addition, the introduction of deep learning technology improves positioning accuracy through semantic maps and neural networks, but the complex algorithm framework further limits the feasibility of its real-time application.

[0003] In the study of indoor parking lot positioning, due to the poor accuracy of traditional satellite positioning technology in indoor environments, researchers have turned to machine vision and multi-sensor fusion technology. For example, by identifying parking identifiers through computer vision or combining multilateral measurement technology, the positioning accuracy in low-light conditions has been improved to a certain extent. However, these methods still cannot meet the needs of high-precision scenarios, and most of them rely on specific scenarios or landmarks, with limited application scope. In addition, in terms of seamless switching between indoor and outdoor parking scenarios, existing research mainly focuses on the integration of GPS and indoor positioning technology, but the positioning stability and accuracy in the transition area still face challenges.

[0004] Overall, vehicle parking positioning technology has made important progress in outdoor and indoor scenarios, but there are still major challenges in terms of method complexity, hardware requirements, accuracy and real-time performance. Future research should focus on simplifying the complexity of the method and reducing the high requirements for hardware. At the same time, improving the adaptability and stability of the method in complex environments is an urgent problem to be solved in order to achieve good adaptability to vehicle positioning in complex scenarios. Summary of the invention

[0005] The present invention proposes a method and system for reversing positioning of a large semi-trailer vehicle based on subjective parking space marking, which can effectively improve the position detection accuracy between the semi-trailer vehicle and the parking space during the reversing process. In view of the problem of limited camera field of view and unstable positioning in complex environments during the reversing process, the present invention provides an efficient and accurate vehicle positioning method by combining red landmark lines and grid markings.

[0006] To achieve the above object, the present invention adopts the following technical solutions: The method for reversing positioning of a large semi-trailer vehicle based on subjective parking space identification includes the following steps: 1) Design a complete multi-camera signal acquisition and processing system, using cameras as data input to provide basic data support for vehicle positioning; 2) Calibrate each camera to obtain the corresponding camera parameters, and convert the image taken by the camera into a bird's-eye view through perspective transformation; 3) Use the checkerboard calibration plate to convert the image coordinate system into the ground coordinate system; 4) Identify the red landmark line in the bird's-eye view based on the ground coordinate system and calculate the geometric position relationship between the vehicle and the parking space; 5) Calculate the position of the vehicle relative to the bottom edge of the parking space by detecting the color markings of the grid next to the parking space and combining the camera's detection results of the grid markings; 6) Based on the detection results of the straight line and the grid, the position between the vehicle and the parking space is calculated and accurately output.

[0007] A further improvement of the present invention is that in step 1), real-time positioning and reversing assistance of large semi-trailer vehicles are achieved through multiple cameras and sensors; first, the system provides a stable DC12V power supply through a UPS power supply, and the camera collects images of the surrounding environment and transmits them to a video decoder through HDMI to convert them into digital video signals; the digital video signals are transmitted to a computer in the cab through a USB, and the relative position between the vehicle and the parking space is calculated in real time by combining the data of the RS485 module, the laser ranging sensor and the angle encoder; by integrating the camera video data and the sensor information, the system can feedback the spatial position of the vehicle and feedback it to the driver.

[0008] A further improvement of the present invention is that in step 2), after the camera is loaded, the camera on the vehicle body is calibrated offline and parameters are calibrated before the vehicle is put into use, that is, during the vehicle positioning process, the camera is accurately calibrated to establish a mapping relationship between the image coordinate system and the world coordinate system.

[0009] A further improvement of the present invention is that in step 3), after completing the camera calibration, chessboard distance matching is performed, and the chessboard is used to realize the conversion of the image coordinate system to the space coordinate system. By matching the known geometric dimensions of the chessboard with the number of pixels between adjacent corner points in the image, an accurate correspondence between a single pixel and the actual physical distance is obtained.

[0010] A further improvement of the present invention is that in step 4), the red landmark line in the bird's-eye view is identified and edge points are extracted, the edge points are fitted using the least squares method to perform straight line detection to obtain the landmark line, and the center line of the landmark line is obtained using the end points of the straight line, and the distance between the center line in the bird's-eye view and the optical center of the camera is calculated; finally, the geometric position relationship between the vehicle and the parking space is inferred based on the deflection angle and distance information of the center line.

[0011] A further improvement of the present invention is that in step 5), the camera image is preprocessed, including image correction, contrast enhancement, normalization and color segmentation, to ensure that the image is clear and the pixel value is standardized; by converting the RGB image into the HSV color space, setting the hue and saturation thresholds, extracting the red, green, blue and yellow areas, and merging them into a color mask; applying the morphological opening operation to expand the edge continuity, and the closing operation to fill the breaks; after extracting the edge using the Canny algorithm, selecting the largest square for processing; segmenting each square according to the color threshold and extracting the centroid coordinates, arranging the colors in the order of upper left, upper right, lower left, and lower right, constructing a color order index library, and matching it with the bottom edge distance of the parking space.

[0012] A further improvement of the present invention is that, in step 6), by arranging square grid signs with four color sequences on both sides of the parking space, auxiliary detection of the distance between the vehicle and the bottom edge of the parking space is achieved; a square grid is arranged every 2 meters, for a total of 20, with the colors arranged symmetrically and consistent with the central axis of the parking space; by obtaining the coordinates of the center point of the square grid and the camera position information, the actual distance and angle between the camera and the bottom edge of the parking space are calculated, and the relative position of the vehicle and the parking space is fed back in real time; combined with the vehicle geometric parameters, the information obtained from the side camera can be used to infer the distance between the rear camera and the bottom of the parking space.

[0013] The large semi-trailer vehicle reversing positioning system based on subjective parking space marking includes: System design module, used to design a complete multi-camera signal acquisition and processing system, using cameras as data input to provide basic data support for vehicle positioning; The preprocessing calibration module is used to calibrate each camera to obtain the corresponding camera parameters and convert the image taken by the camera into a bird's-eye view through perspective transformation; A coordinate system conversion module is used to convert the image coordinate system into the ground coordinate system using a checkerboard calibration plate; The landmark line recognition and positioning module is used to recognize the red landmark line in the bird's-eye view based on the ground coordinate system and calculate the geometric position relationship between the vehicle and the parking space; The index grid positioning module is used to calculate the position of the vehicle relative to the bottom edge of the parking space by detecting the grid color mark next to the parking space and combining the camera's detection result of the grid mark; The parking space positioning module is used to integrate the detection results of straight lines and grids, calculate and accurately output the positioning between the vehicle and the parking space.

[0014] A further improvement of the present invention is that in the system design module, multiple cameras and sensors are used to realize real-time positioning and reversing assistance of large semi-trailer vehicles; first, the system provides a stable DC12V power supply through a UPS power supply, and the camera collects images of the surrounding environment and transmits them to a video decoder through HDMI to convert them into digital video signals; the digital video signals are transmitted to a computer in the cab through a USB, and the relative position between the vehicle and the parking space is calculated in real time by combining the data of the RS485 module, the laser ranging sensor and the angle encoder; by integrating the camera video data and the sensor information, the system can feedback the spatial position of the vehicle and feedback it to the driver.

[0015] A further improvement of the present invention is that in the preprocessing calibration module, after the camera is loaded, the camera on the vehicle body is offline calibrated before the vehicle is put into use, and parameter calibration is performed, that is: during the vehicle positioning process, the camera is accurately calibrated to establish a mapping relationship between the image coordinate system and the world coordinate system.

[0016] Compared with the prior art, the present invention has at least the following beneficial technical effects: The method and system for reversing positioning of a large semi-trailer vehicle based on subjective parking space marking provided by the present invention realizes the positioning of the semi-trailer vehicle from multiple angles and in different environments during the reversing process. In particular, in the scenario where the rear camera of the vehicle cannot fully capture the bottom edge of the parking space, the introduction of the grid marking effectively makes up for the shortcomings of the traditional method and improves the positioning accuracy and reliability.

[0017] The present invention uses the HSV color space to comprehensively detect the red landmark lines and grid marks of parking spaces, which significantly improves the accuracy and stability of positioning during the reversing of semi-trailer vehicles. The grid color coding is used to assist positioning, which solves the problem of positioning error caused by the limited camera field of view during the reversing of semi-trailer vehicles. The real-time data fusion of multiple cameras further improves the positioning reliability in complex scenes. The method has low hardware requirements, is easy to apply in practice, and is suitable for various parking environments, especially for reversing positioning of semi-trailer vehicles in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is a general flow chart of a method and system for reversing positioning of a large semi-trailer vehicle based on subjective parking space identification according to the present invention; Figure 2 Schematic diagram of coordinate system conversion relationship in the present invention; Figure 3 A schematic diagram of a chessboard layout in the present invention; Figure 4 A schematic diagram of the perspective and corner point detection of a chessboard bird's-eye view in the present invention; Figure 5 This is a schematic diagram of actual distance measurement of a single camera in the present invention; Figure 6 A schematic diagram of the relative position of a vehicle and a red identification frame in the present invention; Figure 7 This is a schematic diagram of the grid distribution near the parking space in the present invention; Figure 8 It is a flow chart of Tianzi grid recognition detection in the present invention; Fig. 9 A schematic diagram of the distribution of part of the grid during the reversing process in the present invention; Fig.10 A schematic diagram of distance measurement based on single Tianzi grid recognition in the present invention; Fig.11 It is a schematic diagram of the offset measurement of the rear camera based on the grid in the present invention; Fig.12 This is a schematic diagram of the camera blind area measurement in the present invention; Fig.13 This is a flow chart of the multi-camera signal acquisition and processing system in the present invention. DETAILED DESCRIPTION

[0020] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0021] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0022] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0023] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0024] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0025] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0026] The present invention provides a method for reversing positioning of a large semi-trailer vehicle based on subjective parking space identification, comprising the following steps: 1) Design a complete multi-camera signal acquisition and processing system, using cameras as data input to provide basic data support for vehicle positioning; 2) Calibrate each camera to obtain the corresponding camera parameters, and convert the image taken by the camera into a bird's-eye view through perspective transformation; 3) Use the checkerboard calibration plate to convert the image coordinate system into the ground coordinate system; 4) Identify the red landmark line in the bird's-eye view based on the ground coordinate system and calculate the geometric position relationship between the vehicle and the parking space; 5) Calculate the position of the vehicle relative to the bottom edge of the parking space by detecting the color markings of the grid next to the parking space and combining the camera's detection results of the grid markings; 6) Based on the detection results of the straight line and the grid, the position between the vehicle and the parking space is calculated and accurately output.

[0027] The large semi-trailer vehicle reversing positioning system based on subjective parking space identification provided by the present invention comprises: System design module, used to design a complete multi-camera signal acquisition and processing system, using cameras as data input to provide basic data support for vehicle positioning; The preprocessing calibration module is used to calibrate each camera to obtain the corresponding camera parameters and convert the image taken by the camera into a bird's-eye view through perspective transformation; A coordinate system conversion module is used to convert the image coordinate system into the ground coordinate system using a checkerboard calibration plate; The landmark line recognition and positioning module is used to recognize the red landmark line in the bird's-eye view based on the ground coordinate system and calculate the geometric position relationship between the vehicle and the parking space; The index grid positioning module is used to calculate the position of the vehicle relative to the bottom edge of the parking space by detecting the grid color mark next to the parking space and combining the camera's detection result of the grid mark; The parking space positioning module is used to integrate the detection results of straight lines and grids, calculate and accurately output the positioning between the vehicle and the parking space.

[0028] Embodiment 1: Embodiment 1 of the present invention proposes a vehicle reversing positioning method based on a red identification frame. Figure 6 This is a schematic diagram of the relative positions of a vehicle and a red identification frame in the vehicle reversing positioning method based on a red identification frame proposed in Example 1 of the present invention.

[0029] In the present invention, when using a bird's-eye view for distance measurement, it is necessary to calculate the distance between the vehicle and the critical image in the camera shot, that is, the distance between the chessboard edge line and the vehicle body. This distance is the visual blind spot, which is used to measure the specific size of the blind spot. Taking the left camera as an example, the present invention establishes an accurate ground coordinate system. Since the car body remains parallel to the bottom edge of the chessboard, combined with the intersection of the vertical coordinate line of the camera optical center in the bird's-eye view and the lower edge of the image, when shooting, the lower edge line of the image coincides with the boundary line of the chessboard, so that the distance between the vehicle body line and the chessboard edge line can be measured, as shown in the figure. Fig.12 Using the same method, the blind spot distances of all cameras installed on the vehicle body are measured, and the following results are obtained through experiments: Figure 6 Take the camera layout in as an example, N The blind spot distance of the camera is 20.43 cm. M The blind spot distance of the camera is 20.40 cm, E The blind spot distance of the camera is 20.40 cm, G The blind spot distance of the camera is 20.39 cm. I The camera's blind spot distance is 20.40 cm.

[0030] Based on the above offline data, vehicle positioning can be further realized. The specific operation is that in the actual parking process, the red marking belt (red belt) in the bird's-eye view is firstly detected as a straight line to extract the edge point data of the red belt. Then, the least squares method is used to fit these edge points to obtain the straight line equation of the landmark line. The midline position of the landmark line can be determined by the endpoints of the fitted straight line. Next, the left and right edge points of the center line in the image are extracted, which are ( X l ，Y l )and( X r ，Y r ). According to formula (1), the angle between the center line of the red belt and the vehicle body can be calculated: θ .

[0031] (1) When measuring the vertical distance from the red landmark line to the vehicle distance measuring device (i.e. camera), since the optical center of the camera is located on the center line of the shooting picture, the optical center is also located on the vertical center line of the image in the bird's eye view. Based on this, the intersection of the red belt center line and the vertical center line of the bird's eye view can be determined. O By connecting the intersection O The midpoint of the border of the picture below the bird's-eye view is counted and the number of pixels between the two is converted into actual distance L 1 .Will L 1 Distance from the blind spot measured in advance L 2 Adding them together, we can get the distance from the camera optical center to the intersection point O The total distance is Figure 5 In the figure, the red line frame represents the red landmark line, the blue line frame represents the vehicle, and the yellow dot marks the position of a camera on the left side of the vehicle. According to formula (2), the vertical plane distance from the red mark line on the ground to the vehicle distance measuring device can be further calculated: h .

[0032] (2) Since the semi-trailer has a long body, in practical applications, the vehicle's reversing into position is simplified to a geometric problem. Figure 6 Among them, the red rectangle represents the parking space, and the blue rectangle represents the large semi-trailer. The length of the red rectangle is , the width is , and the length of the blue rectangle is a , the width is bIn order to achieve more accurate measurement, distance measuring devices are installed at the one-third and two-thirds points on the left and right sides of the large semi-trailer. M, N, I, G , a distance measuring device is installed at the center point of the rear of the vehicle E , the vertical distance between the parking space line and the semi-trailer is measured by these distance measuring devices h 1 、h 2 , h 3 、h 4 .

[0033] In order to simplify the calculation, a point on the ground is selected as the origin to establish a plane rectangular coordinate system, and the geometric center point of the large semi-trailer is set as P ( c x , c y ), the geometric center point of the parking space is Q ( d x , d y ). By measuring these distances, we can find P ( c x , c y )and Q ( d x , d y ) to obtain the position relationship of the vehicle relative to the parking space.

[0034] To improve the real-time performance and accuracy of the method, when the rear camera E of the vehicle detects a straight line of the parking space line, the system will enable the distance measuring devices (e.g. G and the tail E ,or M and the tail E ) to make more accurate distance measurements, thereby further improving the vertical distance of the red marker line relative to the vehicle h 3 and h 4 measurement accuracy.

[0035] A ( x a ,y a ) point and P ( c x ,cy ) point is as follows: (3) B ( x b ,y b ) point and P ( c x ,c y ) point is as follows: (4) C ( x c ,y c ) point and P ( c x , c y ) point is as follows: (5) D ( x d ,y d ) point and P ( c x , c y ) point is as follows: (6) With the help of B ( x b ,y b ) point sum C ( x c ,y c ) point, we can calculate E ( x e , y e )point: (7) And through distance measurement, we can get h 4 ,therefore ,and F The distance between the horizontal coordinate of the point and the center of the parking space , so the following formula can be used to obtain P (c x , c y )and Q ( d x , d y )The mathematical relationship between the horizontal axes is (8) Similarly, it can be calculated by the following formula G ( x g , y g )point: (9) And through distance measurement, we can get h 3 ,therefore ,and H The distance between the vertical coordinate of the point and the center of the parking space , so the following formula can be used to obtain P ( c x , c y )and Q ( d x , d y )The mathematical relationship between the vertical coordinates: (10) In summary, by accurately measuring the distance between the red marking line of the parking space and the vehicle, the geometric center point of the vehicle can be derived. P ( c x , c y ) and the geometric center of the parking space Q ( d x , d y ), the relative position between the vehicle and the red parking space can be obtained using the red marking box of the parking space.

[0036] A vehicle reversing positioning method based on a red identification frame proposed in Example 1 of the present invention can realize the positional relationship between the vehicle and the parking space when the rear camera detects the red band at the bottom during the reversing process of the semi-trailer, thereby improving the positioning accuracy of the semi-trailer and the stability of the method, thereby improving the safety and reliability of vehicle reversing in reversing scenarios with high precision requirements.

[0037] The vehicle reversing positioning method based on the red identification frame proposed in Example 1 of the present invention can also be applied to the parking positioning environment of other vehicles, and has strong versatility and practicality.

[0038] Embodiment 2: Embodiment 2 of the present invention is based on the vehicle reversing positioning method with a red marking frame proposed in Embodiment 1, which solves the problem that the distance in the direction of the vehicle body cannot be accurately measured when the vehicle has not completely reached the rear. This problem causes deviations in red belt detection, which in turn affects the positioning accuracy of the vehicle. In actual engineering applications, it is difficult to ensure the robustness and reliability of the system by relying solely on the red belt for positioning, so it is necessary to combine other auxiliary tools to improve positioning accuracy and system stability.

[0039] To solve this problem, Example 2 of the present invention proposes a method for identifying vehicle positioning based on a color sequence index of a coded Tianzi grid. The method uses a subjective marking coded Tianzi grid evenly distributed on one side of the parking space as an auxiliary tool to improve positioning accuracy. Figure 7 , Reasonable arrangement of the Tianzi grid can provide a more stable and continuous reference signal, especially when the red tape cannot be effectively detected, the Tianzi grid can provide a reliable positioning basis.

[0040] The grid is composed of small squares of four different colors, and the color arrangement order of each grid is related to the distance from the geometric center point of the grid to the bottom edge of the parking space. By accurately identifying the color sequence in the grid, the distance between the vehicle and the parking space can be calculated. Therefore, the correct identification of the grid becomes the key to achieving accurate positioning. Based on this, the present invention proposes a color sequence index recognition method based on the coded grid to further complete vehicle positioning. Figure 8 This is a specific process of a vehicle positioning method based on color sequence indexing of a coded grid proposed in Example 2 of the present invention.

[0041] The positioning process of the invention includes multiple steps such as image preprocessing, color segmentation, morphological operation, square extraction and color sequence extraction. Each link is interrelated to ensure the accurate detection of the grid color sequence and the accurate feedback of the relative position of the vehicle.

[0042] Since there may be noise and uneven lighting in the image acquisition environment, direct color recognition and area detection may lead to large errors. Therefore, image preprocessing is crucial before performing Tianzi grid detection and color recognition. First, the bird's-eye view image is corrected and enhanced. The purpose of image correction is to restore the true shape of the object in the image and eliminate the geometric distortion caused by lens distortion. After completing the image correction, contrast enhancement and sharpening are performed to improve the accuracy of Tianzi grid area detection.

[0043] After image preprocessing is completed, the algorithm converts the image from RGB color space to HSV color space, and extracts each color area by analyzing the HSV thresholds of red, green, blue, and yellow. The specific steps include setting the thresholds of the hue (H) and saturation (S) components in the HSV space to accurately extract different color areas. The specific segmentation thresholds are as follows: Red area ( M R ):

[0044] Green area ( M G ):

[0045] Blue area ( M B ):

[0046] Yellow area ( M Y ):

[0047] After obtaining each color region, all color regions are merged into a comprehensive color mask. In order to further optimize the effect of edge detection, especially to eliminate interference caused by light, noise or irregular objects, the algorithm applies morphological opening operations to expand the continuity of edge regions. Next, in order to deal with possible breaks in edge detection, the algorithm uses morphological closing operations, that is, expansion first and then corrosion, to fill the holes that may exist inside the grid, making the edges of the squares smoother, and uses the Canny edge detection algorithm to extract edge information in the image.

[0048] After detecting the edge information, the circumscribed rectangle of the largest connected area in the image is extracted to determine whether its shape is close to a square. In the case of multiple squares, only the square with the longest side is extracted for subsequent processing. Next, each color area in the square is segmented according to the threshold. After segmenting each color area, the centroid of each color area is further extracted. The centroid is calculated based on the shape characteristics of the area, and the area with the largest area is preferentially selected as the color representative. The centroid calculation formula is as follows (11) in, x i and y i are the coordinates of all pixels in the region, N is the total number of pixels in the area. When calculating, first follow y The axis sorts all the centroids and divides them into the upper and lower parts of the grid area, and then sorts the upper and lower parts according tox Axis sorting, finally determine the order of the four colors of red, green, blue, and yellow in the Tianzi grid and the position of the geometric center of the Tianzi grid. After obtaining the centroid coordinates of each color, this section conducts a series of analysis and sorting of the colors in the Tianzi grid. Specifically, the four colors are arranged in the order of upper left, upper right, lower left, and lower right, and the color arrangement order is finally determined.

[0049] Based on the results of these color arrangements, the present invention constructs a color order index library, which associates and matches the arrangement order of different colors with the corresponding parking space bottom edge distance. In practical applications, by identifying the color order in the grid, the distance between the vehicle and the bottom edge of the parking space can be quickly and accurately determined. Then, by extracting the geometric centroid coordinates of the colors in the image, and matching the actual position of the parking space according to the color order table. The detailed information of the color order index is shown in Table 1. By calculating the position of the centroid and combining the pre-set color order rules, the algorithm can accurately output the color sequence of the grid, and it can be seen from the table that the corresponding distance from the bottom of the parking space at this time.

[0050] Table 1 is an index table for sorting different Tianzi grid colors in the present invention.

[0051]

[0052] Embodiment 3: Embodiment 3 of the present invention proposes a vehicle reversing positioning method based on subjective parking space markers, which integrates a vehicle reversing positioning method based on a red marking frame and a color sequence index recognition method based on a coded grid to achieve high-precision positioning of vehicle parking.

[0053] In the process of vehicle positioning, the accuracy of the rear camera is crucial. However, due to the long body of the vehicle, the rear camera is far away from the bottom edge of the parking space when reversing, and it is often difficult to capture the bottom red mark frame in time. To solve this problem, the present invention arranges a "field grid" mark with four colors in sequence near the landmark lines on both sides of the parking space as an auxiliary tool. Fig. 9 The location information diagram of some of the Tianzi grids is shown. The total length of the parking space is 20 meters. A Tianzi grid is placed on the left and right sides every 2 meters, with a total of 20 on both sides. The vertical distance between these Tianzi grids and the red marking frame is d Each grid is composed of four colors (upper left, upper right, lower left, and lower right). Although the colors are arranged differently, they are all centered on the central axis of the parking space. l As the axis of symmetry, the color order of the grid in the left and right camera perspectives is consistent at the same distance, and different color combinations correspond to different distances between the vehicle and the bottom edge of the parking space.

[0054] By identifying the color sequence of the grid, the algorithm can accurately calculate the offset of the rear of the vehicle relative to the bottom edge of the parking space based on the position information of the cameras on both sides of the vehicle body and the rear end camera. After accurately identifying the color sequence of the grid and obtaining the position information, the next step is to calculate the real-time distance between the camera and the bottom edge of the parking space, thereby inferring the exact position of the vehicle. Fig.10 .

[0055] Through the Tianzi grid recognition method, the coordinates of the center point of the Tianzi grid can be accurately calculated ( x 1 , y 1 ), and combined with the camera to capture the bottom center point of the image ( x 2 , y 2 ) to obtain the actual distance between the two. At the same time, using the coordinate information of the two key points, the two points can be connected to further calculate the distance between the connecting line and the image coordinate system. Y The angle between the two lines of the axis . By combining the calculation of distance and angle, the driving assistance system can provide real-time feedback on the relative position between the vehicle and the parking space, allowing the driver to complete the reversing action more accurately in the parking scene. This real-time distance measurement method based on the grid not only improves the accuracy of vehicle positioning, but also enhances the robustness and reliability of the system in dynamic environments. Combined with the previous color sequence recognition method, this process ensures that the relative position of the vehicle can still be detected even when the red marking line cannot be effectively identified.

[0056] When calculating the distance between the camera and the bottom edge of the parking space, first connect the center point with the bottom midline point of the image, calculate the pixel distance between the bottom center point of the image and the center point of the grid, and convert it into the actual distance L 3 Then, we use the slope and inverse cotangent function to calculate the line and the image coordinate system from equation (3.38): Y Angle between axes .

[0057] (12) in, k It is the slope of the line between the center point of the grid and the center point of the bottom edge of the image.

[0058] Finally, based on the color sequence and the calculated distance, the actual distance from the camera position to the bottom of the parking space is calculated by equation (3.39), and the final result is output. d L .

[0059] (13) in, X It is the distance reference value found from the color sequence index library.

[0060] by Figure 6 In M For example, the camera is located on one side of the vehicle body and one-third of the vehicle length relative to the rear end. Through the Tianzi grid recognition method, it can be detected M Distance between the camera and the bottom of the parking space d L Due to the length of the vehicle a and vehicle width b is known, and based on the vehicle's geometric parameters, the tail camera can be further calculated E Distance from the bottom of the parking space h 4 ,See Fig.11 , h 4 The calculation formula is as follows (14) By identifying the color sequence of the grid and combining the coordinate information of key points in the image, the vehicle positioning system of the present invention can provide real-time feedback on the relative position between the vehicle and the parking space. This method effectively makes up for the limitation that the rear camera cannot recognize the red mark frame in the early stage of reversing, and at the same time improves the positioning accuracy and system reliability of the vehicle during reversing.

[0061] In summary, embodiment 3 of the present invention not only makes up for the limitations of the rear camera by combining the detection information of the grid and the red identification frame, but also provides a high-precision and stable vehicle reversing positioning solution by calculating the deflection angle and distance.

[0062] Embodiment 4: Embodiment 4 of the invention proposes a flowchart of a multi-camera signal acquisition and processing system for specific vehicle positioning. Fig.13 The system mainly consists of the following parts: (1) Power supply system: The cameras and other devices in the system are powered by a UPS (DC 12V). The UPS ensures the continuous operation of the entire system during power outages or power fluctuations. The power supply of each device comes from the UPS, ensuring the normal operation of the equipment.

[0063] (2) Camera acquisition system: The system uses a total of 5 cameras, labeled as camera 1 to camera 5. Each camera transmits the analog video signal to the video acquisition module through a transmission channel. The camera uses the common AHD or CV format, and the video signal is transmitted simultaneously on 5 channels.

[0064] (3) Video acquisition module: The system contains two video acquisition devices, each of which can process 2-3 camera signals simultaneously. The video acquisition module converts HDMI signals into USB signals and sends them to the computer for real-time image processing and display.

[0065] (4) Sensor integration and control system: In addition to video acquisition, the system also integrates sensor devices such as laser displacement sensors and angle encoders. These sensors transmit data through RS485 modules. The laser displacement sensor is used to detect the displacement data in the system. The sensor data is transmitted to the USB interface through the RS485 module and then processed by the computer.

[0066] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

[0067] In addition, it should be understood that although this specification is described in accordance with the implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation modes that can be understood by those skilled in the art. The above content is only to illustrate the technical idea of ​​the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A large semi-trailer vehicle reversing positioning method based on subjective parking space identification, characterized in that: The following steps are involved: 1) Design a complete multi-camera signal acquisition and processing system, using cameras as data input to provide basic data support for vehicle positioning; 2) Calibrate each camera to obtain the corresponding camera parameters, and convert the image taken by the camera into a bird's-eye view through perspective transformation; 3) Use the checkerboard calibration plate to convert the image coordinate system into the ground coordinate system; 4) Identify the red landmark line in the bird's-eye view based on the ground coordinate system and calculate the geometric position relationship between the vehicle and the parking space; 5) Calculate the position of the vehicle relative to the bottom edge of the parking space by detecting the color markings of the grid next to the parking space and combining the camera's detection results of the grid markings; 6) Based on the detection results of the straight line and the grid, the position between the vehicle and the parking space is calculated and accurately output.

2. The method for reversing positioning of a large semi-trailer vehicle based on subjective parking space identification according to claim 1 is characterized in that: In step 1), multiple cameras and sensors are used to realize real-time positioning and reversing assistance of large semi-trailer vehicles. First, the system provides a stable DC12V power supply through a UPS power supply. The camera collects images of the surrounding environment and transmits them to the video decoder through HDMI and converts them into digital video signals. The digital video signals are transmitted to the computer in the cab through USB. Combined with the data of the RS485 module, the laser ranging sensor and the angle encoder, the relative position between the vehicle and the parking space is calculated in real time. By integrating the camera video data and sensor information, the system can feedback the spatial position of the vehicle and feedback it to the driver.

3. The method for reversing and positioning a large semi-trailer vehicle based on subjective parking space identification according to claim 1 is characterized in that: In step 2), after the camera is loaded, the camera on the vehicle body is calibrated offline before the vehicle is put into use, and parameter calibration is performed, that is, during the vehicle positioning process, the camera is accurately calibrated to establish a mapping relationship between the image coordinate system and the world coordinate system.

4. The method for reversing and positioning a large semi-trailer vehicle based on subjective parking space identification according to claim 1 is characterized in that: In step 3), after completing the camera calibration, checkerboard distance matching is performed, and the checkerboard is used to realize the conversion of the image coordinate system to the space coordinate system. By matching the known geometric dimensions of the checkerboard with the number of pixels between adjacent corner points in the image, an accurate correspondence between a single pixel and the actual physical distance is obtained.

5. The method for reversing positioning of a large semi-trailer vehicle based on subjective parking space identification according to claim 1 is characterized in that: In step 4), the red landmark line in the bird's-eye view is identified and edge points are extracted. The edge points are fitted using the least squares method to perform straight line detection to obtain the landmark line. The center line of the landmark line is obtained using the endpoints of the line, and the distance between the center line and the optical center of the camera in the bird's-eye view is calculated. Finally, the geometric position relationship between the vehicle and the parking space is inferred based on the deflection angle and distance information of the center line.

6. The method for reversing and positioning a large semi-trailer vehicle based on subjective parking space identification according to claim 1, characterized in that: In step 5), the camera image is preprocessed, including image correction, contrast enhancement, normalization and color segmentation, to ensure that the image is clear and the pixel value is standardized; by converting the RGB image to the HSV color space, setting the hue and saturation thresholds, extracting the red, green, blue and yellow areas, and merging them into a color mask; applying the morphological opening operation to extend the edge continuity, and the closing operation to fill the gaps; after extracting the edge using the Canny algorithm, select the largest square for processing; segment each square according to the color threshold and extract the centroid coordinates, arrange the colors in the order of upper left, upper right, lower left, and lower right, build a color order index library, and match it with the distance to the bottom edge of the parking space.

7. The method for reversing and positioning a large semi-trailer vehicle based on subjective parking space identification according to claim 1, characterized in that: In step 6), a grid of four colors is arranged on both sides of the parking space to assist in detecting the distance between the vehicle and the bottom edge of the parking space; a grid of four colors is arranged every 2 meters, for a total of 20, with symmetrical color arrangement and consistent with the central axis of the parking space; By obtaining the coordinates of the center point of the grid and the camera position information, the actual distance and angle between the camera and the bottom edge of the parking space are calculated, and the relative position of the vehicle and the parking space is fed back in real time. Combined with the vehicle's geometric parameters, the information obtained from the side camera can be used to infer the distance between the rear camera and the bottom of the parking space.

8. A large semi-trailer vehicle reversing positioning system based on subjective parking space identification, characterized in that: include: System design module, used to design a complete multi-camera signal acquisition and processing system, using cameras as data input to provide basic data support for vehicle positioning; The preprocessing calibration module is used to calibrate each camera to obtain the corresponding camera parameters and convert the image taken by the camera into a bird's-eye view through perspective transformation; A coordinate system conversion module is used to convert the image coordinate system into the ground coordinate system using a checkerboard calibration plate; The landmark line recognition and positioning module is used to recognize the red landmark line in the bird's-eye view based on the ground coordinate system and calculate the geometric position relationship between the vehicle and the parking space; The index grid positioning module is used to calculate the position of the vehicle relative to the bottom edge of the parking space by detecting the grid color mark next to the parking space and combining the camera's detection result of the grid mark; The parking space positioning module is used to integrate the detection results of straight lines and grids, calculate and accurately output the positioning between the vehicle and the parking space.

9. The large semi-trailer vehicle reversing positioning system based on subjective parking space identification according to claim 8 is characterized in that: In the system design module, multiple cameras and sensors are used to realize real-time positioning and reversing assistance of large semi-trailer vehicles. First, the system provides a stable DC12V power supply through a UPS power supply. The camera collects images of the surrounding environment and transmits them to the video decoder through HDMI and converts them into digital video signals. The digital video signals are transmitted to the computer in the cab through USB, and the relative position between the vehicle and the parking space is calculated in real time by combining the data of the RS485 module, the laser ranging sensor and the angle encoder. By integrating the camera video data and the sensor information, the system can feedback the spatial position of the vehicle and feedback it to the driver.

10. The large semi-trailer vehicle reversing positioning system based on subjective parking space identification according to claim 8 is characterized in that: In the preprocessing calibration module, after the camera is loaded, the camera on the vehicle body is calibrated offline before the vehicle is put into use, and parameter calibration is performed, that is, during the vehicle positioning process, the camera is accurately calibrated to establish a mapping relationship between the image coordinate system and the world coordinate system.

Images