Track line dynamic adjustment method and device, equipment and medium

Abstract

The invention discloses a trajectory dynamic adjustment method and device, equipment and a medium, and the method is applied to a management server, and comprises the steps: obtaining a real-time environment image collected by a camera module; preprocessing the real-time environment image according to a preset visual segmentation rule to obtain an effective ground area; acquiring a real-time distance, acquired by the radar ranging module, between the obstacle and the current vehicle; performing mask data fusion according to the effective ground area and the real-time distance to obtain a target ground area; and drawing a real-time trajectory of the current vehicle according to the target ground area and the vehicle position data. According to the method, through dynamic adjustment and real-time display of the trajectory, the reversing process is more visual and easy to understand, and the driving experience and safety performance of a driver are improved.

Description

Technical Field

[0001] The present invention relates to the field of broadcasting technology, and in particular to a trajectory dynamic adjustment method, device, equipment and medium. Background Art

[0002] When a car is put into reverse gear, the reversing assist function is usually enabled, and the reversing track line is one of the key functions. Generally speaking, the ideal track line should fit closely to the ground, and only display the ground part that can actually be reached, so as to ensure that the driver can accurately judge the actual distance between the vehicle and the obstacle. However, traditional reversing track lines often adopt fixed lengths and styles. This design may cause some problems when there are obstacles behind the vehicle; for example, when the obstacle is close to the rear of the vehicle, the traditional track line cannot adjust the display effect by itself, so it may cover the obstacle, giving the driver a visual effect that the track line seems to be suspended in the air, affecting the driver's user experience and safety. Summary of the invention

[0003] The embodiments of the present invention provide a method, device, equipment and medium for dynamically adjusting a trajectory line, aiming to solve the display effect problem of a reversing trajectory line in the prior art methods.

[0004] In a first aspect, an embodiment of the present invention provides a method for dynamically adjusting a trajectory line, wherein the method is applied to a management server, and the method includes:

[0005] Acquire real-time environment images collected by the camera module;

[0006] Preprocessing the real-time environment image according to a preset visual segmentation rule to obtain a valid surface area;

[0007] Get the real-time distance between the obstacle and the current vehicle collected by the radar ranging module;

[0008] Performing mask data fusion according to the effective ground area and the real-time distance to obtain a target ground area;

[0009] The real-time trajectory line of the current vehicle is drawn according to the target ground area and the vehicle position data.

[0010] In a second aspect, an embodiment of the present invention further provides a trajectory line dynamic adjustment device, wherein the device is applied to a management server, and the device includes:

[0011] An image acquisition unit, used to acquire real-time environment images acquired by a camera module;

[0012] A preprocessing unit, used for preprocessing the real-time environment image according to a preset visual segmentation rule to obtain a valid surface area;

[0013] A real-time distance measurement unit is used to obtain the real-time distance between the obstacle and the current vehicle collected by the radar distance measurement module;

[0014] A data processing unit, used for performing mask data fusion according to the effective ground area and the real-time distance to obtain a target ground area;

[0015] The trajectory drawing unit is used to draw the real-time trajectory of the current vehicle according to the target ground area and the vehicle position data.

[0016] In a third aspect, an embodiment of the present invention further provides a computer device, wherein the device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0017] Memory, used to store computer programs;

[0018] The processor is used to implement the steps of the trajectory line dynamic adjustment method described in the first aspect when executing the program stored in the memory.

[0019] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for dynamically adjusting the trajectory line as described in the first aspect above are implemented.

[0020] The embodiment of the present invention provides a method, device, equipment and medium for dynamic adjustment of trajectory line, the method comprising: obtaining a real-time environment image collected by a camera module; preprocessing the real-time environment image according to a preset visual segmentation rule to obtain an effective ground area; obtaining the real-time distance between the obstacle and the current vehicle collected by the radar ranging module; performing mask data fusion according to the effective ground area and the real-time distance to obtain a target ground area; drawing the real-time trajectory line of the current vehicle according to the target ground area and the vehicle position data. The above-mentioned method for dynamic adjustment of trajectory line is applied to a management server, which not only makes the reversing process more intuitive and easy to understand through the dynamic adjustment and real-time display of the trajectory line, thereby improving the driver's driving experience; at the same time, by dynamically adjusting the trajectory line, the driver can more accurately judge the actual distance between the vehicle and the obstacle, thereby avoiding potential collision risks and improving safety; in addition, this solution can also be applied to different types of vehicles and driving environments, and has strong versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0022] Figure 1 A method flow chart of a method for dynamically adjusting a trajectory provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of an application scenario of the trajectory dynamic adjustment method provided by an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of another application scenario of the trajectory line dynamic adjustment method provided by an embodiment of the present invention;

[0025] Figure 4 Another method flow chart of the trajectory line dynamic adjustment method provided by an embodiment of the present invention;

[0026] Figure 5 A schematic block diagram of a trajectory dynamic adjustment device provided by an embodiment of the present invention;

[0027] Figure 6 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0030] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0031] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0032] See also Figure 1 and Figure 2 As shown in the figure, the embodiment of the present invention provides a method for dynamically adjusting the trajectory line, which is applied to the management server 10, which can be the management server of the vehicle system, that is, the server for managing the vehicle. The management server 10 obtains the real-time environment image collected by the camera module 20; pre-processes the real-time environment image according to the preset visual segmentation rules to obtain the effective ground area; obtains the real-time distance between the obstacle and the current vehicle collected by the radar ranging module 30; performs mask data fusion according to the effective ground area and the real-time distance to obtain the target ground area; draws the real-time trajectory line of the current vehicle according to the target ground area and the vehicle position data. Specifically, the management server will capture the obstacle information behind the vehicle in real time, and dynamically adjust the length and style of the trajectory line according to this information. That is, the scheme can realize the function of dynamically adjusting the reversing trajectory line by combining the data of radar distance and camera image visual analysis.

[0033] In this way, the track line can fit the ground closely, and only the ground part that can be actually reached is displayed, avoiding the floating effect of the traditional track line. Therefore, this method can make the reversing process more intuitive and easy to understand through the dynamic adjustment and real-time display of the track line, improving the driver's driving experience; at the same time, by dynamically adjusting the track line, the driver can more accurately judge the actual distance between the vehicle and the obstacle, thereby avoiding potential collision risks and improving safety; in addition, this solution can also be applied to different types of vehicles and driving environments, with strong versatility and adaptability.

[0034] like Figure 1 As shown, in one embodiment, the trajectory line dynamic adjustment method is applied to a management server, which can be a server in a vehicle system. The method includes steps S110 to S150.

[0035] Step S110, acquiring a real-time environment image captured by a camera module.

[0036] In this embodiment, the camera module may be a high-definition camera or other types of cameras. The camera module can capture images of the rear of the vehicle in real time. That is, the management server can obtain the real-time environment image collected through the camera, and the real-time environment image may be an image of the rear of the current vehicle.

[0037] Step S120, preprocessing the real-time environment image according to a preset visual segmentation rule to obtain a valid surface area.

[0038] In this embodiment, the management server can pre-process the acquired real-time environment image according to the preset visual segmentation rules to obtain the effective ground area. For example, the real-time environment image can be converted into a grayscale image, and the grayscale image can be pre-processed with bilateral filtering to reduce noise interference in the grayscale image while retaining edge details. Then, the grayscale image contour is searched, and the contour of the rear position of the vehicle is marked as white, which is passed as a marking parameter to the watershed algorithm to segment the image, so that the part close to the rear of the vehicle is marked as the foreground, and the preliminary ground ROI area, that is, the effective ground area, is identified.

[0039] Among them, the ground ROI area, namely the ground region of interest (Region of Interest), is mainly used in computer vision, image processing, remote sensing image analysis and other fields. Usually this area contains key information that needs to be identified, analyzed or processed, such as specific landforms, buildings, vegetation types, etc.

[0040] In one embodiment, the step S120 includes the following steps:

[0041] The real-time environment image is converted into a grayscale image; the grayscale image is pre-processed for denoising using bilateral filtering; the image contour of the denoised grayscale image is determined; and the image contour is segmented according to a preset watershed algorithm and marking parameters to obtain a valid surface area.

[0042] After acquiring the real-time environment image through a camera or other image acquisition device, the color real-time environment image is converted into a grayscale image using an image processing algorithm to simplify the image information, reduce the amount of calculation for subsequent processing, and retain sufficient image features for subsequent processing. The image processing algorithm can be an algorithm for conversion based on the values ​​of the three RGB color channels. Common conversion methods include the maximum value method, the average value method, and the weighted average value method; for example, the weighted average value method weights according to the sensitivity of the human eye to different colors, and the weights commonly used are R: 0.299, G: 0.587, and B: 0.114.

[0043] The bilateral filtering algorithm is a filtering algorithm that combines spatial domain and value domain information. It can not only remove noise from the image, but also avoid over-smoothing while retaining the edge features of the image. The design of the bilateral filter is based on the Gaussian function, and the weight is determined by calculating the spatial distance and brightness difference between each pixel and the surrounding pixels, thereby achieving the filtering effect. The management server uses bilateral filtering to perform denoising preprocessing on the grayscale image, which can effectively remove noise from the image while retaining the edge features of the image.

[0044] The management server can also use edge detection algorithms to extract image contours, such as Canny edge detector, Sobel operator, Prewitt operator, etc. These algorithms detect edges by calculating the gradient or difference of pixel points in the image, thereby obtaining image contour information.

[0045] Watershed algorithm: This is a morphological segmentation method based on topological theory. It regards the image as a geodesic topological landform and uses gradient information to simulate the waterlogging process, so as to construct a watershed in the image to segment different areas. In practical applications, in order to improve the accuracy of segmentation, marker parameters (such as seed points, marker images, etc.) are usually combined to guide the segmentation process of the watershed algorithm. Setting of marker parameters: The setting of marker parameters is crucial to the segmentation effect of the watershed algorithm. This usually needs to be determined according to the specific application scenario and image features. For example, when extracting the ground area, the watershed algorithm can be guided to accurately segment the ground area by setting reasonable seed points and marker images.

[0046] Therefore, the management server can convert the real-time environment image into a grayscale image, and use bilateral filtering to pre-process the real-time environment image to reduce noise interference in the real-time environment image while retaining edge details. Then find the grayscale image contour, and mark the contour of the rear position of the current vehicle as white, and pass it as a mark parameter to the watershed algorithm to segment the grayscale image, so as to mark the part close to the rear of the vehicle as the foreground, so as to identify the preliminary ground ROI area, that is, to obtain the effective ground area.

[0047] The innovative application of the watershed algorithm for ground and obstacle segmentation in reversing scenarios effectively solves the problem of ground and obstacle segmentation, improves the accuracy and stability of ground recognition, provides strong support for subsequent fusion analysis, and enhances the safety of vehicles during driving.

[0048] In a further embodiment, before the step of segmenting the image contour according to a preset watershed algorithm and marking parameters to obtain a valid surface area, the method further includes:

[0049] The outline of the parking space position of the current vehicle is marked with a preset identification color; and the outline marked with the identification color is determined as a marking parameter.

[0050] The management server can also mark the outline of the parking space position of the current vehicle as a preset identification color, determine the outline mark by the identification color, and determine the outline mark as a marking parameter.

[0051] Step S130, obtaining the real-time distance between the obstacle and the current vehicle collected by the radar ranging module.

[0052] In this embodiment, the management server can collect the real-time distance between the obstacle and the current vehicle through the radar ranging module. The working principle of the radar ranging module is based on radar technology, which mainly calculates the distance between the object and the ranging module by sending electromagnetic waves and detecting the reflected signal. The core of radar ranging is to calculate the time difference between the transmission and reception of the radar signal to determine the target distance. Its main technical means include pulse ranging, frequency modulated continuous wave and pulse compression technology. In addition, if the distance between the obstacle and the current vehicle is greater than 300cm, it will be ignored.

[0053] Therefore, the comprehensive application of radar ranging and image fusion analysis technology not only improves the accuracy and real-time performance of obstacle detection, but also realizes the precise identification of effective ground areas, providing a reliable basis for drawing trajectory lines during vehicle driving.

[0054] In one embodiment, step S130 also includes the following steps: recording the emission time point when the radar ranging module emits the ultrasonic wave, and the receiving time point when the ultrasonic wave returned by the obstacle is received; and calculating the real-time distance between the obstacle and the current vehicle based on the time difference between the receiving time point and the emission time point.

[0055] When the ultrasonic radar ranging module is triggered, an ultrasonic pulse is emitted, and the triggering moment can be recorded as the emission time point. When the ultrasonic pulse encounters an obstacle, it will be reflected back and received by the radar ranging module, and the reception moment can be recorded as the reception time point. The time difference refers to the difference between the reception time point and the emission time point, which indicates the time from the emission to the reception of the ultrasonic pulse.

[0056] The distance between the obstacle and the radar ranging module can be calculated based on the propagation speed of ultrasound in the air (usually 340 meters per second, but this speed may vary slightly due to factors such as temperature and air pressure) and the time difference. The specific calculation formula is as follows: distance = (time difference × speed of sound) / 2. The " / 2" here is because the ultrasound needs to travel back and forth between the obstacle and the radar ranging module, so the actual distance is half of the distance corresponding to the time difference.

[0057] Step S140 , performing mask data fusion according to the effective ground area and the real-time distance to obtain a target ground area.

[0058] In this embodiment, the distance and orientation information of the obstacle is in the ground plane of the world coordinate system, and generally requires projection transformation to convert it to the same screen coordinate system as the camera image. Then, the mask is fused with the ground ROI area to exclude the part occupied by the obstacle in the ground ROI area, thereby updating the effective ground area, which does not contain any obstacles. Specifically, in the combined application of image processing and radar ranging technology, the management server can perform mask data fusion based on the effective ground area and the real-time distance information obtained by the radar ranging module, so as to accurately extract the target ground area.

[0059] In one embodiment, the step S140 may specifically include the following steps: determining a two-dimensional position image of the obstacle in a standard coordinate system based on the real-time distance and real-time orientation information of the obstacle; converting the two-dimensional position image to the same coordinate system as the real-time environment image according to a preset projection change rule to obtain an intermediate position image; and performing mask data fusion on the intermediate position image and the effective ground area to obtain the target ground area.

[0060] like Figure 3 As shown in the figure, it is a generated diagram between the trajectory line and the obstacle after ranging. For example, the management server can generate a distance mask for each measured point or area based on the real-time distance information provided by the radar ranging module. This mask can be a two-dimensional matrix, in which the value of each element represents the distance between the position and the vehicle; for points beyond the range of interest (such as the set maximum detection distance), their distance values ​​can be set to infinity or a specific invalid value so as to be excluded in subsequent processing. The management server can also use the valid ground area extracted by the image processing algorithm to convert it into a ground area mask; the mask is also a two-dimensional matrix, in which elements belonging to the ground area are marked as valid values ​​(such as 1), and elements that do not belong to the ground area are marked as invalid values ​​(such as 0).

[0061] Then, the management server can fuse the distance mask and the ground area mask. The fusion method can be a logical AND operation, that is, only when a certain position satisfies both the ground area and the distance is within the valid range, the position is marked as the target ground area; the fused mask will be a two-dimensional matrix that combines the distance information and the ground area information, in which the value of each element indicates whether the position belongs to the target ground area. Based on the fused mask, the target ground area can be easily extracted, for example, the image area corresponding to the element marked as a valid value in the mask can be extracted as the final target ground area.

[0062] Step S150, drawing the real-time trajectory of the current vehicle according to the target ground area and the vehicle position data.

[0063] In this embodiment, the management server can draw the real-time trajectory of the current vehicle based on the target ground area and the vehicle location data. The management server can obtain the current vehicle location data through GPS, inertial navigation system (INS) or other positioning technology, including longitude, latitude, altitude (optional) and possible direction information. In addition, to ensure the continuity and accuracy of the trajectory, it is also necessary to determine the timestamp associated with each location data.

[0064] Specifically, the management server needs to ensure that all location data are correctly aligned to the same coordinate system; sort the location data according to the timestamp to ensure the continuity of the trajectory line. For each location data point, project it onto the target ground area; this may require considering the curvature of the earth (for large-scale applications) or a simple two-dimensional plane projection (for small-scale applications); mark these projection points as trajectory points on the image or map of the target ground area; in addition, the management server also uses appropriate interpolation algorithms (such as linear interpolation, spline interpolation, etc.) to connect adjacent trajectory points to form a smooth trajectory line; ensure that the directionality of the trajectory line is consistent with the actual movement direction of the vehicle; at the same time, the trajectory line can also be superimposed on the image or map of the target ground area for visualization, and different trajectory segments can be distinguished or specific status information (such as speed, acceleration, etc.) can be represented by using different colors, line types or markers.

[0065] The management server can dynamically draw a track line that fits the ground using a drawing algorithm based on the effective ground area obtained through fusion analysis. At the same time, the track line is updated in real time according to the steering angle and movement status of the vehicle, and the color of the track line is dynamically adjusted according to the distance and urgency of the obstacle. This technology not only improves the accuracy and real-time performance of the track line, but also provides intuitive visual guidance to the driver through color changes, enhancing the safety and comfort of the vehicle during driving.

[0066] In one embodiment, before step 150, the method further includes: obtaining the current steering angle of the vehicle in real time through a steering wheel angle sensor of the vehicle body.

[0067] Among them, the steering wheel angle sensor (SAS) is a device used to measure the steering wheel rotation angle and rotation direction. It is widely used in traditional steering systems as well as advanced driver assistance systems and self-driving cars. The steering wheel angle sensor can obtain the vehicle's steering angle in real time. This is because its internal components can quickly generate corresponding signals as the steering wheel turns. After these signals are processed, they can immediately provide accurate steering angle information to the vehicle's electronic control system. By obtaining the steering angle, the vehicle's position information can be determined in order to generate the subsequent auxiliary reverse trajectory.

[0068] In one embodiment, step 150 may specifically include the following steps:

[0069] The color of the track line is determined according to the real-time distance; the vertex of the steering angle is extracted from a preset track line data set; an initial track line is drawn according to the extracted vertex angle; and the initial track line is fused with the target ground area through mask data to obtain a real-time track line.

[0070] The management server can process the distance in segments, that is, divide the possible distance range into several intervals, and assign a specific color to each distance interval. For example, the distance can be divided into three intervals: short distance, medium distance, and long distance. The short distance can be represented by red to warn of potential collision risks; the medium distance can be represented by yellow to remind the driver to pay attention; and the long distance can be represented by green to indicate a safe distance. At this time, the management server can select the corresponding color to draw the trajectory line according to the interval to which the distance belongs.

[0071] The trajectory data set may include the data of the trajectory generated by the current vehicle, so the vertex of the steering angle may be extracted from the preset trajectory data set, and the coordinates of the vertex may be obtained based on the vehicle's dynamic model or actual test data. When the vehicle is driving, the current steering angle is obtained through the steering wheel angle sensor, and then the trajectory vertex that best matches the steering angle is found in the preset data set for subsequent trajectory drawing. Specifically, the extracted trajectory vertices may be connected using an appropriate interpolation algorithm (such as linear interpolation, spline interpolation, etc.) to form a smooth initial trajectory, and the initial trajectory is smoothed to reduce fluctuations or jitters caused by the interpolation algorithm.

[0072] The management server can also compare the initial trajectory line with the mask generated based on the target ground area obtained from the previous processing, and check whether each point on the trajectory line is within the effective area of ​​the mask to crop the part of the trajectory line that is not in the target ground area; for the cropped part of the trajectory line, appropriate adjustments or extensions can be made as needed to ensure the continuity and integrity of the trajectory line; at the same time, the management server also needs to continuously update the trajectory line as the vehicle moves and the steering angle changes.

[0073] like Figure 4 FIG. 2 is a flowchart of another real-time example of the present invention. The trajectory line dynamic adjustment method may further include:

[0074] 1. The camera collects images;

[0075] 2. Bilateral filtering denoising;

[0076] 3. Segment the image by watershed to identify the effective ground area;

[0077] 4. Ultrasonic radar ranging;

[0078] 5. Does the radar have obstacle data?

[0079] 6. If yes, the mask data is fused, the valid ground area is updated, and step 7 is executed;

[0080] 7. If not, draw a track line based on the valid ground area.

[0081] Among them, the present invention uses bilateral filtering to pre-process the image, effectively removes the noise, and retains the edge details of the image; then, the watershed algorithm is used to segment the image, which can more accurately identify the ground ROI area; this combination method significantly improves the accuracy of image processing compared to the traditional single filtering or segmentation algorithm; thereby improving the recognition accuracy of the ground ROI area and reducing the misjudgment and missed judgment. At the same time, it also combines visual segmentation processing and accurate ultrasonic radar data to improve the accuracy of the data. Finally, by dynamically adjusting the position and color of the track line, the track line is only displayed in the effective ground area, providing intuitive visual guidance for the driver. This guidance method is not only easy to understand, but also can reflect the driving status of the vehicle and changes in the surrounding environment in real time.

[0082] In summary, the above-mentioned trajectory line dynamic adjustment method is applied to the management server. This method can not only make the reversing process more intuitive and easy to understand through the dynamic adjustment and real-time display of the trajectory line, thereby improving the driver's driving experience; at the same time, by dynamically adjusting the trajectory line, the driver can more accurately judge the actual distance between the vehicle and the obstacle, thereby avoiding potential collision risks and improving safety; in addition, this solution can also be applied to different types of vehicles and driving environments, and has strong versatility and adaptability.

[0083] The embodiment of the present invention further provides a trajectory line dynamic adjustment device, which can be configured in a management server, and is used to execute any embodiment of the aforementioned trajectory line dynamic adjustment method. Figure 5 , Figure 5 A schematic block diagram of a trajectory dynamic adjustment device provided in an embodiment of the present invention.

[0084] like Figure 5 As shown, a trajectory line dynamic adjustment device 100 is applied to a management server, and includes an image acquisition unit 110 , a pre-processing unit 120 , a real-time distance measurement unit 130 , a data processing unit 140 and a trajectory line drawing unit 150 .

[0085] The image acquisition unit 110 is used to acquire real-time environment images acquired by the camera module.

[0086] In this embodiment, the camera module may be a high-definition camera or other types of cameras. The camera module can capture images of the rear of the vehicle in real time. That is, the management server can obtain the real-time environment image collected through the camera, and the real-time environment image may be an image of the rear of the current vehicle.

[0087] The preprocessing unit 120 is used to preprocess the real-time environment image according to a preset visual segmentation rule to obtain a valid surface area.

[0088] In this embodiment, the management server can pre-process the acquired real-time environment image according to the preset visual segmentation rules to obtain the effective ground area. For example, the real-time environment image can be converted into a grayscale image, and the grayscale image can be pre-processed with bilateral filtering to reduce noise interference in the grayscale image while retaining edge details. Then, the grayscale image contour is searched, and the contour of the rear position of the vehicle is marked as white, which is passed as a marking parameter to the watershed algorithm to segment the image, so that the part close to the rear of the vehicle is marked as the foreground, and the preliminary ground ROI area, that is, the effective ground area, is identified.

[0089] Among them, the ground ROI area, namely the ground region of interest (Region of Interest), is mainly used in computer vision, image processing, remote sensing image analysis and other fields. Usually this area contains key information that needs to be identified, analyzed or processed, such as specific landforms, buildings, vegetation types, etc.

[0090] In one embodiment, the preprocessing unit 120 is further configured to perform the following steps:

[0091] The real-time environment image is converted into a grayscale image; the grayscale image is pre-processed for denoising using bilateral filtering; the image contour of the denoised grayscale image is determined; and the image contour is segmented according to a preset watershed algorithm and marking parameters to obtain a valid surface area.

[0092] After acquiring the real-time environment image through a camera or other image acquisition device, the color real-time environment image is converted into a grayscale image using an image processing algorithm to simplify the image information, reduce the amount of calculation for subsequent processing, and retain sufficient image features for subsequent processing. The image processing algorithm can be an algorithm for conversion based on the values ​​of the three RGB color channels. Common conversion methods include the maximum value method, the average value method, and the weighted average value method; for example, the weighted average value method weights according to the sensitivity of the human eye to different colors, and the weights commonly used are R: 0.299, G: 0.587, and B: 0.114.

[0093] The bilateral filtering algorithm is a filtering algorithm that combines spatial domain and value domain information. It can not only remove noise from the image, but also avoid over-smoothing while retaining the edge features of the image. The design of the bilateral filter is based on the Gaussian function, and the weight is determined by calculating the spatial distance and brightness difference between each pixel and the surrounding pixels, thereby achieving the filtering effect. The management server uses bilateral filtering to perform denoising preprocessing on the grayscale image, which can effectively remove noise from the image while retaining the edge features of the image.

[0094] The management server can also use edge detection algorithms to extract image contours, such as Canny edge detector, Sobel operator, Prewitt operator, etc. These algorithms detect edges by calculating the gradient or difference of pixel points in the image, thereby obtaining image contour information.

[0095] Watershed algorithm: This is a morphological segmentation method based on topological theory. It regards the image as a geodesic topological landform and uses gradient information to simulate the waterlogging process, so as to construct a watershed in the image to segment different areas. In practical applications, in order to improve the accuracy of segmentation, marker parameters (such as seed points, marker images, etc.) are usually combined to guide the segmentation process of the watershed algorithm. Setting of marker parameters: The setting of marker parameters is crucial to the segmentation effect of the watershed algorithm. This usually needs to be determined according to the specific application scenario and image features. For example, when extracting the ground area, the watershed algorithm can be guided to accurately segment the ground area by setting reasonable seed points and marker images.

[0096] Therefore, the management server can convert the real-time environment image into a grayscale image, and use bilateral filtering to pre-process the real-time environment image to reduce noise interference in the real-time environment image while retaining edge details. Then find the grayscale image contour, and mark the contour of the rear position of the current vehicle as white, and pass it as a mark parameter to the watershed algorithm to segment the grayscale image, so as to mark the part close to the rear of the vehicle as the foreground, so as to identify the preliminary ground ROI area, that is, to obtain the effective ground area.

[0097] The innovative application of the watershed algorithm for ground and obstacle segmentation in reversing scenarios effectively solves the problem of ground and obstacle segmentation, improves the accuracy and stability of ground recognition, provides strong support for subsequent fusion analysis, and enhances the safety of vehicles during driving.

[0098] In a further embodiment, before executing the step of segmenting the image contour according to a preset watershed algorithm and marking parameters to obtain a valid ground area, the preprocessing unit 120 is also used to execute: marking the contour of the parking position of the current vehicle with a preset identification color; and determining the contour mark marked with the identification color as a marking parameter.

[0099] The management server can also mark the outline of the parking space position of the current vehicle as a preset identification color, determine the outline mark by the identification color, and determine the outline mark as a marking parameter.

[0100] The real-time distance measuring unit 130 is used to obtain the real-time distance between the obstacle and the current vehicle collected by the radar distance measuring module.

[0101] In this embodiment, the management server can collect the real-time distance between the obstacle and the current vehicle through the radar ranging module. The working principle of the radar ranging module is based on radar technology, which mainly calculates the distance between the object and the ranging module by sending electromagnetic waves and detecting the reflected signal. The core of radar ranging is to calculate the time difference between the transmission and reception of the radar signal to determine the target distance. Its main technical means include pulse ranging, frequency modulated continuous wave and pulse compression technology. In addition, if the distance between the obstacle and the current vehicle is greater than 300cm, it will be ignored.

[0102] Therefore, the comprehensive application of radar ranging and image fusion analysis technology not only improves the accuracy and real-time performance of obstacle detection, but also realizes the precise identification of effective ground areas, providing a reliable basis for drawing trajectory lines during vehicle driving.

[0103] In one embodiment, the real-time ranging unit 130 is further used to perform the following steps: record the emission time point of the ultrasonic wave emitted by the radar ranging module and the reception time point of the ultrasonic wave returned by the obstacle; and calculate the real-time distance between the obstacle and the current vehicle based on the time difference between the reception time point and the emission time point.

[0104] When the ultrasonic radar ranging module is triggered, an ultrasonic pulse is emitted, and the triggering moment can be recorded as the emission time point. When the ultrasonic pulse encounters an obstacle, it will be reflected back and received by the radar ranging module, and the reception moment can be recorded as the reception time point. The time difference refers to the difference between the reception time point and the emission time point, which indicates the time from the emission to the reception of the ultrasonic pulse.

[0105] The distance between the obstacle and the radar ranging module can be calculated based on the propagation speed of ultrasound in the air (usually 340 meters per second, but this speed may vary slightly due to factors such as temperature and air pressure) and the time difference. The specific calculation formula is as follows: distance = (time difference × speed of sound) / 2. The " / 2" here is because the ultrasound needs to travel back and forth between the obstacle and the radar ranging module, so the actual distance is half of the distance corresponding to the time difference.

[0106] The data processing unit 140 is used to perform mask data fusion according to the effective ground area and the real-time distance to obtain a target ground area.

[0107] In this embodiment, the distance and orientation information of the obstacle is in the ground plane of the world coordinate system, and generally requires projection transformation to convert it to the same screen coordinate system as the camera image. Then, the mask is fused with the ground ROI area to exclude the part occupied by the obstacle in the ground ROI area, thereby updating the effective ground area, which does not contain any obstacles. Specifically, in the combined application of image processing and radar ranging technology, the management server can perform mask data fusion based on the effective ground area and the real-time distance information obtained by the radar ranging module, so as to accurately extract the target ground area.

[0108] In one embodiment, the data processing unit 140 can specifically perform the following steps: determine the two-dimensional position image of the obstacle in the standard coordinate system based on the real-time distance and real-time orientation information of the obstacle; convert the two-dimensional position image to the same coordinate system as the real-time environment image according to a preset projection change rule to obtain an intermediate position image; and perform mask data fusion on the intermediate position image and the effective ground area to obtain the target ground area.

[0109] like Figure 3 As shown in the figure, it is a generated diagram between the trajectory line and the obstacle after ranging. For example, the management server can generate a distance mask for each measured point or area based on the real-time distance information provided by the radar ranging module. This mask can be a two-dimensional matrix, in which the value of each element represents the distance between the position and the vehicle; for points beyond the range of interest (such as the set maximum detection distance), their distance values ​​can be set to infinity or a specific invalid value so as to be excluded in subsequent processing. The management server can also use the valid ground area extracted by the image processing algorithm to convert it into a ground area mask; the mask is also a two-dimensional matrix, in which elements belonging to the ground area are marked as valid values ​​(such as 1), and elements that do not belong to the ground area are marked as invalid values ​​(such as 0).

[0110] Then, the management server can fuse the distance mask and the ground area mask. The fusion method can be a logical AND operation, that is, only when a certain position satisfies both the ground area and the distance is within the valid range, the position is marked as the target ground area; the fused mask will be a two-dimensional matrix that combines the distance information and the ground area information, in which the value of each element indicates whether the position belongs to the target ground area. Based on the fused mask, the target ground area can be easily extracted, for example, the image area corresponding to the element marked as a valid value in the mask can be extracted as the final target ground area.

[0111] The trajectory drawing unit 150 is used to draw the real-time trajectory of the current vehicle according to the target ground area and the vehicle position data.

[0112] In this embodiment, the management server can draw the real-time trajectory of the current vehicle based on the target ground area and the vehicle location data. The management server can obtain the current vehicle location data through GPS, inertial navigation system (INS) or other positioning technology, including longitude, latitude, altitude (optional) and possible direction information. In addition, to ensure the continuity and accuracy of the trajectory, it is also necessary to determine the timestamp associated with each location data.

[0113] Specifically, the management server needs to ensure that all location data are correctly aligned to the same coordinate system; sort the location data according to the timestamp to ensure the continuity of the trajectory line. For each location data point, project it onto the target ground area; this may require considering the curvature of the earth (for large-scale applications) or a simple two-dimensional plane projection (for small-scale applications); mark these projection points as trajectory points on the image or map of the target ground area; in addition, the management server also uses appropriate interpolation algorithms (such as linear interpolation, spline interpolation, etc.) to connect adjacent trajectory points to form a smooth trajectory line; ensure that the directionality of the trajectory line is consistent with the actual movement direction of the vehicle; at the same time, the trajectory line can also be superimposed on the image or map of the target ground area for visualization, and different trajectory segments can be distinguished or specific status information (such as speed, acceleration, etc.) can be represented by using different colors, line types or markers.

[0114] The management server can dynamically draw a track line that fits the ground using a drawing algorithm based on the effective ground area obtained through fusion analysis. At the same time, the track line is updated in real time according to the steering angle and movement status of the vehicle, and the color of the track line is dynamically adjusted according to the distance and urgency of the obstacle. This technology not only improves the accuracy and real-time performance of the track line, but also provides intuitive visual guidance to the driver through color changes, enhancing the safety and comfort of the vehicle during driving.

[0115] In one embodiment, before the trajectory drawing unit 150, the device 100 further includes: an angle acquisition unit, which is used to acquire the current steering angle of the vehicle in real time through a steering wheel angle sensor of the vehicle body.

[0116] Among them, the steering wheel angle sensor (SAS) is a device used to measure the steering wheel rotation angle and rotation direction. It is widely used in traditional steering systems as well as advanced driver assistance systems and self-driving cars. The steering wheel angle sensor can obtain the vehicle's steering angle in real time. This is because its internal components can quickly generate corresponding signals as the steering wheel turns. After these signals are processed, they can immediately provide accurate steering angle information to the vehicle's electronic control system. By obtaining the steering angle, the vehicle's position information can be determined in order to generate the subsequent auxiliary reverse trajectory.

[0117] In one embodiment, the trajectory drawing unit 150 may specifically perform the following steps:

[0118] The color of the track line is determined according to the real-time distance; the vertex of the steering angle is extracted from a preset track line data set; an initial track line is drawn according to the extracted vertex angle; and the initial track line is fused with the target ground area through mask data to obtain a real-time track line.

[0119] The management server can process the distance in segments, that is, divide the possible distance range into several intervals, and assign a specific color to each distance interval. For example, the distance can be divided into three intervals: short distance, medium distance, and long distance. The short distance can be represented by red to warn of potential collision risks; the medium distance can be represented by yellow to remind the driver to pay attention; and the long distance can be represented by green to indicate a safe distance. At this time, the management server can select the corresponding color to draw the trajectory line according to the interval to which the distance belongs.

[0120] The trajectory data set may include the data of the trajectory generated by the current vehicle, so the vertex of the steering angle may be extracted from the preset trajectory data set, and the coordinates of the vertex may be obtained based on the vehicle's dynamic model or actual test data. When the vehicle is driving, the current steering angle is obtained through the steering wheel angle sensor, and then the trajectory vertex that best matches the steering angle is found in the preset data set for subsequent trajectory drawing. Specifically, the extracted trajectory vertices may be connected using an appropriate interpolation algorithm (such as linear interpolation, spline interpolation, etc.) to form a smooth initial trajectory, and the initial trajectory is smoothed to reduce fluctuations or jitters caused by the interpolation algorithm.

[0121] The management server can also compare the initial trajectory line with the mask generated based on the target ground area obtained from the previous processing, and check whether each point on the trajectory line is within the effective area of ​​the mask to crop the part of the trajectory line that is not in the target ground area; for the cropped part of the trajectory line, appropriate adjustments or extensions can be made as needed to ensure the continuity and integrity of the trajectory line; at the same time, the management server also needs to continuously update the trajectory line as the vehicle moves and the steering angle changes.

[0122] The above-mentioned trajectory dynamic adjustment device can be implemented in the form of a computer program, which can be used in Figure 6 Runs on the computer device shown.

[0123] See also Figure 6 , Figure 6 is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device may be a user terminal or a server for executing a trajectory line dynamic adjustment method to perform automatic price comparison according to a trajectory line dynamic adjustment module imported into an operating system.

[0124] See also Figure 6 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a communication bus 501 , wherein the memory may include a storage medium 503 and an internal memory 504 .

[0125] The storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 may execute the trajectory dynamic adjustment method, wherein the storage medium 503 may be a volatile storage medium or a non-volatile storage medium.

[0126] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0127] The internal memory 504 provides an environment for the operation of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the trajectory dynamic adjustment method.

[0128] The network interface 505 is used for network communication, such as providing data information transmission, etc. Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device 500 to which the solution of the present invention is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0129] The processor 502 is used to run a computer program 5032 stored in the memory to implement the corresponding functions in the above-mentioned trajectory dynamic adjustment method.

[0130] Those skilled in the art will understand that Figure 6 The embodiments of the computer device shown in the figure do not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the computer device may only include a memory and a processor. In such embodiments, the structure and function of the memory and the processor are the same as those of the embodiment of the present invention. Figure 6 The embodiments shown are consistent and will not be described again here.

[0131] It should be understood that in the embodiment of the present invention, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0132] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps included in the above-mentioned trajectory line dynamic adjustment method are implemented.

[0133] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0134] In the several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. Units with the same function may also be combined into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be an electrical, mechanical or other form of connection.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0136] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a computer-readable storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a magnetic disk or an optical disk.

[0138] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A trajectory line dynamic adjustment method, characterized in that: The method is applied to a management server, and the method comprises: Acquire real-time environment images collected by the camera module; Preprocessing the real-time environment image according to a preset visual segmentation rule to obtain a valid surface area; Get the real-time distance between the obstacle and the current vehicle collected by the radar ranging module; Performing mask data fusion according to the effective ground area and the real-time distance to obtain a target ground area; The real-time trajectory line of the current vehicle is drawn according to the target ground area and the vehicle position data.

2. The trajectory dynamic adjustment method according to claim 1, characterized in that: The step of preprocessing the real-time environment image according to a preset visual segmentation rule to obtain a valid surface area includes: Converting the real-time environment image into a grayscale image; Performing denoising preprocessing on the grayscale image using bilateral filtering; Determine the image contour of the denoised grayscale image; The image contour is segmented according to a preset watershed algorithm and marking parameters to obtain a valid surface area.

3. The trajectory dynamic adjustment method according to claim 2, characterized in that: Before the step of segmenting the image contour according to the preset watershed algorithm and marking parameters to obtain the effective surface area, the method further includes: Mark the outline of the current parking space of the vehicle with a preset identification color; The contour markers marked with the identification color are determined as marker parameters.

4. The trajectory dynamic adjustment method according to claim 3, characterized in that: The step of obtaining the real-time distance between the obstacle and the current vehicle collected by the radar ranging module also includes: Record the time when the radar ranging module transmits the ultrasonic wave and the time when the ultrasonic wave returned by the obstacle is received; The real-time distance between the obstacle and the current vehicle is calculated based on the time difference between the receiving time point and the transmitting time point.

5. The trajectory dynamic adjustment method according to claim 4, characterized in that: The step of performing mask data fusion according to the effective ground area and the real-time distance to obtain the target ground area includes: Determine the two-dimensional position image of the obstacle in the standard coordinate system according to the real-time distance and real-time orientation information of the obstacle; The two-dimensional position image is converted into the same coordinate system as the real-time environment image according to a preset projection change rule to obtain an intermediate position image; The intermediate position image is fused with the effective ground area through mask data to obtain a target ground area.

6. The trajectory dynamic adjustment method according to claim 5, characterized in that: Before the step of drawing the real-time trajectory of the current vehicle according to the target ground area and the vehicle position data, the method further includes: The current steering angle of the vehicle is obtained in real time through the steering wheel angle sensor of the vehicle body.

7. The trajectory dynamic adjustment method according to claim 6, characterized in that: If the step of drawing the real-time trajectory of the current vehicle according to the target ground area and the vehicle position data comprises: Determine the color of the track line according to the real-time distance; Extracting the vertex of the steering angle from a preset trajectory data set; Draw an initial trajectory line according to the extracted vertex angle; The initial trajectory is fused with the mask data of the target ground area to obtain the real-time trajectory.

8. A trajectory dynamic adjustment device, characterized in that: The device is applied to a management server, and comprises: An image acquisition unit, used to acquire real-time environment images acquired by a camera module; A preprocessing unit, used for preprocessing the real-time environment image according to a preset visual segmentation rule to obtain a valid surface area; A real-time distance measurement unit is used to obtain the real-time distance between the obstacle and the current vehicle collected by the radar distance measurement module; A data processing unit, used for performing mask data fusion according to the effective ground area and the real-time distance to obtain a target ground area; The trajectory drawing unit is used to draw the real-time trajectory of the current vehicle according to the target ground area and the vehicle position data.

9. A computer device, characterized in that: The device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the trajectory dynamic adjustment method described in any one of claims 1-7 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the trajectory line dynamic adjustment method according to any one of claims 1 to 7 are implemented.

Images