Trolley collision detection method based on three-dimensional collision bodies

Abstract

The application provides a trolley collision detection method based on a three-dimensional collision body, relates to the technical field of data processing, and comprises the following steps: collecting images in front of a driving path of a vehicle, calculating depth values of pixel points in each image according to the images, extracting each obstacle region in the images, obtaining obstacle structure data, analyzing stability of edges of the obstacle regions, identifying a flexible region in the images, obtaining flexible region data, identifying a dynamic morphological change process of the flexible region, determining a voxel distribution sequence of the flexible region changing with time, constructing a three-dimensional collision body model, performing spatial overlap analysis on the three-dimensional collision body model according to a real-time spatial position and motion parameters of the vehicle, judging whether a collision event exists on the driving path of the target vehicle, and adjusting the driving path and the motion parameters of the vehicle according to the collision event when the collision event exists. The application detects whether a vehicle will collide by constructing a three-dimensional collision body.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a vehicle collision detection method based on three-dimensional collision bodies. BACKGROUND

[0002] Under the background of continuous development of image processing and physical simulation technology, three-dimensional collision bodies, as a component for simulating the collision behavior of objects in three-dimensional space, have been widely introduced into intelligent driving systems. A common implementation is to obtain image data of the front road through a vehicle-mounted camera to generate a depth map of the scene, then convert the road obstacles into three-dimensional voxels or grid structures, and construct corresponding three-dimensional collision bodies to analyze the contact possibility of the target vehicle and objects in the environment, thereby assisting the vehicle to avoid obstacles.

[0003] In the automatic driving scene, to ensure driving safety, the vehicle body and obstacles are usually modeled as three-dimensional bounding boxes, and whether there is potential overlap between the two is calculated through continuously updated position and attitude information. This collision detection technology performs well in urban roads, especially for the identification and avoidance of explicit objects such as pedestrians, vehicles, and curbs, but its recognition ability for abnormal objects may be limited.

[0004] For example, when encountering a flexible object such as a plastic cloth that suddenly falls on the road, the existing image analysis system often cannot construct stable three-dimensional collision bodies due to the lack of rigid boundary information, resulting in volume collapse or position drift of the obstacle model, which may cause the vehicle system to misjudge that there is no substantial collision risk with the object, and thus fail to avoid it in time. SUMMARY

[0005] The purpose of the present application is to provide a vehicle collision detection method based on three-dimensional collision bodies, aiming to solve the problems mentioned in the background.

[0006] To solve the above technical problems, the technical solution of the present application is as follows:

[0007] The vehicle collision detection method based on three-dimensional collision bodies comprises:

[0008] Collecting images in front of the vehicle driving path, and calculating the depth value of each pixel point in each image to construct the corresponding spatial mapping relationship and obtain depth mapping data;

[0009] According to the depth mapping data, extracting each obstacle region in the image to obtain obstacle structure data;

[0010] According to the obstacle structure data, analyzing the stability of the edge of the obstacle region to identify the flexible region in the image and obtain flexible region data;

[0011] According to the flexible region data, a dynamic morphological change process of the flexible region is identified, a voxel distribution sequence of the flexible region changing over time is determined, and flexible structure data is obtained;

[0012] The flexible structure data and the obstacle structure data are merged to construct a three-dimensional collision body model;

[0013] According to the real-time spatial position and motion parameters of the vehicle, spatial overlap analysis is performed on the three-dimensional collision body model to determine whether a collision event exists on the driving path of the target vehicle;

[0014] When a collision event exists, the collision region information is obtained, and the driving path and motion parameters of the vehicle are adjusted according to the collision region information.

[0015] Further, images in front of the driving path of the vehicle are collected, and the depth values of the pixel points in each image are calculated according to the images to construct a corresponding spatial mapping relationship to obtain depth mapping data, including:

[0016] The continuous images in front of the driving path of the vehicle are collected by the image collection device, and arranged in time sequence to obtain a set of original images;

[0017] According to the set of original images, feature points containing local texture, corner points and boundary features are extracted to obtain a set of feature points;

[0018] According to the spatial distribution of the set of feature points in the continuous images, the corresponding relationship of each feature point in different images is determined to obtain pixel point matching data;

[0019] According to the pixel point matching data, the pixel position offset of each pixel point in different images is calculated, and the depth value of each pixel point is calculated according to the pixel position offset to obtain a set of depth values;

[0020] According to the set of original images and the set of depth values, the mapping relationship of each pixel point in the three-dimensional spatial coordinates is determined to obtain the depth mapping data.

[0021] Further, according to the pixel point matching data, the pixel position offset of each pixel point in different images is calculated, and the depth value of each pixel point is calculated according to the pixel position offset to obtain a set of depth values, including:

[0022] According to the pixel point matching data, the coordinates of the pixel points in each image in the set of original images are extracted, and the spatial displacement of the pixel points in the continuous images is calculated;

[0023] According to the pixel point matching data, the feature region of the pixel point is determined, and the feature region density correction value is calculated through the number of feature points in the feature region and the area of the feature region; the included angle between the motion directions of adjacent pixel points is calculated to obtain the motion trend angle;

[0024] The spatial displacement of the pixel point, the feature region density correction value and the motion trend angle are accumulated to obtain a total amount of motion change of the pixel point, and the total amount of motion change is converted into a distance unit to calculate a depth value in combination with a displacement of the camera and a focal length.

[0025] Further, the identification module comprises:

[0026] According to the obstacle structure data, the stability of the edge of the obstacle region is analyzed, a flexible region in the image is identified, and flexible region data is obtained, including:

[0027] According to the obstacle structure data, boundary points of each obstacle region in consecutive images are extracted to obtain boundary contour data;

[0028] According to the boundary contour data, a boundary contour change trajectory of each obstacle region in consecutive image frames is identified to obtain boundary contour trajectory data;

[0029] According to the boundary contour trajectory data, a displacement change amount of the boundary point is calculated to obtain the stability of the obstacle region, and the stability is compared with a preset stability range to obtain boundary stability evaluation data;

[0030] According to the boundary stability evaluation data, an obstacle region with insufficient stability is determined to obtain the flexible region data.

[0031] Further, according to the boundary contour trajectory data, a displacement change amount of the boundary point is calculated to obtain the stability of the obstacle region, and the stability is compared with a preset stability range to obtain boundary stability evaluation data, including:

[0032] According to the obstacle structure data, coordinates of the boundary point are extracted, a displacement change amount of the boundary point is calculated, and displacement change amounts of all images are accumulated to obtain a total amount of accumulated displacement of the obstacle region;

[0033] According to a forward speed of the vehicle, an image frame interval acquisition time, a focal length of the camera, an angle between a driving direction of the vehicle and a horizontal axis of an image coordinate system, a pitch angle between the camera and a horizontal line of the ground, a depth value of the boundary point and an actual physical length of a pixel, an expected pixel displacement amount of the boundary point in consecutive frame images is predicted;

[0034] According to the total amount of accumulated displacement and the expected pixel displacement amount, the stability of the obstacle region is determined;

[0035] According to the stability of the obstacle region and a preset stability range, it is determined that when the stability of the obstacle region is not within the preset stability range, the obstacle region is marked as a non-rigid region.

[0036] Further, according to the flexible region data, a dynamic morphological change process of the flexible region is identified, a voxel distribution sequence of the flexible region changing over time is determined, and flexible structure data is obtained, including:

[0037] According to the flexible region data, a position change trajectory of the flexible region in the continuous images is identified, and a trajectory data set is obtained;

[0038] According to the trajectory data set, a displacement change amount and a rotation change amount of the flexible region in the time sequence are extracted, and pose change data is obtained;

[0039] According to the pose change data, a shape contour change trend of the flexible region between the continuous images is identified, and morphological change data is obtained;

[0040] According to the morphological change data, the flexible region at each time is spatially discretized to form a single-frame voxel set;

[0041] The single-frame voxel set is arranged in time sequence, the voxel distribution sequence of the flexible region changing over time is determined, and flexible structure data is obtained.

[0042] Further, according to the trajectory data set, a displacement change amount and a rotation change amount of the flexible region in the time sequence are extracted, and pose change data is obtained, including:

[0043] According to the trajectory data set, the spatial position of the center point of the flexible region in the continuous images is extracted, and trajectory point data is obtained;

[0044] According to the trajectory point data, the three-dimensional spatial displacement amount of the center point of the flexible region between the continuous images is calculated, and displacement change amount data is obtained;

[0045] According to the boundary contour shape change of the flexible region in the continuous images, a direction vector is extracted, and direction vector data is obtained;

[0046] According to the direction vector data, the rotation angle change of the flexible region between the continuous images is calculated, and rotation change amount data is obtained;

[0047] The displacement change amount data and the rotation change amount data are combined to determine the overall motion state of the flexible region, and the pose change data is obtained.

[0048] Further, according to the real-time spatial position and motion parameters of the vehicle, spatial overlap analysis is performed on the three-dimensional collision body model to determine whether a collision event exists for the target vehicle on the driving path, including:

[0049] According to the vehicle positioning device, the spatial position of the vehicle in the driving path is collected in real time, and vehicle spatial position data is obtained;

[0050] According to the vehicle sensor, the speed, acceleration and heading angle parameters of the vehicle are extracted to obtain vehicle motion parameter data;

[0051] According to the vehicle spatial position data and the vehicle motion parameter data, it is predicted whether the vehicle has spatial overlap with the three-dimensional collision body model to obtain overlap data;

[0052] According to the overlap data, it is judged whether there is a collision risk on the driving path of the vehicle, and when the result is yes, a collision event is generated.

[0053] Further, according to the vehicle spatial position data and the vehicle motion parameter data, it is predicted whether the vehicle has spatial overlap with the three-dimensional collision body model to obtain overlap data, including:

[0054] According to the vehicle spatial position data and the vehicle motion parameter data, the predicted trajectory point of the vehicle on the driving path is calculated to obtain vehicle predicted trajectory data;

[0055] According to the vehicle size, the vehicle contour is extended outward at equal intervals to generate extended vehicle contour data, and the extended vehicle contour data is arranged continuously along the vehicle predicted trajectory data to build an extended trajectory area of the vehicle on the driving path to obtain extended trajectory area data;

[0056] The extended trajectory area data and the three-dimensional collision body model are compared in spatial position to detect whether the extended trajectory area overlaps with the three-dimensional collision body model to obtain overlap data.

[0057] Further, when there is a collision event, the collision area information is obtained, and the driving path and the motion parameter of the vehicle are adjusted according to the collision area information, including:

[0058] According to the collision event, the collision area number and the corresponding three-dimensional spatial position of the collision are extracted to obtain collision area information;

[0059] According to the collision area information, the area in the neighborhood of the vehicle where the collision exists is removed to obtain passable area data;

[0060] According to the passable area data, the risk avoidance path of the vehicle is planned, and safety detection is performed to ensure that the vehicle does not have spatial overlap with the three-dimensional collision body model;

[0061] According to the risk avoidance path, the speed, acceleration and heading angle parameters of the vehicle are adjusted to ensure that the vehicle drives according to the risk avoidance path.

[0062] The above scheme of the present application at least includes the following beneficial effects:

[0063] The application can identify the flexible region in the image by analyzing the edge stability in the obstacle structure data, and further track the dynamic change, break through the problem of model collapse or drift in the prior art when dealing with flexible obstacles, and the flexible region often shows unstable boundary characteristics in continuous image frames due to material or structural characteristics, and the existing system mainly relies on a static three-dimensional boundary box model, which cannot capture these change characteristics, thereby causing the flexible obstacle to be not effectively identified, through boundary stability evaluation and flexible region dynamic modeling, even if the obstacle is a flexible object with non-rigidity and large edge fluctuation, such as plastic cloth, film or soft bag, etc., the existence of the obstacle in space can be accurately identified, and the dynamic characteristics thereof are effectively modeled, thereby avoiding the collision risk caused by misjudgment, and realizing the adaptability improvement of the diversified obstacle environment.

[0064] The application constructs a spatial model of the obstacle through depth mapping data, and forms a complete three-dimensional collision body model in combination with flexible structure data, and performs spatial overlap analysis of the vehicle and the three-dimensional collision body on this basis, compared with the traditional system relying on two-dimensional image analysis, the method introduces depth information and flexible characteristics, so that the collision detection is extended from the plane to the real three-dimensional space, and has dynamic adaptability, the real-time spatial position and motion parameters of the vehicle are fully considered, accurate collision prediction in the dynamic environment is realized, not only suitable for detection of static obstacles, but more importantly, when facing flexible objects with changing shapes, the potential collision possibility can still be accurately calculated, thereby improving the response ability of the vehicle to complex road environment, effectively preventing accidents caused by insufficient obstacle modeling, and ensuring the stability and safety in the driving process.

[0065] The application constructs a three-dimensional model of the obstacle, especially a voxel distribution sequence of the flexible obstacle changing over time is tracked in real time, so that the system can reflect the latest state of the environment in real time, and is not limited by the lag of the traditional static modeling method, under high-speed driving or variable road conditions, such as object change caused by wind blowing or disturbance of the vehicle itself, the system can adjust the collision detection judgment in time through the continuously updated flexible structure data, maintain efficient and accurate response ability, greatly improve the real-time performance and environmental adaptability of the collision detection system, and ensure that the vehicle can still make accurate obstacle avoidance decisions even in a complex and dynamic external environment, and avoid judgment errors caused by information lag.

[0066] The application can construct a three-dimensional collision body model based on image data, further introduce the spatial position and motion parameters of the vehicle, comprehensively analyze the vehicle speed, acceleration, heading angle and other information, predict the future motion trajectory of the vehicle and perform overlapping analysis with the obstacle model, which is superior to the single collision judgment based on image features, improves the comprehensiveness and accuracy of the collision detection, especially in complex driving situations such as curved roads, uphill or downhill and other special road forms, the method can combine the current motion state of the vehicle to dynamically predict the possible collision risk in the future for a certain period of time, provide a more scientific basis for path adjustment for the vehicle, effectively avoid the limitations of simple path overlapping judgment, accurately calculate the collision possibility according to the specific motion trend, enhance the prediction ability of the system to the sudden risk, and thus improve the overall driving safety level.

[0067] The application plans the optimal risk avoidance path through comprehensive analysis of the three-dimensional collision body model and the extended trajectory region, and performs real-time safety verification on the path, ensures that the new path has no other potential collision risks, when the system determines that there is a potential collision event, not only can the specific collision region information be identified, but also the drivable area is analyzed to dynamically adjust the driving path and motion parameters of the vehicle, this active and intelligent risk avoidance mechanism enables the vehicle to have autonomous path optimization capability when facing complex obstacle environment, effectively improves the continuity and comfort of driving, improves the intelligent level of the automatic driving system, can adapt to the changes of obstacles in real time, and ensures that the vehicle is always in a safe passing state. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is the flow chart of the trolley collision detection method based on the three-dimensional collision body provided by the embodiment of the application. DETAILED DESCRIPTION

[0069] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0070] As Figure 1 shown, the embodiment of the application proposes a trolley collision detection method based on a three-dimensional collision body, which comprises:

[0071] Collecting images in front of the driving path of the vehicle, and calculating the depth value of each pixel point in each image according to the depth mapping data;

[0072] According to the depth mapping data, the image is extracted to obtain the obstacle structure data;

[0073] According to the obstacle structure data, the stability of the edge of the obstacle region is analyzed, the flexible region in the image is identified, and flexible region data is obtained;

[0074] According to the flexible region data, the dynamic morphological change process of the flexible region is identified, the voxel distribution sequence of the flexible region with time is determined, and flexible structure data is obtained;

[0075] The flexible structure data and the obstacle structure data are merged to construct a three-dimensional collision body model;

[0076] According to the real-time spatial position and motion parameters of the vehicle, spatial overlap analysis is performed on the three-dimensional collision body model to determine whether there is a collision event on the driving path of the target vehicle;

[0077] When there is a collision event, the collision region information is obtained, and the driving path and motion parameters of the vehicle are adjusted according to the collision region information.

[0078] In the embodiment of the present application, the image in front of the driving path of the vehicle is collected, and the depth value of each pixel point in each image is calculated according to the image to construct a corresponding spatial mapping relationship to obtain depth mapping data. By constructing the spatial mapping relationship, the transition from two-dimensional image to three-dimensional space is realized, providing a spatial data basis for subsequent obstacle modeling. According to the depth mapping data, each obstacle region in the image is extracted to obtain obstacle structure data, which can accurately identify the obstacle region in the image and extract obstacle data with complete depth and morphological characteristics, improving the integrity and accuracy of subsequent model establishment. According to the obstacle structure data, the stability of the edge of the obstacle region is analyzed, the flexible region in the image is identified, and flexible region data is obtained, effectively separating rigid obstacles from flexible obstacles and improving the system's ability to identify flexible objects such as cloth and paper. According to the flexible region data, the dynamic morphological change process of the flexible region is identified, the voxel distribution sequence of the flexible region with time is determined, and flexible structure data is obtained, realizing accurate modeling of the dynamic change process of the flexible obstacle, so that the system can not only identify the current state of the flexible region, but also predict its future morphological change, improving the avoidance ability of dynamic obstacles.

[0079] By merging flexible structure data and obstacle structure data, a 3D collision body model is constructed, providing a 3D environment model with completeness, dynamism, and physical feature compatibility. This allows collision detection to extend beyond static objects to include dynamically changing areas, improving the overall comprehensiveness and realism of the detection. Based on the vehicle's real-time spatial position and motion parameters, spatial overlap analysis is performed on the 3D collision body model to determine whether a collision event exists on the target vehicle's driving path. This enables dynamic interactive analysis of the vehicle's future motion trend and the obstacle model, providing early warning of potential collision risks. When a collision event occurs, collision area information is acquired, and the vehicle's driving path and motion parameters are adjusted accordingly. This ensures that the vehicle can quickly make decisions after detecting a collision risk, adjust its driving path, and avoid danger, improving the vehicle's autonomous risk avoidance capabilities and overall operational safety and reliability.

[0080] Specifically, based on the depth mapping data, obstacle regions in the image are extracted to obtain obstacle structure data, including:

[0081] During vehicle operation, continuous images are acquired via an onboard camera. Based on the depth information of pixels in the images, a mapping relationship between the images and three-dimensional space is established. First, the acquired depth mapping data undergoes preliminary processing using noise suppression algorithms, such as bilateral filtering or median filtering, to remove depth anomalies caused by acquisition errors or lighting interference. Then, a clustering algorithm is used to partition the depth mapping data, grouping pixels with similar depth values ​​into the same group, thus initially separating potential obstacle areas.

[0082] Next, edge enhancement processing is performed on the separated regions. Image edge detection algorithms such as the Canny operator are used to identify the edge contours of each obstacle region. Based on the edge contours and the depth variation trend of each region, the boundary range of the obstacle is further precisely defined. On this basis, the geometric features of the obstacle are extracted, including volume, boundary shape, contour connectivity, and three-dimensional spatial position. This information is then transformed into structured obstacle structure data. To improve the stability of obstacle recognition, temporal consistency correction is introduced to verify the consistency of obstacle regions extracted from consecutive frames, confirming the stable existence of obstacles in space and removing temporary noise regions. The final obstacle structure data includes the three-dimensional position, boundary contour, depth distribution, and spatial dimensions of each obstacle, serving as the basis for subsequent flexibility analysis and collision body construction.

[0083] This involves merging flexible structure data and obstacle structure data to construct a three-dimensional collision body model, specifically including:

[0084] Based on the previously extracted obstacle structure data and flexible structure data, firstly, the spatial positions of the two types of data are aligned in coordinates, ensuring that they are accurately mapped in a unified three-dimensional space reference system. For the part marked as rigid in the obstacle structure data, the original boundary voxel definition is maintained, and a three-dimensional grid model of the rigid boundary is established to record its fixed spatial characteristics. For the flexible structure data, according to the voxel distribution sequence of the flexible region over time, the voxel set of the flexible region is arranged in time sequence, and each voxel is assigned dynamic attributes such as displacement vector and deformation coefficient, forming a flexible voxel model with time evolution characteristics.

[0085] In the merging process, a spatial interpolation method is used to transition the junction part of the rigid region and the flexible region, preventing cracks in the model boundary. Through voxel fusion technology, the flexible structure voxels and the rigid structure voxels are integrated into the same three-dimensional model to form a complete three-dimensional collision body. To improve the computational efficiency of the model, an octree data structure can be used to hierarchically divide the three-dimensional space voxels, optimizing the computational performance of the model in spatial overlap analysis. At the same time, the material properties of each obstacle region, such as rigidity or flexibility, are labeled and an attribute index is established to facilitate subsequent processing strategies based on obstacle characteristics in collision detection. The final three-dimensional collision body model not only reflects the overall shape of the obstacles in the environment, but also dynamically presents the change process of the flexible region, providing accurate spatial data support for real-time collision detection and path optimization of vehicles.

[0086] In a preferred embodiment of the present application, images in front of the vehicle driving path are collected, and the depth values of each pixel point in each image are calculated based on the images to construct the corresponding spatial mapping relationship to obtain depth mapping data, including:

[0087] The continuous images in front of the vehicle driving path are collected by the image collection device, and arranged in time sequence to obtain a set of original images;

[0088] According to the set of original images, feature points containing local texture, corner points and boundary features are extracted to obtain a set of feature points;

[0089] According to the spatial distribution of the set of feature points in the continuous images, the corresponding relationship of each feature point in different images is determined to obtain pixel point matching data;

[0090] According to the pixel point matching data, the pixel position offset of each pixel point in different images is calculated, and the depth value of each pixel point is calculated based on the pixel position offset to obtain a set of depth values;

[0091] According to the set of original images and the set of depth values, the mapping relationship of each pixel point in the three-dimensional space coordinates is determined to obtain the depth mapping data.

[0092] In the embodiment of the present application, the image acquisition device acquires continuous images in front of the vehicle driving path, and arranges them in time sequence to obtain a set of original images, ensuring that the vehicle can obtain continuous visual information of the current driving environment in real time, providing sufficient data basis for subsequent image feature extraction and depth calculation; according to the set of original images, feature points containing local texture, corner and boundary features are extracted to obtain a set of feature points, enhancing the system's ability to capture diversified obstacle features and improving the accuracy of subsequent model reconstruction; according to the spatial distribution of the set of feature points in the continuous images, the corresponding relationship of each feature point in different images is determined to obtain pixel point matching data, through high-precision pixel point matching, the reliability of subsequent depth value calculation is ensured, effectively avoiding the error of depth information caused by false matching; according to the pixel point matching data, the pixel position offset of each pixel point in different images is calculated, and the depth value of each pixel point is calculated according to the pixel position offset to obtain a set of depth values, realizing effective conversion from two-dimensional image to three-dimensional space information, so that the system can accurately grasp the actual position of each pixel point in space; according to the set of original images and the set of depth values, the mapping relationship of each pixel point in three-dimensional space coordinates is determined to obtain depth mapping data, through accurate space mapping, the vehicle can perceive and understand the three-dimensional environment, and the system's ability to analyze the spatial structure is improved.

[0093] Among them, according to the set of original images and the set of depth values, the mapping relationship of each pixel point in three-dimensional space coordinates is determined to obtain depth mapping data, specifically including:

[0094] After obtaining the set of original images and the corresponding set of depth values, the position of each pixel point in the image plane needs to be converted into the coordinates in the three-dimensional space to realize the complete mapping from two-dimensional visual information to three-dimensional space information. First, the two-dimensional pixel coordinates of each pixel point are extracted from the set of original images, which are usually represented in the form of image row and column indexes, denoted as . Among them represents the horizontal coordinate of the pixel point in the image, represents the vertical coordinate of the pixel point in the image. For each pixel point, combined with the corresponding depth value in the set of depth values , the position of the pixel point in the three-dimensional space can be deduced.

[0095] In order to realize the coordinate mapping from the image plane to the three-dimensional space, the coordinate transformation needs to be performed according to the intrinsic matrix of the camera. The intrinsic matrix of the camera includes focal length , and optical center coordinates The focal length parameter reflects the imaging proportional relationship of the camera, and the optical center coordinate represents the position of the image center point in the image coordinate system. Through the following formula, the two-dimensional coordinates of each pixel point and the depth value can be jointly operated to obtain the three-dimensional space coordinates :

[0096] , , ,

[0097] Among them, represents the horizontal actual physical position of the pixel point in the camera coordinate system, represents the vertical physical position, that is, the distance of the pixel point in front of the camera, that is, the depth value.

[0098] In the above calculation process, in order to improve the accuracy of three-dimensional space mapping, the distortion parameters of the camera need to be corrected. In the imaging process of the actual camera device, the image often appears radial or tangential distortion due to the geometric characteristics of the lens. By correcting the distortion of the original image, the pixel position is restored to the true imaging position, further improving the accuracy of the three-dimensional space coordinates. At the same time, in order to improve the operation efficiency and data consistency, the system uniformly normalizes all calculated three-dimensional coordinate points, so that the space data has comparability in different sampling periods, and can be used for subsequent obstacle structure modeling and three-dimensional collision body construction. Finally, the three-dimensional space coordinates of all pixel points are stored in order to form complete depth mapping data.

[0099] In a preferred embodiment of the present application, according to the pixel point matching data, the pixel position offset of each pixel point in different images is calculated, and the depth value of each pixel point is calculated according to the pixel position offset to obtain a depth value set, including:

[0100] According to the pixel point matching data, the coordinates of the pixel point in each image in the original image set are extracted, and the spatial displacement of the pixel point in the continuous image is calculated;

[0101] According to the pixel point matching data, the feature area of the pixel point is determined, and the feature area density correction value is calculated through the number of feature points in the feature area and the area of the feature area. The included angle between the motion directions of adjacent pixel points is calculated to obtain the motion trend included angle;

[0102] The spatial displacement of the pixel point, the feature area density correction value and the motion trend included angle are accumulated to obtain the total amount of motion change of the pixel point, and the total amount of motion change is converted into distance units. The depth value is calculated in combination with the displacement and focal length of the camera.

[0103] In this embodiment of the invention, based on pixel matching data, the coordinates of each pixel in the original image set are extracted, and the spatial displacement of the pixel in consecutive images is calculated. This accurately captures the motion trajectory of feature pixels in consecutive image frames, providing basic data for subsequent depth value calculation. Based on the pixel matching data, the feature regions of the pixels are determined, and the feature region density correction value is calculated using the number of feature points and the area of ​​the feature region. Through feature density analysis, high-texture regions and low-texture regions in the image are effectively distinguished, avoiding depth calculation errors caused by insufficient features in low-texture regions. The clamping distance between the motion directions of adjacent pixels is calculated. The angle is used to obtain the motion trend angle. The overall evaluation of the motion trend of adjacent pixels can determine whether the target pixel is in a consistent motion area, improving the ability to identify local motion consistency during depth estimation. The spatial displacement of the pixel, the feature region density correction value, and the motion trend angle are accumulated to obtain the total change of pixel motion, which is then converted into distance units. The depth value is calculated by combining the camera displacement and focal length. By comprehensively considering multiple factors, the depth estimation of the pixel is dynamically adjusted, which significantly improves the accuracy of the depth value. Combined with physical parameters, the depth distortion problem caused by system parameter differences in estimation methods based solely on pixel displacement is avoided.

[0104] The formula for calculating the depth value is as follows:

[0105] ,

[0106] in, The depth value of a pixel. An index for images in the original image set. The total number of images in the original image set. For this pixel point at the th The x-coordinate in each image For this pixel point at the th The ordinate in each image, For this pixel point at the th The x-coordinate in each image For this pixel point at the th The ordinate in each image, The distance the camera moves to capture the native image set. For camera focal length, This is the conversion ratio between pixel units and distance units. This represents the number of feature points within the feature neighborhood of that pixel. The radius of the feature neighborhood of this pixel. It is a constant. and For coefficients, For this pixel point at the th the motion angle generated between the first image and the second image, , , is the vertical coordinate of the adjacent pixel point in the first image, is the vertical coordinate of the adjacent pixel point in the first image, is the vertical coordinate of the adjacent pixel point in the first image, is the vertical coordinate of the adjacent pixel point in the first image, is the horizontal coordinate of the adjacent pixel point in the first image, is the horizontal coordinate of the adjacent pixel point in the first image, is the horizontal coordinate of the adjacent pixel point in the first image.

[0107] wherein, is the displacement of the pixel point in the continuous image frame, according to the motion parallax principle, the displacement of the near object is larger, and the displacement of the far object is smaller, so if the displacement is large, the depth is small, if the displacement is small, the depth is large, which is a mathematical embodiment of the depth value being inversely proportional to the pixel displacement, and is one of the core mechanisms of depth estimation.

[0108] wherein, is used to correct the image feature quality of the local area, the feature point density is high, the area information is rich, the depth estimation is reliable, the correction coefficient is improved, and the displacement influence of the area is emphasized; the feature point is sparse, the depth estimation is uncertain, the correction coefficient is reduced, and the contribution of the area to the overall depth is weakened; by introducing the feature density influence, the depth calculation is more robust, and errors caused by lack of information in low texture areas are avoided.

[0109] wherein, is the motion direction angle of the pixel point and the adjacent pixel point, reflecting the motion consistency of the local area, if the angle is small, that is, the motion consistency is high, then is approximately 0, and the correction term has little effect; if the angle is large, that is, the motion in the area is chaotic, then is increased, the denominator value is reduced, and the depth estimation reliability is reduced, which reflects the adaptive suppression of local motion abnormalities and strengthens the accuracy of the depth value in the stable motion area.

[0110] wherein, is a minimum constant, preventing the denominator from being zero and ensuring numerical stability. The coefficient is used to adjust the influence weight of the feature point density on the depth value estimation, which amplifies or weakens the effect of the feature point density of the local image area on the pixel displacement correction, usually taking a small positive number in the range of 0.1-1.0, if the value is too large, the feature density influence is increased, which may cause the depth calculation in the feature dense area to be too high, and if ​If the value is too small, the feature density influence is weakened, and the system's ability to distinguish texture regions is reduced. According to the image resolution and the performance of the feature extraction algorithm, the initial It can be set to 0.3, which is suitable for natural light environments in general road scenes. The coefficient is used to adjust the suppression or amplification of local pixel motion trend differences on depth estimation, reflecting the influence of pixel motion consistency within the region on depth reliability, and enhancing the resistance to local motion abnormalities, It is usually a small positive number in the range of 0.05-0.5, The larger it is, the stronger the motion trend angle's influence on depth correction, which may over-inhibit normal motion regions, The smaller it is, the weaker the response to local motion direction differences, which may cause abnormal motion to affect depth values. According to the pixel motion noise in common scenes, The initial value is recommended to be 0.2 in stable environments with few dynamic obstacles, It can be slightly smaller.

[0111] In a preferred embodiment of the present application, according to the obstacle structure data, the stability of the obstacle region edge is analyzed, the flexible region in the image is identified, and the flexible region data is obtained, including:

[0112] According to the obstacle structure data, the boundary points of each obstacle region in consecutive images are extracted, and the boundary contour data is obtained;

[0113] According to the boundary contour data, the boundary contour change trajectory of each obstacle region in consecutive image frames is identified, and the boundary contour trajectory data is obtained;

[0114] According to the boundary contour trajectory data, the displacement change of the boundary points is calculated, the stability of the obstacle region is obtained, and it is compared with the preset stability range to obtain the boundary stability evaluation data;

[0115] According to the boundary stability evaluation data, the obstacle region with insufficient stability is determined, and the flexible region data is obtained.

[0116] In the embodiment of the present application, according to the obstacle structure data, the boundary points of each obstacle region in the continuous images are extracted to obtain boundary contour data, and the spatial shape characteristics of the obstacle are accurately expressed by systematically extracting the boundary point information of the obstacle in the continuous images; according to the boundary contour data, the boundary contour change trajectory of each obstacle region in the continuous image frames is identified to obtain boundary contour trajectory data, the change trajectory of the obstacle boundary over time is completely recorded, and the dynamic characteristics of the obstacle region edge are reflected; according to the boundary contour trajectory data, the displacement change of the boundary point is calculated to obtain the stability of the obstacle region, and the boundary stability evaluation data is obtained by comparing it with the preset stability range, so that the quantification of the dynamic change of the obstacle boundary is realized, and the boundary stability of the flexible and rigid obstacle has a calculable and comparable basis; according to the boundary stability evaluation data, the obstacle region with insufficient stability is determined to obtain flexible region data, so that the flexible obstacle can be accurately identified, and the system can distinguish different physical properties of the obstacle type.

[0117] According to the boundary contour data, the boundary contour change trajectory of each obstacle region in the continuous image frames is identified to obtain boundary contour trajectory data, and specifically includes:

[0118] After the system obtains the obstacle structure data and extracts the boundary points of each obstacle region in the continuous images, first, all the image frames are numbered in time sequence to ensure the continuity of the obstacle boundary contour data in time. Then, for each obstacle region, the boundary point coordinate set in each frame of image is extracted, and the boundary point coordinates are usually represented in the form of two-dimensional pixel coordinates. Next, through the boundary point matching algorithm, the boundary points of the same obstacle region in adjacent image frames are matched one by one. The boundary point matching can be based on the KLT optical flow method, which tracks the moving trajectory of each boundary point in the time sequence by calculating the least square difference of the gray scale distribution of the boundary points in adjacent frames to obtain the motion vector of the boundary points in the continuous frames. The matching process of each boundary point needs to ensure the matching accuracy, so the preliminary matching result can be optimized by combining the local features of the boundary points, such as corner information or texture gradient, to eliminate abnormal point pairs with large errors.

[0119] After the boundary point matching is completed, the system generates the trajectory line of each boundary point in the continuous image frames, which records the spatial motion process of the boundary point over time. Specifically, for each boundary point in the obstacle region, a time sequence array is constructed, which contains the coordinate information of the point in each frame of image to form the trajectory data. Through the integration of all the boundary point trajectory data, the complete boundary contour change trajectory of the obstacle region is formed, that is, how the boundary of each obstacle region changes over time in the continuous image frames.

[0120] In order to enhance the robustness of the trajectory data, the system also smoothes the preliminary generated boundary contour trajectory, adopts the Bezier curve fitting, suppresses the high frequency noise existing in the boundary point trajectory, makes the trajectory change more continuous and physically meaningful, and the processing is helpful to eliminate the abnormal movement of the boundary points caused by the slight jitter in image acquisition, light change or image resolution limit. Finally, the boundary contour trajectory data output by the system not only contains the time sequence coordinates of each boundary point of the obstacle region, but also includes the change trend of the boundary overall morphology with time, such as boundary expansion, contraction, fluctuation and other morphological characteristics, and this data will be an important basis for subsequent boundary stability calculation, used to judge the physical properties of the obstacle, especially the distinction between flexible and rigid boundaries.

[0121] In a preferred embodiment of the present application, according to the boundary contour trajectory data, the displacement change amount of the boundary point is calculated, the stability of the obstacle region is obtained, and compared with the preset stability range, the boundary stability evaluation data is obtained, including:

[0122] According to the obstacle structure data, the coordinates of the boundary point are extracted, the displacement change amount of the boundary point is calculated, and the displacement change amount of all images is accumulated to obtain the total amount of cumulative displacement of the obstacle region;

[0123] According to the forward speed of the vehicle, the image frame acquisition interval time, the camera focal length, the angle between the vehicle driving direction and the horizontal axis of the image coordinate system, the pitch angle between the camera and the horizontal line of the ground, the depth value of the boundary point and the actual physical length of the pixel, the expected pixel displacement amount of the boundary point in the continuous frame image is predicted;

[0124] According to the total amount of cumulative displacement and the expected pixel displacement amount, the stability of the obstacle region is determined;

[0125] According to the stability of the obstacle region and the preset stability range, when the stability of the obstacle region is not in the preset stability range, the obstacle region is marked as a non-rigid region.

[0126] In this embodiment of the invention, the coordinates of boundary points are extracted based on obstacle structure data, the displacement change of boundary points is calculated, and the displacement change of all images is accumulated to obtain the total cumulative displacement of the obstacle region. By analyzing the cumulative displacement of boundary points in the time series, the stability characteristics of the obstacle boundary in the dynamic environment can be comprehensively reflected. Based on the vehicle's forward speed, the image frame acquisition interval, the camera focal length, the angle between the vehicle's direction of travel and the horizontal axis of the image coordinate system, the pitch angle between the camera and the ground horizontal line, the depth value of the boundary point, and the actual physical length of the pixel, the expected pixel displacement of the boundary point in consecutive frame images is predicted. By comprehensively utilizing the vehicle motion parameters and the geometric characteristics of the camera system, the theoretical displacement of the boundary point is accurately predicted. The stability of the obstacle region is determined based on the total cumulative displacement and the expected pixel displacement. The stability of the obstacle region is judged based on the stability of the obstacle region and the preset stability range. When the stability of the obstacle region is not within the preset stability range, the obstacle region is marked as a non-rigid region. The quantitative comparison of stability can objectively distinguish between rigid and flexible obstacles, avoiding identification errors caused by misjudging the material properties of obstacles in complex environments.

[0127] The formula for calculating stability is as follows:

[0128] ,

[0129] in, For the stability of the obstacle area, This represents the total cumulative displacement of the boundary points of the obstacle region. An index for images in the original image set. The total number of images in the original image set. For the boundary point at the th The x-coordinate in each image For the boundary point at the th The ordinate in each image, For the boundary point at the th The x-coordinate in each image For the boundary point at the th The ordinate in each image, This represents the positive speed of the vehicle. The acquisition interval between adjacent images in the original image set. For camera focal length, The angle between the vehicle's direction of travel and the horizontal axis of the image coordinate system. The angle between the camera and the horizontal line of the road where the vehicle is traveling. The depth value of the boundary point. This is the actual physical length of the boundary point.

[0130] wherein, The total amount of accumulated displacement of the boundary point of the obstacle region, through the accumulated displacement, the system can quantify the dynamic change degree of the obstacle boundary, and a larger displacement of the boundary point in different frame images usually indicates that the object boundary is in an unstable state, which may be a flexible or force change affected object, on the contrary, a smaller displacement indicates that the object boundary is stable, which has a higher possibility of being a rigid obstacle, providing the system with actual observation data basis, enabling the system to identify the rigid and flexible properties of the object, and assisting the collision model to more accurately construct the obstacle characteristics.

[0131] wherein, The expected pixel displacement amount of the boundary point in the ideal rigid state is predicted according to the vehicle motion state and the camera parameters, which reflects the standard pixel displacement of the rigid object boundary that should be generated with the vehicle movement in the camera view angle during the vehicle movement process. Through the vehicle speed, image acquisition frequency, optical characteristics of the imaging system and other parameters, combined with the obstacle depth, the normal visual displacement of the rigid obstacle in the continuous images due to the vehicle advancing can be simulated. If the actual observed displacement is much larger than the expected displacement, the obstacle has its own boundary change, which tends to be flexible; if it is consistent, it is rigid; the purpose of this design is to construct a displacement judgment baseline caused by environmental changes that is independent of the vehicle motion state, to provide a theoretical comparison standard for the boundary stability of the system, and to enhance the scientificity and accuracy of the discrimination.

[0132] wherein, the numerator part of the stability calculation formula is the difference between the actual observation result and the theoretical expected result, which reflects the deviation of the dynamic behavior of the obstacle boundary in the real environment, and the denominator part is the theoretical expectation, which is derived from the vehicle motion parameters and the physical imaging model, and the behavior of the rigid boundary in the ideal state should be. Through the ratio of the two, the system can determine the stability of the obstacle boundary, so as to accurately distinguish between flexible and rigid obstacles, and optimize the adaptability of three-dimensional collision body modeling.

[0133] wherein, The actual physical length of the boundary point is the physical length parameter obtained by dividing the obstacle boundary into several continuous sections and uniformly calculating each section, which is used for conversion between image coordinates and physical space. First, according to the boundary contour of the obstacle in the image, the whole boundary is divided into several sections according to the spatial continuity or depth change trend, for example, one section per 100 pixels, or according to the continuous part with a depth change less than a certain threshold. After the division, for each section, the depth value of a representative boundary point in the section is sampled to obtain the depth information of these points, and then the average depth value of the section is calculated Based on the average depth of the section, combined with the relevant parameters of the camera (such as focal length , sensor width image width resolution , unify the actual physical length of the segment ), and the specific conversion formula is , is the actual width of the camera sensor, in millimeters, is the focal length, in millimeters, is the number of horizontal pixels of the image, is the unified physical length conversion value of all boundary points in the segment, and when applied, the pixel displacement amount of all boundary points in the segment will be uniformly converted into physical displacement by , and the calculation formula is: physical displacement = pixel displacement x In this way, the calculation of depth and physical length for each boundary point is avoided, which significantly improves the system calculation efficiency, while maintaining the conversion accuracy under the premise of small boundary depth change, and adapts to the balance between real-time and accuracy in dynamic environment.

[0134] In a preferred embodiment of the present application, according to the flexible region data, the dynamic morphological change process of the flexible region is identified, the voxel distribution sequence of the flexible region changing over time is determined, and the flexible structure data is obtained, including:

[0135] According to the flexible region data, the position change trajectory of the flexible region in the continuous image is identified to obtain a trajectory data set;

[0136] According to the trajectory data set, the displacement change amount and the rotation change amount of the flexible region in the time sequence are extracted to obtain pose change data;

[0137] According to the pose change data, the shape contour change trend of the flexible region between the continuous images is identified to obtain morphological change data;

[0138] According to the morphological change data, the flexible region at each time is spatially discretized to form a single-frame voxel set;

[0139] The single-frame voxel set is arranged in time sequence to determine the voxel distribution sequence of the flexible region changing over time, and the flexible structure data is obtained.

[0140] In this embodiment of the invention, flexible region data is used to identify the positional change trajectory of the flexible region in continuous images, obtaining a trajectory dataset. This effectively tracks the dynamic positional changes of flexible obstacles in continuous images, ensuring the system has continuous and complete control over the spatial motion characteristics of the flexible region. Based on the trajectory dataset, the displacement and rotational changes of the flexible region in the time series are extracted to obtain pose change data, accurately quantifying the posture changes of the flexible obstacle during spatial motion. Based on the pose change data, the trend of the shape contour change of the flexible region between continuous images is identified, obtaining morphological change data. This not only identifies the flexible region... The system can capture subtle changes in the spatial location of a region, enhancing the completeness of modeling the dynamic behavior of flexible objects. Based on the morphological change data, the flexible region at each moment is spatially discretized to form a single-frame voxel set, realizing the discretized expression of the spatial morphology of flexible obstacles. This allows complex continuous morphologies to be quantified through standardized voxel units. By arranging the single-frame voxel set in chronological order, the voxel distribution sequence of the flexible region over time is determined, resulting in flexible structure data. This allows the dynamic changes of flexible obstacles to be expressed systematically and continuously, giving the collision detection system the ability to control time-series features.

[0141] Specifically, based on the flexible region data, the trajectory of positional changes of the flexible region is identified in continuous images to obtain a trajectory dataset, which includes:

[0142] To identify the positional trajectory of flexible regions in continuous images, the system first needs to determine the boundary range of the flexible region in each frame based on the flexible region data. The system extracts a set of boundary points from the identified flexible region data and determines the bounding box of the flexible region based on the spatial distribution of these points. By calculating the centroid of the boundary points of the flexible region in each frame, the coordinates of the center point of the flexible region in that frame are obtained. The centroid calculation uses a weighted average method, where the weights are adjusted according to the boundary point density to enhance adaptability to irregular boundary changes. Subsequently, the system sorts the calculated center point coordinates of each image frame according to time sequence, forming the spatial position trajectory of the flexible region changing over time. To improve the continuity and accuracy of the trajectory data, the system introduces a Kalman filter algorithm to smooth the trajectory point data, eliminating outliers caused by image noise or short-term occlusion, ensuring that the trajectory dataset truly reflects the continuous motion characteristics of the flexible region. Finally, the trajectory dataset contains the sequence of center positions of the flexible region in continuous image frames, which is used for subsequent pose change and morphological analysis.

[0143] Specifically, based on pose change data, the shape contour change trend of the flexible region between consecutive images is identified to obtain morphological change data, which includes:

[0144] The trend of the shape contour change of the flexible region is identified by first extracting the boundary contour of the flexible region in each frame of image according to the position information in the trajectory data set, the system extracts the boundary points of the flexible region with high precision, and the contour fitting algorithm is used to smooth the boundary shape, so that the interference caused by the image acquisition accuracy or the change of light is eliminated, for the boundary of the flexible region in the continuous image frames, the system calculates the shape index of the boundary points in the space, including the boundary length, the boundary curvature, the boundary convex hull area and other parameters, the shape characteristics of the flexible region at different time points are fully described, then, the system compares the boundary shape parameters between the continuous frames, calculates the change rate of the boundary contour, extracts the displacement vector field of the boundary points, and further analyzes the deformation trend of the boundary as a whole. Through the principal component analysis technology, the main direction and change amplitude of the boundary deformation are extracted, and the system can identify the stretching, bending or twisting of the flexible region in the time sequence. Finally, the morphological change data record the contour change trend of the flexible region between the continuous images in the form of time sequence, which provides accurate dynamic morphological basis for subsequent spatial discretization.

[0145] According to the morphological change data, the flexible region at each time is discretized in space to form a single-frame voxel set, specifically including:

[0146] The dynamic morphology of the flexible region is converted into a spatial voxel set. First, a three-dimensional boundary of the flexible region is constructed according to the morphological change data in each frame of image. The system reconstructs the specific morphology of the flexible region in three-dimensional space using the boundary contour and depth mapping data, and encloses it in a smallest circumscribed three-dimensional boundary box. Then, the system divides the three-dimensional boundary box into equally spaced voxel units according to the preset spatial resolution. For each voxel unit, the system determines whether the voxel is activated according to whether the center point falls within the boundary contour of the flexible region. If it falls within, the voxel is marked as "occupied state", otherwise as "idle state". During this process, the system also corrects the height information of each voxel unit in combination with the depth value to ensure accurate restoration of the three-dimensional structure of the flexible region in the spatial discretization process. The collection of all voxel units marked as occupied state constitutes a single-frame voxel set under the frame of image. To further optimize the voxel representation effect, the system filters out isolated voxel points and reduces redundant voxels through voxel fusion technology to improve the compactness and computational efficiency of the model. Finally, the system forms complete voxel data corresponding to each frame of image to construct a three-dimensional dynamic model of the flexible region and provide basic data support for collision detection.

[0147] In a preferred embodiment of the present application, the displacement change and the rotation change of the flexible region in the time sequence are extracted according to the trajectory data set to obtain the pose change data, including:

[0148] According to the trajectory data set, the spatial positions of the center points of the flexible region in continuous images are extracted to obtain trajectory point data;

[0149] According to the trajectory point data, the three-dimensional spatial displacement of the center points of the flexible region between continuous images is calculated to obtain displacement change data;

[0150] According to the shape change of the boundary contour of the flexible region in continuous images, a direction vector is extracted to obtain direction vector data;

[0151] According to the direction vector data, the change of the rotation angle of the flexible region between continuous images is calculated to obtain rotation change data;

[0152] The displacement change data and the rotation change data are combined to determine the overall motion state of the flexible region to obtain pose change data.

[0153] In the embodiment of the present application, according to the trajectory data set, the spatial positions of the center points of the flexible region in continuous images are extracted to obtain trajectory point data, realizing the preliminary quantification of the position change of the flexible region in continuous time; according to the trajectory point data, the three-dimensional spatial displacement of the center points of the flexible region between continuous images is calculated to obtain displacement change data, realizing the three-dimensional restoration of the motion path of the flexible region, and the system is no longer limited to two-dimensional image plane motion analysis, but can accurately capture the three-dimensional movement of the flexible region in the actual space; according to the shape change of the boundary contour of the flexible region in continuous images, a direction vector is extracted to obtain direction vector data, so that the system can identify the rotation trend of the flexible region in space, which is an important dimension of dynamic analysis of flexible objects; according to the direction vector data, the change of the rotation angle of the flexible region between continuous images is calculated to obtain rotation change data, so that the system can accurately depict the change of the angle and posture of the flexible region in the dynamic process; the displacement change data and the rotation change data are combined to determine the overall motion state of the flexible region to obtain pose change data, so that the system has a comprehensive understanding of the comprehensive dynamic behavior of the flexible region, not only can master its motion trajectory, but also can understand the influence of its posture change on space occupation.

[0154] According to the shape change of the boundary contour of the flexible region in continuous images, a direction vector is extracted to obtain direction vector data, specifically including:

[0155] After the boundary contour of the flexible region is obtained, the system firstly performs fitting processing on the flexible region boundary in each frame of image, and a commonly used method is principal component analysis, a covariance matrix of a boundary point set is calculated, and an eigenvector and an eigenvalue are solved, wherein the eigenvector corresponding to the maximum eigenvalue is a principal direction vector of the flexible region, and then three-dimensional coordinate representation of the principal direction vector is recorded as the direction vector data of the current image frame. For continuous image frames, the above process is repeated, the principal direction vector of the flexible region in each frame of image is extracted, is arranged according to the time sequence of the image frames, and the direction vector data set of the flexible region in the whole image sequence is formed.

[0156] Wherein, according to the direction vector data, the rotation angle change of the flexible region between continuous images is calculated to obtain the rotation change data, and specifically includes:

[0157] Supposing that the direction vector in the previous frame of image is , and the direction vector in the next frame of image is , then the rotation angle of the flexible region is , which can be calculated by the following formula, In order to improve the stability of the rotation angle calculation, a weighted smoothing filter can be introduced in the included angle operation to adjust the angle change between continuous frames, and the angle jitter caused by image noise is avoided. In order to completely describe the rotation characteristics of the flexible region, the axis and the rotation direction of the rotation need to be determined, and the complete rotation change data can be obtained by projecting the direction vector onto each axis of the three-dimensional coordinate system and analyzing the distribution of the rotation in each dimension.

[0158] Wherein, the displacement change data and the rotation change data are combined to determine the overall motion state of the flexible region to obtain the pose change data, and specifically includes:

[0159] Firstly, the three-dimensional space position of the flexible region in each frame of image is combined with the corresponding direction vector to form the pose description unit of the frame of image. Then, the pose description units in adjacent frames of image are paired, the complete motion state of the flexible region between the two time points is solved through a rigid body transformation model, including a three-dimensional displacement vector and a plane or space rotation matrix. In this process, the homogeneous coordinate transformation method can be used to uniformly represent the translation and rotation of the flexible region as a 4x4 transformation matrix, so as to facilitate the system to iteratively update the motion state of the flexible region in the continuous time period. Through accumulation of the pose transformation of each frame of image, the complete pose change data of the flexible region in the whole observation time period is finally obtained.

[0160] In a preferred embodiment of the present application, according to the real-time spatial position and motion parameters of the vehicle, spatial overlap analysis is performed on the three-dimensional collision body model to determine whether there is a collision event of the target vehicle on the driving path, including:

[0161] According to the vehicle positioning device, the spatial position of the vehicle in the driving path is collected in real time to obtain vehicle spatial position data;

[0162] According to the vehicle sensor, the speed, acceleration and heading angle parameters of the vehicle are extracted to obtain vehicle motion parameter data;

[0163] According to the vehicle spatial position data and the vehicle motion parameter data, it is predicted whether the vehicle has spatial overlap with the three-dimensional collision body model to obtain overlap data;

[0164] According to the overlap data, it is judged whether there is a collision risk of the vehicle on the driving path, and when the result is yes, a collision event is generated.

[0165] In the embodiment of the present application, according to the vehicle positioning device, the spatial position of the vehicle in the driving path is collected in real time to obtain vehicle spatial position data, which can accurately grasp the real-time spatial position of the vehicle, ensure that the subsequent collision detection is based on accurate position basis, and avoid collision judgment errors caused by position errors; according to the vehicle sensor, the speed, acceleration and heading angle parameters of the vehicle are extracted to obtain vehicle motion parameter data, so that the system can completely grasp the current motion state of the vehicle, and provide accurate dynamic information basis for trajectory prediction and spatial overlap analysis, and improve the prediction accuracy of the future driving path; according to the vehicle spatial position data and the vehicle motion parameter data, it is predicted whether the vehicle has spatial overlap with the three-dimensional collision body model to obtain overlap data, through the efficient and accurate spatial overlap detection method, the possible collision risk of the vehicle is quickly judged; according to the overlap data, it is judged whether there is a collision risk of the vehicle on the driving path, and when the result is yes, a collision event is generated, through setting clear collision judgment standard, realizing instant identification and quantitative expression of potential risk, providing direct basis for subsequent obstacle avoidance path planning and motion parameter adjustment of the system.

[0166] In a preferred embodiment of the present application, according to the vehicle spatial position data and the vehicle motion parameter data, it is predicted whether the vehicle has spatial overlap with the three-dimensional collision body model to obtain overlap data, comprising:

[0167] According to the vehicle spatial position data and the vehicle motion parameter data, the predicted trajectory point of the vehicle on the driving path is calculated to obtain vehicle predicted trajectory data;

[0168] According to the vehicle size, the vehicle contour is extended equidistantly to generate extended vehicle contour data, and the extended vehicle contour data is arranged continuously along the vehicle predicted trajectory data to build an extended trajectory area of the vehicle on the driving path to obtain extended trajectory area data;

[0169] The extended trajectory region data and the three-dimensional collision body model are compared in spatial position, whether the extended trajectory region overlaps with the three-dimensional collision body model is detected, and overlap data is obtained.

[0170] In the embodiment of the present application, the predicted trajectory point of the vehicle on the driving path is calculated according to the vehicle spatial position data and the vehicle motion parameter data, the vehicle predicted trajectory data is obtained, the high-precision prediction of the future motion path of the vehicle is realized, and the basic data support is provided for the subsequent collision detection; the extended trajectory region data is obtained by extending the vehicle contour outward at equal intervals according to the vehicle size, arranging the extended vehicle contour data continuously along the vehicle predicted trajectory data, and constructing the extended trajectory region of the vehicle on the driving path, which effectively enhances the rigor and fault tolerance of the collision detection; the spatial position comparison is performed between the extended trajectory region data and the three-dimensional collision body model, whether the extended trajectory region overlaps with the three-dimensional collision body model is detected, and the overlap data is obtained, so that the dynamic overlap analysis between the predicted path of the vehicle and the environmental obstacles is realized.

[0171] In the embodiment of the present application, the predicted trajectory point of the vehicle on the driving path is calculated according to the vehicle spatial position data and the vehicle motion parameter data, the vehicle predicted trajectory data is obtained, the high-precision prediction of the future motion path of the vehicle is realized, and the basic data support is provided for the subsequent collision detection; the extended trajectory region data is obtained by extending the vehicle contour outward at equal intervals according to the vehicle size, arranging the extended vehicle contour data continuously along the vehicle predicted trajectory data, and constructing the extended trajectory region of the vehicle on the driving path, which effectively enhances the rigor and fault tolerance of the collision detection; the spatial position comparison is performed between the extended trajectory region data and the three-dimensional collision body model, whether the extended trajectory region overlaps with the three-dimensional collision body model is detected, and the overlap data is obtained, so that the dynamic overlap analysis between the predicted path of the vehicle and the environmental obstacles is realized.

[0172] Firstly, the operation of extending the vehicle contour outward at equal intervals according to the vehicle size to generate the extended vehicle contour data needs to obtain the geometric parameters of the vehicle first, which include the length L, the width W and the height H of the vehicle, and are usually provided by the vehicle manufacturing data. Based on the two-dimensional planar contour of the vehicle, the length and the width of the vehicle are given priority, and the influence of the height on the ground obstacle collision is ignored. Then, according to the lateral deviation, the path prediction error and the uncertainty of the dynamic environment that the vehicle may generate in the driving process, the vehicle standard contour is extended by a preset range of safety. The extension distance may be dynamically adjusted according to the current speed v of the vehicle and the complexity of the driving environment, for example, when driving at low speed , it can be set to 0.2 meters, and when driving at high speed, it can be extended to more than 0.5 meters, so as to ensure sufficient collision detection redundancy. Through the dilation operation in mathematical morphology or the buffer algorithm of geometric graphics, the standard rectangular contour of the vehicle is uniformly extended outward to obtain the extended vehicle contour data. This data is in the form of a polygon vertex coordinate set, which is convenient for subsequent spatial overlap analysis.

[0173] Subsequently, the generated extended vehicle contour data is arranged continuously along the vehicle prediction trajectory. The vehicle prediction trajectory is composed of spatial positions at a plurality of discrete time points, each time point corresponding to a spatial coordinate of the vehicle. At each time, the extended vehicle contour data is rotationally transformed according to the heading angle of the vehicle, so as to align the actual moving direction of the vehicle, and the spatial position of the extended contour at the time is positioned with the spatial coordinate of the vehicle as the center. The process is repeated to sequentially superimpose the extended contours at all times, forming a continuous spatial occupation area of the vehicle in the prediction time range, i.e., an extended trajectory area, which is essentially a spatial pipeline model of the vehicle advancing along the time dimension, covering the entire spatial range in which the vehicle can move.

[0174] In the space overlap detection process, for each extended contour in the extended trajectory area, it is judged whether there is a spatial overlap between the extended contour and any voxel of the three-dimensional collision body model. The specific implementation can be completed by using bounding box detection calculation. For the detection of a two-dimensional extended contour and a three-dimensional voxel, the three-dimensional voxel can be projected onto a two-dimensional plane for preliminary plane overlap screening, and after screening, further three-dimensional overlap verification can be performed. Whenever an overlap event is detected, the time, the extended contour index, the voxel number, and the overlap area or volume parameters of the overlap are recorded, and finally an overlap data set is formed. The overlap data serves as direct evidence of collision risk, is used to determine whether there is an actual spatial conflict of the vehicle in the driving path, and provides a basis for subsequent path adjustment or avoidance strategy.

[0175] The three-dimensional collision body model is a spatial representation of obstacles in the driving path, which is usually composed of a set of voxels, recording the occupation position of each obstacle in three-dimensional space. First, the spatial coordinates of the extended trajectory area and the three-dimensional collision body model are unified to the same coordinate system to ensure consistency of the spatial positions. Then, an efficient spatial index structure such as an R-tree is used to index the three-dimensional collision body model to accelerate subsequent overlap detection.

[0176] In the space overlap detection process, for each extended contour in the extended trajectory area, it is judged whether there is a spatial overlap between the extended contour and any voxel of the three-dimensional collision body model. The specific implementation can be completed by using bounding box detection calculation. For the detection of a two-dimensional extended contour and a three-dimensional voxel, the three-dimensional voxel can be projected onto a two-dimensional plane for preliminary plane overlap screening, and after screening, further three-dimensional overlap verification can be performed. Whenever an overlap event is detected, the time, the extended contour index, the voxel number, and the overlap area or volume parameters of the overlap are recorded, and finally an overlap data set is formed. The overlap data serves as direct evidence of collision risk, is used to determine whether there is an actual spatial conflict of the vehicle in the driving path, and provides a basis for subsequent path adjustment or avoidance strategy.

[0177] In a preferred embodiment of the present application, when there is a collision event, collision region information is obtained, and the driving path and motion parameters of the vehicle are adjusted according to the collision region information, including:

[0178] According to the collision event, the collision region number and the corresponding three-dimensional spatial position where the collision occurs are extracted to obtain the collision region information;

[0179] According to the collision region information, the regions in the neighborhood of the vehicle where the collision exists are removed to obtain passable region data;

[0180] According to the passable region data, a safe path of the vehicle is planned, and safety detection is performed to ensure that the safe path does not overlap with the three-dimensional collision body model in space.

[0181] According to the safe path, the speed, acceleration and heading angle parameters of the vehicle are adjusted to ensure that the vehicle travels according to the safe path.

[0182] In the embodiment of the present application, according to the collision event, the collision region number and its corresponding three-dimensional space position where the collision occurs are extracted to obtain collision region information, which can accurately locate the specific obstacle region where the collision risk occurs, and provide data support for subsequent planning of a safe path. According to the collision region information, the regions in the vehicle neighborhood range where the collision exists are removed to obtain passable region data, which ensures that the vehicle obtains a real and feasible safe passing region in a local environment. According to the passable region data, a safe path of the vehicle is planned, and safety detection is performed to ensure that the safe path does not overlap with the three-dimensional collision body model in space, which guarantees that the safe path not only has the shortest path characteristic, but also has absolute safety in the whole process, avoiding global collision hidden dangers caused by local optimal path. According to the safe path, the speed, acceleration and heading angle parameters of the vehicle are adjusted to ensure that the vehicle travels according to the safe path, which guarantees accurate execution of the vehicle on the safe path, reduces path tracking errors, and improves the effectiveness of the safe operation.

[0183] According to the collision region information, the regions in the vehicle neighborhood range where the collision exists are removed to obtain passable region data, and the passable region data specifically includes:

[0184] Firstly, the system filters and processes the neighborhood space where the vehicle is currently located according to the extracted collision region information. The vehicle neighborhood range is defined by the current position of the vehicle as the center, a preset forward detection distance, a lateral expansion width and a longitudinal safety interval, to form a neighborhood space model with specific three-dimensional boundaries. The model can be expressed as a spatial set in a local three-dimensional coordinate system, to represent all the space regions where the vehicle can travel in the next period of time. The system performs a spatial Boolean operation on the neighborhood space model and the known three-dimensional collision body model, especially an overlap analysis on the three-dimensional coordinate set corresponding to the collision region that currently triggers the collision. For the overlapped space regions, the system dynamically marks them as “no-passing regions” according to the relationship between their spatial positions and the motion trend of the vehicle, and these regions are directly excluded from the vehicle neighborhood space model. In specific implementation, the system performs voxel-level spatial discretization operation on the neighborhood space model, and excludes any voxel unit that overlaps with the collision region model as a potential danger zone. The remaining voxel units that are not marked as no-passing are combined and calculated as continuous and well-connected “passable regions”, which are arranged by the system as passable region data, and a basic navigation space framework relied on by subsequent path planning is constructed based on the data.

[0185] wherein, according to the passable region data, a safety path of the vehicle is planned and safety detection is performed to ensure that the safety path does not spatially overlap with the three-dimensional collision body model, and the safety path planning specifically comprises:

[0186] After obtaining the passable region data, the system first starts a path planning module according to the spatial position, motion state of the current vehicle and the geometric shape of the passable region, and generates a safety path from the current position of the vehicle, which avoids the collision region and finally returns to the normal driving trajectory, according to the preset safety priority strategy. The path planning process adopts a path optimization algorithm based on graph search, and comprehensively considers the length, smoothness, dynamic feasibility and other indicators of the path to perform weight distribution and search on the path nodes in the passable region, so as to obtain the optimal safety path. After the path is initially generated, the system also performs refinement processing on the path, calculates the spatial coordinates of each discrete point on the path and the corresponding vehicle attitude information, including the speed, acceleration and heading angle parameters of the vehicle at each path point. In order to ensure that the safety path has absolute safety in actual execution, the system performs safety detection on the entire path, which is to expand the generated safety path in the time-space dimension and re-analyze the spatial overlap with the complete three-dimensional collision body model. If any node or its expanded region on the path overlaps with the three-dimensional collision body, the path is considered unsafe, and the system will trigger the re-planning process, which is iterated until a completely collision-free safe path is generated. The safety detection also includes verification of the dynamic characteristics of the vehicle to ensure that the acceleration and turning radius required by the path are within the mechanical limits of the vehicle, so as to avoid path failure due to dynamic infeasibility. Finally, the path that passes the safety detection is confirmed as a feasible path for the vehicle to execute the obstacle avoidance operation.

[0187] The above is the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should also be considered within the scope of protection of the present application.

Claims

1. A method for collision detection of a trolley based on a three-dimensional collision body, characterized in that, The method comprises: Collecting images in front of the vehicle driving path, and calculating the depth value of each pixel point in each image according to the images, constructing the corresponding space mapping relationship, and obtaining the depth mapping data; According to the depth mapping data, the obstacle region in the image is extracted to obtain the obstacle structure data; According to the obstacle structure data, the stability of the edge of the obstacle region is analyzed, the flexible region in the image is identified, and the flexible region data is obtained; According to the flexible region data, the dynamic morphological change process of the flexible region is identified, the voxel distribution sequence of the flexible region with time is determined, and the flexible structure data is obtained; The flexible structure data and the obstacle structure data are merged to construct a three-dimensional collision body model; According to the real-time spatial position and motion parameters of the vehicle, the spatial overlap analysis of the three-dimensional collision body model is carried out, and it is judged whether there is a collision event on the driving path of the target vehicle; When there is a collision event, the collision region information is obtained, and the driving path and motion parameters of the vehicle are adjusted according to the collision region information.

2. The trolley collision detection method based on three-dimensional collision bodies according to claim 1, characterized in that, Collecting images in front of the vehicle driving path, and calculating the depth value of each pixel point in each image according to the images, constructing the corresponding space mapping relationship, and obtaining the depth mapping data, comprising: Collecting continuous images in front of the vehicle driving path through the image acquisition device, and arranging them in time sequence to obtain the original image set; According to the original image set, the feature points containing local texture, corner and boundary features are extracted to obtain the feature point set; According to the spatial distribution of the feature point set in the continuous images, the corresponding relationship of each feature point in different images is determined, and the pixel point matching data is obtained; According to the pixel point matching data, the pixel position offset of each pixel point in different images is calculated, and the depth value of each pixel point is calculated according to the pixel position offset, and the depth value set is obtained; According to the original image set and the depth value set, the mapping relationship of each pixel point in three-dimensional space coordinates is determined, and the depth mapping data is obtained.

3. The trolley collision detection method based on three-dimensional collision bodies according to claim 2, characterized in that, According to the pixel point matching data, the pixel position offset of each pixel point in different images is calculated, and the depth value of each pixel point is calculated according to the pixel position offset, and the depth value set is obtained, comprising: According to the pixel point matching data, the coordinates of the pixel points in each image in the original image set are extracted, and the spatial displacement of the pixel points in the continuous images is calculated; According to the pixel point matching data, the feature region of the pixel point is determined, and the feature region density correction value is calculated through the number of feature points in the feature region and the area of the feature region; the included angle between the motion directions of adjacent pixel points is calculated to obtain the motion trend angle; The spatial displacement of the pixel point, the feature region density correction value and the motion trend angle are accumulated to obtain the total amount of motion change of the pixel point, which is converted into distance unit, and the depth value is calculated combined with the displacement and focal length of the camera.

4. The trolley collision detection method based on three-dimensional collision bodies according to claim 3, characterized in that, According to the obstacle structure data, the stability of the edge of the obstacle region is analyzed, the flexible region in the image is identified, and the flexible region data is obtained, comprising: According to the obstacle structure data, the boundary points of each obstacle region in the continuous images are extracted to obtain the boundary contour data; According to the boundary contour data, the boundary contour change track of each obstacle region in the continuous image frames is identified to obtain the boundary contour track data; According to the boundary contour trajectory data, the displacement change amount of the boundary point is calculated to obtain the stability of the obstacle region, and the stability is compared with a preset stability range to obtain boundary stability evaluation data; According to the boundary stability evaluation data, the obstacle region with insufficient stability is determined to obtain flexible region data.

5. The trolley collision detection method based on three-dimensional collision bodies according to claim 4, characterized in that, According to the boundary contour trajectory data, the displacement change amount of the boundary point is calculated to obtain the stability of the obstacle region, and the stability is compared with a preset stability range to obtain boundary stability evaluation data, including: According to the obstacle structure data, the coordinates of the boundary point are extracted, the displacement change amount of the boundary point is calculated, and the displacement change amounts of all images are accumulated to obtain the total accumulated displacement amount of the obstacle region; According to the forward speed of the vehicle, the image frame acquisition interval time, the camera focal length, the angle between the vehicle driving direction and the horizontal axis of the image coordinate system, the pitch angle between the camera and the horizontal line of the ground, the depth value of the boundary point and the actual physical length of the pixel, the expected pixel displacement amount of the boundary point in the continuous frame image is predicted; According to the total accumulated displacement amount and the expected pixel displacement amount, the stability of the obstacle region is determined; According to the stability of the obstacle region and the preset stability range, when the stability of the obstacle region is not within the preset stability range, the obstacle region is marked as a non-rigid region.

6. The trolley collision detection method based on three-dimensional collision bodies according to claim 5, characterized in that, According to the flexible region data, the dynamic morphological change process of the flexible region is identified, the voxel distribution sequence of the flexible region with time change is determined, and the flexible structure data is obtained, including: According to the flexible region data, the position change trajectory of the flexible region in the continuous image is identified to obtain a trajectory data set; According to the trajectory data set, the displacement change amount and the rotation change amount of the flexible region in the time sequence are extracted to obtain pose change data; According to the pose change data, the shape contour change trend of the flexible region between continuous images is identified to obtain morphological change data; According to the morphological change data, the flexible region at each time is spatially discretized to form a single-frame voxel set; The single-frame voxel set is arranged in time sequence to determine the voxel distribution sequence of the flexible region with time change, and the flexible structure data is obtained.

7. The trolley collision detection method based on three-dimensional collision bodies according to claim 6, characterized in that, According to the trajectory data set, the displacement change amount and the rotation change amount of the flexible region in the time sequence are extracted to obtain pose change data, including: According to the trajectory data set, the spatial position of the center point of the flexible region in the continuous image is extracted to obtain trajectory point data; According to the trajectory point data, the three-dimensional spatial displacement amount of the center point of the flexible region between continuous images is calculated to obtain displacement change amount data; According to the boundary contour shape change of the flexible region in the continuous image, the direction vector is extracted to obtain direction vector data; According to the direction vector data, the rotation angle change of the flexible region between continuous images is calculated to obtain rotation change amount data; The displacement change amount data and the rotation change amount data are combined to determine the overall motion state of the flexible region to obtain the pose change data.

8. The trolley collision detection method based on three-dimensional collision bodies according to claim 7, characterized in that, According to the real-time spatial position and motion parameters of the vehicle, spatial overlap analysis is performed on the three-dimensional collision body model to determine whether there is a collision event on the driving path of the target vehicle, including: According to the vehicle positioning device, the spatial position of the vehicle in the driving path is collected in real time to obtain vehicle spatial position data; According to the vehicle sensor, the speed, acceleration and heading angle parameters of the vehicle are extracted to obtain vehicle motion parameter data; According to the vehicle spatial position data and the vehicle motion parameter data, it is predicted whether the vehicle has spatial overlap with the three-dimensional collision body model to obtain overlap data; According to the overlap data, it is judged whether there is a collision risk for the vehicle on the driving path, and when the result is yes, a collision event is generated.

9. The trolley collision detection method based on three-dimensional collision bodies according to claim 8, characterized in that, According to the vehicle spatial position data and the vehicle motion parameter data, it is predicted whether the vehicle has spatial overlap with the three-dimensional collision body model to obtain overlap data, including: According to the vehicle spatial position data and the vehicle motion parameter data, the predicted trajectory point of the vehicle on the driving path is calculated to obtain vehicle predicted trajectory data; According to the vehicle size, the vehicle contour is extended outward at equal intervals to generate extended vehicle contour data, which is arranged continuously along the vehicle predicted trajectory data to build an extended trajectory area of the vehicle on the driving path to obtain extended trajectory area data; The extended trajectory area data and the three-dimensional collision body model are compared in spatial position to detect whether the extended trajectory area overlaps with the three-dimensional collision body model to obtain overlap data.

10. The trolley collision detection method based on three-dimensional collision bodies according to claim 9, characterized in that, When there is a collision event, the collision area information is obtained, and the driving path and motion parameters of the vehicle are adjusted according to it, including: According to the collision event, the collision region number and its corresponding three-dimensional spatial position where the collision occurs are extracted to obtain collision region information; According to the collision region information, the areas where collisions exist in the vehicle neighborhood range are removed to obtain passable area data; According to the passable area data, the safety path of the vehicle is planned and safety detection is performed to ensure that it does not have spatial overlap with the three-dimensional collision body model; According to the safety path, the speed, acceleration and heading angle parameters of the vehicle are adjusted to ensure that the vehicle drives according to the safety path.

Images