A method, apparatus, equipment, medium, and product for determining parking spaces.

Abstract

This invention discloses a parking space determination method, apparatus, device, medium, and product, comprising: acquiring an environmental image, trajectory information, and ultrasonic information of a vehicle; performing semantic segmentation processing on the environmental image to obtain a to-be-processed image containing semantic information; determining obstacle points based on the semantic information and the trajectory information to obtain an obstacle point set; and performing parking space fitting calculation based on the obstacle point set and the ultrasonic information to obtain a target parking space. These technical features improve the accuracy of parking space construction, thereby ensuring the smooth operation of subsequent automatic parking and enhancing the user experience.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a parking space determination method, device, equipment, medium, and product. Background Technology

[0002] With the development of autonomous driving technology, automated parking systems have become an important research direction in the field of intelligent vehicles. Accurate detection of parking spaces is one of the prerequisites for achieving automated parking in automated parking systems.

[0003] Currently, for unmarked parking spaces, parking space search mainly relies on ultrasonic detection. However, the accuracy of corner points calculated by ultrasonic detection is affected by vehicle speed and the relative posture of the vehicle body and obstacles, which may lead to insufficient accuracy of the constructed ultrasonic parking space, resulting in poor parking performance or even parking failure. Summary of the Invention

[0004] This invention provides a parking space determination method, device, equipment, medium, and product, which improves the accuracy of parking space construction, thereby ensuring the smooth operation of subsequent automatic parking and improving the user experience.

[0005] According to a first aspect of the present invention, a parking space determination method is provided, comprising:

[0006] Acquire environmental images, trajectory information, and ultrasonic information of the vehicle;

[0007] The environmental image is subjected to semantic segmentation to obtain an image to be processed containing semantic information;

[0008] Obstacle points are determined based on the semantic information and the flight path information to obtain a set of obstacle points;

[0009] The target parking space is obtained by performing parking space fitting calculation based on the set of obstacle points and the ultrasonic information.

[0010] According to a second aspect of the present invention, a parking space determination device is provided, comprising:

[0011] The information acquisition module is used to acquire environmental images, trajectory information, and ultrasonic information of the vehicle.

[0012] The image processing module is used to perform semantic segmentation on the environmental image to obtain an image to be processed containing semantic information;

[0013] An obstacle point determination module is used to determine obstacle points based on the semantic information and the trajectory information to obtain an obstacle point set.

[0014] The parking space determination module is used to perform parking space fitting calculations based on the set of obstacle points and the ultrasonic information to obtain the target parking space.

[0015] According to a third aspect of the present invention, a vehicle is provided, comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a parking space determination method according to any embodiment of the present invention.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement a parking space determination method according to any embodiment of the present invention.

[0020] According to a fifth aspect of the present invention, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements a parking space determination method according to any embodiment of the present invention.

[0021] The technical solution of this invention involves acquiring environmental images, trajectory information, and ultrasonic information of a vehicle; performing semantic segmentation on the environmental images to obtain a processing image containing semantic information; determining obstacle points based on the semantic information and the trajectory information to obtain an obstacle point set; and performing parking space fitting calculation based on the obstacle point set and the ultrasonic information to obtain a target parking space. This technical feature combines visual images and ultrasonic information, determining obstacle points through the processing image, tracking obstacle points through trajectory information to obtain an obstacle point set, and correcting the parking space corner points corresponding to the ultrasonic information based on the obtained obstacle point set. This achieves more accurate parking space corner point determination and fits a more accurate target parking space, solving the problem of low accuracy in constructing parking spaces solely based on ultrasonic waves. It improves the accuracy of parking space fitting and construction, thereby ensuring the smooth operation of subsequent automatic parking and enhancing the user experience.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a parking space determination method provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is an example image of the image to be processed in a parking space determination method according to Embodiment 1 of the present invention;

[0026] Figure 3 This is a flowchart of a parking space determination method provided according to Embodiment 2 of the present invention;

[0027] Figure 4 This is an example diagram of the current target obstacle point in a parking space determination method according to Embodiment 2 of the present invention;

[0028] Figure 5 This is an example diagram of obstacle tracking in a parking space determination method according to Embodiment 2 of the present invention;

[0029] Figure 6 This is a schematic diagram of a parking space determination device according to Embodiment 3 of the present invention;

[0030] Figure 7 This is a structural schematic diagram of a vehicle provided according to Embodiment 4 of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] Existing ultrasonic parking space detection methods are affected by vehicle speed and driving angle, resulting in inaccurate detection of parking spaces. Excessive deviation can cause parking to be interrupted by obstacles, leading to parking failure. Alternatively, the ultrasonic output of the parking space may not be accurate enough, causing the parking space to be out of center, affecting users getting out of their cars or users in adjacent vehicles, seriously impacting the user experience. Therefore, this invention provides a parking space determination method based on the combination of visual images and ultrasonic information to improve the accuracy of parking space determination.

[0035] Figure 1 This is a flowchart of a parking space determination method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where parking spaces are constructed based on visual images and ultrasonic information. The method can be executed by a parking space determination device, which can be implemented in hardware and / or software and can be integrated into a vehicle. Figure 1 As shown, the method includes:

[0036] S101. Acquire environmental images, trajectory information, and ultrasonic information of the vehicle.

[0037] In this embodiment, environmental images can be understood as images encompassing the vehicle's surrounding environment, such as a bird's-eye view of the vehicle, acquired and synthesized by cameras on the vehicle body. Dead Reckoning (DR) information can be understood as the trajectory information of the vehicle during its movement, i.e., the vehicle's real-time positioning information, which may include track coordinates. Ultrasonic information can be understood as information obtained after the vehicle emits ultrasonic waves to detect its surroundings, such as the corner position information of obstacle corners, acquired and processed by radar or other ultrasonic monitoring devices.

[0038] Specifically, for parking environments with obstacle corners, at each time frame during the vehicle's movement, environmental information representing the vehicle's surrounding environment, trajectory information representing the vehicle's real-time position, and ultrasonic information representing obstacle corners around the vehicle are acquired.

[0039] It is understood that the obstacle corner here refers to a corner point that has a prominent position relative to a relatively flat ground, and its specific type is not limited. It can be any obstacle such as an ice cream bucket, curb, stone block, or vehicle.

[0040] S102. Perform semantic segmentation on the environmental image to obtain the image to be processed containing semantic information.

[0041] In this embodiment, semantic information can be understood as a statement or representation of the objects contained in the environmental image. The image to be processed can be understood as an image on which semantic information is superimposed onto the environmental image, and the semantic information is visualized.

[0042] The semantic information includes at least one of parking space lines, lane lines, drivable areas and non-drivable areas. The non-drivable area includes at least the area where the obstacle is located and the area where the vehicle is currently located, and may also include the curb. Figure 2 This is an example image of the image to be processed in a parking space determination method according to Embodiment 1 of the present invention, such as... Figure 2 As shown, in the image to be processed in the current frame, purple lines represent lane lines, red lines represent parking lines, and black areas represent non-drivable areas. Within the non-drivable areas, the black area in the middle is the area where the vehicle itself is located, and the black areas on the left and right sides are the areas where obstacle vehicles are located, i.e., the obstacle vehicles themselves.

[0043] Specifically, semantic segmentation is performed on the environmental image for each time frame. Semantic segmentation methods such as Mask Region-based Convolutional Neural Network (Mask R-CNN) or U-Net are used to identify lane lines, parking lines, curbs, obstructing vehicles, and the vehicle itself in the environmental image, obtaining corresponding semantic information. The areas containing curbs, obstructing vehicles, and the vehicle itself are merged into a non-drivable region. All other regions are considered drivable. This semantic information is then visualized on the environmental image to obtain the image to be processed.

[0044] The above technical features allow the obtained image to accurately represent the shape and position of obstacles, providing a precise reference for subsequent parking space determination.

[0045] S103. Obstacle points are determined based on semantic information and track information to obtain a set of obstacle points.

[0046] In this embodiment, the set of obstacle points can be understood as the set of obstacle points that are closest to the vehicle during the vehicle's movement.

[0047] Specifically, at the start of the method execution, an empty obstacle point set is established. Based on the current time frame, i.e., the positional distance between the position coordinates of the obstacle (obstacle vehicle) corresponding to the semantic information of the current frame and the position coordinates of the vehicle itself, the obstacle point closest to the vehicle is determined, and this obstacle point is designated as the target obstacle point and added to the obstacle point set. It can be understood that the obstacle point set is updated in real time; in a new time frame, the current target obstacle point and its coordinate information corresponding to that frame are added to the obstacle point set.

[0048] While adding the current target obstacle point and its coordinate information to the obstacle point set, the coordinate information of each historical target obstacle point in the obstacle point set at this time frame is updated based on the track information of the current frame and the previous frame, or based on the track information corresponding to each frame in the historical time period, so as to obtain the obstacle point set with accurate relative position at the current time frame.

[0049] For example, in the first frame, the current target obstacle point is determined based on the semantic information corresponding to the image to be processed in the first frame, an empty obstacle point set is constructed, and the current target obstacle point and its corresponding coordinate information corresponding to the first frame are added to the obstacle point set. In the second frame, the current target obstacle point in the first frame is determined as the historical target obstacle point, the current target obstacle point is determined based on the semantic information corresponding to the image to be processed in the second frame, the current target obstacle point and its corresponding coordinate information corresponding to the second frame are added to the obstacle point set, and the coordinate information of the first frame is updated based on the track information of the second frame and the track information of the first frame, thereby updating the obstacle point set. In the third frame, the current target obstacle point in the second frame is identified as a historical target obstacle point. Based on the semantic information of the image to be processed in the third frame, the current target obstacle point is determined. The current target obstacle point and its coordinate information in the third frame are added to the obstacle point set. The coordinate information of the first and second frames is updated based on the track information of the third and second frames (or based on the track information of the third, second, and first frames), thus updating the obstacle point set. Taking the first three frames as an example, subsequent steps continuously repeat the process of adding the current target obstacle point and updating the historical target obstacle point.

[0050] S104. Based on the obstacle point set and ultrasonic information, perform parking space fitting calculation to obtain the target parking space.

[0051] In this embodiment, the target parking space can be understood as a fitted parking space that is available for vehicles to park in.

[0052] Specifically, the nearest obstacle corner point is determined based on ultrasonic information, and the parking space type is determined based on the positional relationship of each obstacle corner point. The parking space type can include parallel parking spaces and reverse parking spaces, or other types of parking spaces; this embodiment does not limit this. Given the possibility of errors or instability in the obstacle corner points, the position coordinates of each obstacle point in the obstacle point set are compared with the corner point position coordinates to determine the target obstacle point closest to the obstacle corner point. This target obstacle point is used as the corrected obstacle corner point. Based on the corrected obstacle corner point and the parking space type, a parking space fitting calculation is performed. If the fitted parking space matches the vehicle's parameters (e.g., vehicle width and length), the fitted parking space is determined as the target parking space. The target parking space is then output and fed back to the vehicle control system or visualized on the driver's end.

[0053] The parking space determination method provided in this embodiment improves the acquisition of environmental images, trajectory information, and ultrasonic information of the vehicle; performs semantic segmentation processing on the environmental image to obtain a processing image containing semantic information; determines obstacle points based on the semantic information and trajectory information to obtain an obstacle point set; and performs parking space fitting calculation based on the obstacle point set and ultrasonic information to obtain the target parking space. These technical features combine visual images and ultrasonic information, determining obstacle points through the processing image, tracking obstacle points through trajectory information to obtain an obstacle point set, and correcting the parking space corner points corresponding to the ultrasonic information based on the obtained obstacle point set, achieving more accurate parking space corner point determination and fitting a more accurate target parking space. This solves the problem of low accuracy in constructing parking spaces solely based on ultrasonic waves, improves the accuracy of parking space fitting and construction, and thus ensures the smooth operation of subsequent automatic parking, improving the user experience.

[0054] As a first optional embodiment of this method, the method further includes controlling the vehicle to park in the target parking space.

[0055] Specifically, after the target parking space is output to the vehicle control system, the system generates a parking command and controls the vehicle to automatically park in the target space based on the command. Alternatively, the target parking space can be output to the driver's end (i.e., the target parking space is visually displayed). When the driver sees the target parking space and triggers a parking operation, the system generates a parking command in response to the triggering of the corresponding parking control, and the vehicle automatically parks in the target space based on the command.

[0056] Example 2

[0057] Figure 3This is a flowchart of a parking space determination method provided in Embodiment 2 of the present invention. This embodiment is a further extension of the above embodiments and is applicable to the case of parking space fitting and construction based on visual images and ultrasonic information.

[0058] like Figure 3 As shown, the method includes:

[0059] S201. Acquire environmental images, trajectory information, and ultrasonic information of the vehicle.

[0060] S202. Perform semantic segmentation on the environmental image to obtain an image to be processed containing semantic information.

[0061] S203. Traverse the semantic information of the image to be processed, determine the current target obstacle point of the vehicle based on the semantic information, and add the current target obstacle point and the first coordinate information associated with the current target obstacle point to the obstacle point set.

[0062] In this embodiment, the current target obstacle point can be understood as the obstacle point closest to the vehicle in the current frame. The first coordinate information can be understood as information representing the position of the current target obstacle point and the position of the vehicle itself when the current target obstacle point is determined, including the obstacle point coordinates of the current target obstacle point in the current frame coordinate system and the trajectory coordinates of the vehicle itself in the current frame. The first coordinate information is the initially stored information and has not been updated.

[0063] Specifically, the semantic information of the image to be processed is traversed, and each semantic information has corresponding coordinate information, such as the coordinate information of each obstacle point of the vehicle in the drivable area. Based on the position coordinates of each obstacle point corresponding to each semantic information and the position coordinates of the vehicle itself, the obstacle point closest to the vehicle is determined, and this obstacle point is identified as the current target obstacle point. The position coordinates of the current target obstacle point and the trajectory information of the vehicle itself collected in the current frame are determined as the first coordinate information and stored in the obstacle point set.

[0064] Figure 4 This is an example diagram of the current target obstacle point in a parking space determination method according to Embodiment 2 of the present invention, as shown in the figure. Figure 4 As shown, in the image to be processed, the position of the vehicle's rearview mirror is taken as the target position of the vehicle itself. A horizontal line is generated along the X-axis at the target position. The semantic information in the image to be processed is traversed, and the intersection of the horizontal line with the obstacle vehicle is taken, that is, the position of the yellow dot. This position is closest to the target position of the vehicle on the horizontal line, and this position is determined as the current target obstacle point.

[0065] Optionally, the current target obstacle point of the vehicle is determined based on semantic information, and the current target obstacle point and the first coordinate information associated with the current target obstacle point are added to the obstacle point set, including:

[0066] S2031. Establish the image coordinate system corresponding to the image to be processed and the vehicle coordinate system relative to the vehicle.

[0067] In this embodiment, the image coordinate system C1 can be understood as an XY-axis rectangular coordinate system established with the corner point of the image to be processed as the origin. The origin can be the upper left or lower left corner of the image to be processed. For example, the image coordinate system can be established with the upper left corner of the image to be processed as the origin, with the X direction to the right and the Y direction downward. The vehicle coordinate system C2 can be understood as an XY-axis rectangular coordinate system established with the center of the vehicle as the origin. The origin can be the center of the rear axle, the center of the front axle, or the center of the entire vehicle, etc. For example, the vehicle coordinate system can be established with the center of the entire vehicle as the origin, with the X direction to the right and the Y direction downward.

[0068] Specifically, the image to be processed contains vehicles, so the image coordinate system of the image to be processed can be considered to contain the vehicle coordinate system of the vehicles. The origin of the vehicle coordinate system has a specific coordinate position in the image coordinate system. Therefore, the coordinates can be converted between the image coordinate system and the vehicle coordinate system.

[0069] S2032. In the image coordinate system, identify the coordinates of each obstacle point corresponding to the semantic information of the obstacle, and determine the current target obstacle point based on the distance between the coordinates of each obstacle point and the target position coordinates on the vehicle.

[0070] In this embodiment, obstacle point coordinates can be understood as the coordinates of the edge of an obstacle (e.g., an obstacle vehicle). The target position can be understood as a selected position on the vehicle, such as the location of the left and right rearview mirrors. The target position coordinates are the coordinates of the target location.

[0071] Specifically, the semantic information of obstacles is visualized in the image to be processed. When constructing the image coordinate system of the image to be processed, obstacles have specific coordinate regions in the image coordinate system. Correspondingly, each obstacle point at the upper edge of the obstacle also has its specific position coordinates in the image coordinate system, that is, obstacle point coordinates. Similarly, the target position on the vehicle also has specific position coordinates in the image coordinate system, that is, target position coordinates (Xm, Ym). By traversing the coordinates of each obstacle point, the obstacle point coordinates (Px, Py) with the smallest difference between the Y coordinate and Ym are determined, and the obstacle point corresponding to (Px, Py) is taken as the current target obstacle point.

[0072] S2033. Convert the first position coordinates of the current target obstacle point in the image coordinate system to the second position coordinates in the vehicle coordinate system, and determine the second position coordinates and the track information of the current frame as the first coordinate information.

[0073] In this embodiment, the first position coordinates can be understood as the coordinates of the current target obstacle point in the image coordinate system, and the second position coordinates can be understood as the coordinates of the current target obstacle point in the vehicle coordinate system.

[0074] Specifically, since the vehicle coordinate system is contained within the image coordinate system, based on the relative relationship between the origins of the vehicle coordinate system and the image coordinate system, the first position coordinates (Px, Py) of the current target obstacle point in the image coordinate system are transformed to obtain the second position coordinates (Cx, Cy) of the current target obstacle point in the vehicle coordinate system. The first position coordinates (Px, Py) and the second position coordinates (Cx, Cy) represent the same current target obstacle point; the only difference is the coordinate values ​​due to the change in the reference coordinate system. The second position coordinates of the current target obstacle point and the trajectory information of the current frame are used to determine the first coordinate information.

[0075] S2034. Associate the current target obstacle point and the first coordinate information and add them to the obstacle point set.

[0076] In this embodiment, the current target obstacle point is associated with the first coordinate information. Therefore, when the current target obstacle point is added to the obstacle point set, its associated first coordinate information is also added to the obstacle point set.

[0077] S204. Track obstacles based on flight path information, determine the second coordinate information of each historical target obstacle in the obstacle point set, and update the obstacle point set based on the second coordinate information.

[0078] In this embodiment, the historical target obstacle point can be understood as the obstacle point closest to the vehicle within a past historical time frame. The second coordinate information can be understood as information representing the relative position of the historical target obstacle point to the vehicle in the current frame and the vehicle's own position when the historical target obstacle point was determined. This includes the obstacle point coordinates of the historical target obstacle point in the current frame coordinate system and the vehicle's own track coordinates in the historical frame. The second coordinate information is the position information calculated in the current frame, and is the updated information.

[0079] Specifically, for each historical target obstacle in the obstacle point set, the attitude change between the current frame track information and the track information associated with each historical target obstacle is determined. Based on the attitude change, the obstacle point coordinates in the current frame coordinate system are calculated to achieve obstacle point tracking. The obstacle point coordinates and the vehicle's own track coordinates in the corresponding historical frame are used as the second coordinate information to replace the first or second coordinate information for that historical target obstacle in the obstacle point set, thereby updating the obstacle point set.

[0080] It is understandable that during obstacle tracking, the vehicle's trajectory information remains unchanged for each historical target obstacle in the corresponding historical frame. Obstacle tracking and obstacle set updates are achieved only by updating the obstacle coordinates.

[0081] Optionally, obstacle point tracking is performed based on the flight path information to determine the second coordinate information of each historical target obstacle point in the obstacle point set, including:

[0082] S2041. For each historical target obstacle in the obstacle point set, determine the vehicle's position change information and angle change information based on the current frame's track information and the track information corresponding to the historical target obstacle.

[0083] In this embodiment, position change information can be understood as the change in vehicle position between the current frame and the historical frame during vehicle movement, and angle change information can be understood as the change in vehicle steering angle between the current frame and the historical frame during vehicle movement.

[0084] Specifically, for a single historical target obstacle in the obstacle point set, the track information associated with the historical target obstacle is compared with the track information (curDR_X, curDR_Y) of the current frame, and the difference between the track coordinates (X0, Y0) and the difference 'a' between the vehicle driving angles corresponding to the two track information are taken.

[0085] S2042. Based on the track information, position change information and angle change information of the current frame, correct the position coordinates and determine the second coordinate information of the historical target obstacle point.

[0086] In this embodiment, the vehicle center point of the current frame is recorded based on the trajectory information (curDR_X, curDR_Y) of the current frame. Let the rotation matrix be... Let the translation matrix be... By calculating Pn = R * T * P, the obstacle coordinates Pn of the historical target obstacle point in the current frame's image coordinate system are obtained, enabling obstacle tracking of the historical target obstacle point. The obstacle coordinates Pn replace the original obstacle coordinates associated with the historical target obstacle point, and the corrected obstacle coordinates Pn are combined with the track information of the historical frame originally associated with the historical target obstacle point to form the second coordinate information.

[0087] Figure 5 This is an example diagram of obstacle tracking in a parking space determination method according to Embodiment 2 of the present invention, as shown in the figure. Figure 5As shown, the top yellow dot in the image represents the current target obstacle in the current frame, while the other yellow dots below it represent historical target obstacle points from different time frames. Each yellow dot and its corresponding coordinates are stored in an obstacle point set. Based on the vehicle's movement, target obstacle points are tracked using the vehicle's trajectory information, enabling real-time updates of obstacle point coordinates. This can be understood as... Figure 5 The green dots represent the boundary points of the image to be processed. When the vehicle is moving forward, if there is no intersection between the horizontal line of the target position and the obstacle, and the line extends directly to the edge of the image to be processed, then the intersection of the target position and the boundary of the image to be processed is taken as a non-obstacle point and represented by a green dot.

[0088] S205. Determine the corner point of the first parking space and the parking space type based on the ultrasonic information.

[0089] In this embodiment, the first parking space corner point can be understood as a physical obstacle point identified based on ultrasonic information. The parking space type can be understood as the parking type determined based on the first parking space corner point, including reverse parking spaces and parallel parking spaces, etc.

[0090] Specifically, ultrasonic positioning is performed based on ultrasonic information to determine the position of each corner point of the physical obstacle, and the parking space type is determined as a reverse parking space or a parallel parking space based on the relative distance or relative positional relationship between each corner point.

[0091] S206. Correct the first parking space corner point according to the obstacle point set to obtain the second parking space corner point.

[0092] In this embodiment, the second parking space corner point can be understood as the corrected obstacle point, that is, the edge point of the parking space to be fitted.

[0093] Specifically, the obstacle point whose Y-coordinate is closest to the Y-coordinate of the first parking space corner point is selected from the obstacle point set. This obstacle point is then designated as the second parking space corner point, thus correcting the first parking space corner point and obtaining a more accurate second parking space corner point. Alternatively, filtering and outlier filtering can be applied to the first parking space corner point according to actual needs to obtain a more accurate second parking space corner point.

[0094] Optionally, the first parking space corner point is corrected based on the obstacle point set to obtain the second parking space corner point, including:

[0095] S2061. Determine the expansion area of ​​the first parking space corner point based on the sum and difference between the position coordinates of the first parking space corner point and the set expansion threshold.

[0096] In this embodiment, the set expansion threshold can be understood as a pre-set floating value. The expansion area of ​​the first parking space corner point can be understood as the error range of the first parking space corner point.

[0097] Specifically, if there are two first parking space corner points P1(X1, Y1) and P2(X2, Y2), the sum and difference between the Y coordinate value of the first parking space corner point and the set outward expansion threshold offsetY are taken to obtain the corresponding expansion area [Y1-offestY, Y1+offestY] of the first parking space corner point P1 in the Y direction and the expansion area [Y2-offestY, Y2+offestY] of P2 in the Y direction.

[0098] S2062. Among the obstacle points in the obstacle point set, the obstacle points whose distance from the extended area satisfies the corner point correction condition are determined as the second parking space corner points.

[0099] In this embodiment, the corner correction condition can be understood as the condition used to select the obstacle point for correcting the corner point of the first parking space.

[0100] Specifically, the obstacle point with the smallest difference between its Y-coordinate and the Y-value contained in the extended region is selected from the set of obstacle points, and this obstacle point is determined to satisfy the corner point correction condition. Specifically, obstacle point PN1 with the smallest difference between its Y-coordinate and the Y-value contained in the extended region [Y1-offestY, Y1+offestY] of the first corner point P1, and obstacle point PN2 with the smallest difference between its Y-coordinate and the Y-value contained in the extended region [Y2-offestY, Y2+offestY] of the first corner point P2 are selected. Obstacle point PN1 is determined as the second parking space corner point after correction relative to the first parking space corner point P1, and obstacle point PN2 is determined as the second parking space corner point after correction relative to the first parking space corner point P2.

[0101] S207. Based on the corner point of the second parking space and the parking space type, fit and form the parking space to be output.

[0102] In this embodiment, the parking space to be output can be understood as the parking space formed by fitting.

[0103] Specifically, the coordinates of the corner point of the second parking space are filled into the corner point position of the contour corresponding to the parking space type, and the parking space to be output is formed by fitting.

[0104] S208. Determine whether the parking space to be output meets the parking space entry conditions. If so, determine the parking space to be output as the target parking space.

[0105] In this embodiment, the parking space entry condition can be understood as the condition used to determine whether a vehicle can safely park in the parking space to be output.

[0106] Specifically, the vehicle's own vehicle parameter information (such as vehicle width and length) is compared with the fitted parking space to be output. If the vehicle width is greater than the width of the parking space to be output and / or the vehicle length is greater than the length of the parking space to be output, it is determined that the current parking space entry conditions are not met. At this time, no action is taken, the vehicle continues to move forward, and continues to collect new environmental images, track information and ultrasonic information for the next parking space determination. If the vehicle width is less than the width of the parking space to be output and the vehicle length is less than the length of the parking space to be output, it is determined that the current parking space entry conditions are met, and the parking space to be output is determined as the target parking space.

[0107] This embodiment provides a parking space determination method, which involves acquiring environmental images, trajectory information, and ultrasonic information of a vehicle; performing semantic segmentation on the environmental image to obtain a to-be-processed image containing semantic information; traversing the semantic information of the to-be-processed image, determining the vehicle's current target obstacle point based on the semantic information, and adding the current target obstacle point and its associated first coordinate information to an obstacle point set; tracking the obstacle point based on the trajectory information, determining the second coordinate information of each historical target obstacle point in the obstacle point set, and updating the obstacle point set based on the second coordinate information; determining a first parking space corner point and parking space type based on ultrasonic information; correcting the first parking space corner point based on the obstacle point set to obtain a second parking space corner point; fitting and forming a parking space to be output based on the second parking space corner point and parking space type; and determining whether the parking space to be output meets the parking space entry conditions. If so, the parking space to be output is determined as the target parking space. The above technical solution introduces image semantic segmentation technology, updates the obstacle point set in real time based on visual images and trajectory information, and corrects the obstacle edge corner point position obtained by ultrasonic detection based on the obstacle point set to obtain a more accurate parking space corner point. This enables a more accurate fitting of the target parking space, improves the accuracy of parking space fitting and construction, and thus ensures the safety and success rate of subsequent automatic parking, and improves the user experience.

[0108] Example 3

[0109] Figure 6 This is a schematic diagram of a parking space determination device according to Embodiment 3 of the present invention. Figure 6 As shown, the device includes:

[0110] Information acquisition module 31 is used to acquire environmental images, trajectory information and ultrasonic information of the vehicle;

[0111] Image processing module 32 is used to perform semantic segmentation processing on the environmental image to obtain an image to be processed containing semantic information;

[0112] The obstacle point determination module 33 is used to determine obstacle points based on the semantic information and the trajectory information to obtain an obstacle point set.

[0113] The parking space determination module 34 is used to perform parking space fitting calculations based on the set of obstacle points and the ultrasonic information to obtain the target parking space.

[0114] The parking space determination device provided in this embodiment combines visual images and ultrasonic information. It determines obstacle points through the image to be processed and tracks obstacle points through flight path information to obtain a set of obstacle points. Based on the obtained set of obstacle points, it corrects the parking space corner points corresponding to the ultrasonic information, thereby achieving more accurate determination of parking space corner points and fitting a more accurate target parking space. This solves the problem of low accuracy in constructing parking spaces by relying solely on ultrasonic waves, improves the accuracy of parking space fitting and construction, and thus ensures the smooth operation of subsequent automatic parking, improving the user experience.

[0115] Optionally, the obstacle point determination module 33 includes:

[0116] The current obstacle point addition unit is used to traverse the semantic information of the image to be processed, determine the current target obstacle point of the vehicle according to the semantic information, and add the current target obstacle point and the first coordinate information associated with the current target obstacle point to the obstacle point set;

[0117] The historical obstacle point update unit is used to track obstacles based on the flight path information, determine the second coordinate information of each historical target obstacle point in the obstacle point set, and update the obstacle point set based on the second coordinate information.

[0118] Optionally, add a unit to the current obstacle point, specifically for:

[0119] Establish an image coordinate system relative to the image to be processed and a vehicle coordinate system relative to the vehicle;

[0120] In the image coordinate system, the coordinates of each obstacle point corresponding to the semantic information are identified, and the current target obstacle point is determined based on the distance between each obstacle point coordinate and the target position coordinates on the vehicle.

[0121] The first position coordinates of the current target obstacle point in the image coordinate system are converted into the second position coordinates in the vehicle coordinate system, and the second position coordinates and the trajectory information of the current frame are determined as the first coordinate information;

[0122] The current target obstacle point and the first coordinate information are associated and added to the obstacle point set.

[0123] Optional, historical obstacle point update unit, specifically used for:

[0124] For each historical target obstacle in the obstacle point set, the vehicle's position change information and angle change information are determined based on the track information of the current frame and the track information corresponding to the historical target obstacle.

[0125] Based on the track information, position change information, and angle change information of the current frame, the position coordinates are corrected to determine the second coordinate information of the historical target obstacle point.

[0126] Optionally, the parking space determination module 34 includes:

[0127] A corner point determination unit is used to determine the first parking space corner point and parking space type based on the ultrasonic information;

[0128] A corner correction unit is used to correct the first parking space corner based on the set of obstacle points to obtain the second parking space corner.

[0129] The parking space fitting unit is used to fit and form the parking space to be output based on the corner point of the second parking space and the parking space type;

[0130] The parking space determination unit is used to determine whether the parking space to be output meets the parking space entry conditions. If so, the parking space to be output is determined as the target parking space.

[0131] Optional, corner correction unit, specifically used for:

[0132] The expansion area of ​​the first parking space corner point is determined based on the sum and difference between the position coordinates of the first parking space corner point and the set expansion threshold.

[0133] The obstacle points in the set of obstacle points whose distance from the extended area satisfies the corner point correction condition are identified as the second parking space corner points.

[0134] Optionally, the device further includes a parking module for controlling the vehicle to park in the target parking space.

[0135] Optionally, the semantic information includes at least one of parking space lines, lane lines, drivable areas, and non-drivable areas, wherein the non-drivable areas include at least the area where the obstacle is located and the area where the vehicle is currently located.

[0136] The parking space determination device provided in the embodiments of the present invention can execute the parking space determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0137] Example 4

[0138] Figure 7This is a schematic diagram of the structure of a vehicle 40 according to Embodiment 4 of the present invention. The vehicle is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The vehicle can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0139] like Figure 7 As shown, vehicle 40 includes at least one processor 41 and a memory, such as read-only memory (ROM) 42 and random access memory (RAM) 43, communicatively connected to at least one processor 41. The memory stores computer programs executable by at least one processor. Processor 41 can perform various appropriate actions and processes based on the computer program stored in ROM 42 or loaded from storage unit 48 into RAM 43. RAM 43 can also store various programs and data required for the operation of vehicle 40. Processor 41, ROM 42, and RAM 43 are interconnected via bus 44. Input / output (I / O) interface 45 is also connected to bus 44.

[0140] Multiple components in vehicle 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of displays, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows vehicle 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0141] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as the parking space determination method.

[0142] In some embodiments, the parking space determination method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on vehicle 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the parking space determination method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the parking space determination method by any other suitable means (e.g., by means of firmware).

[0143] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0145] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0146] To provide interaction with the user, the systems and technologies described herein can be implemented in a vehicle having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the vehicle. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0147] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0148] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0149] In one embodiment, the present invention further includes a computer program product, which includes a computer program that, when executed by a processor, implements the parking space determination method of any embodiment of the present invention.

[0150] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0151] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0152] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A parking space determination method characterized by comprising: The method comprises: acquiring an environment image, track information and ultrasonic information of a vehicle; performing semantic segmentation processing on the environment image to obtain a to-be-processed image containing semantic information; determining obstacle points according to the semantic information and the track information to obtain an obstacle point set; performing parking space fitting calculation according to the obstacle point set and the ultrasonic information to obtain a target parking space; wherein the performing parking space fitting calculation according to the obstacle point set and the ultrasonic information to obtain a target parking space comprises: determining a first parking space corner point and a parking space type according to the ultrasonic information; determining an expansion region of the first parking space corner point according to the sum and difference between the position coordinates of the first parking space corner point and a set expansion threshold value; determining a second parking space corner point from the obstacle point set, which satisfies an angle point correction condition with the expansion region; fitting a to-be-output parking space according to the second parking space corner point and the parking space type; judging whether the to-be-output parking space satisfies a parking space parking-in condition, and if so, determining the to-be-output parking space as the target parking space.

2. The method of claim 1, wherein, The determining obstacle points according to the semantic information and the track information to obtain an obstacle point set comprises: traversing the semantic information of the to-be-processed image, determining a current target obstacle point of the vehicle according to the semantic information, and adding the current target obstacle point and first coordinate information associated with the current target obstacle point to the obstacle point set; tracking the obstacle points according to the track information to determine second coordinate information of each historical target obstacle point in the obstacle point set, and updating the obstacle point set according to the second coordinate information.

3. The method of claim 2, wherein, The determining a current target obstacle point of the vehicle according to the semantic information and adding the current target obstacle point and first coordinate information associated with the current target obstacle point to the obstacle point set comprises: establishing an image coordinate system relative to the to-be-processed image and a vehicle coordinate system relative to the vehicle; in the image coordinate system, identifying each obstacle point coordinate of an obstacle corresponding to the semantic information, and determining a current target obstacle point according to the distance between each obstacle point coordinate and a target position coordinate of a target position on the vehicle; converting the first position coordinate of the current target obstacle point in the image coordinate system into a second position coordinate in the vehicle coordinate system, and determining the second position coordinate and the track information of the current frame as the first coordinate information; associating and adding the current target obstacle point and the first coordinate information to the obstacle point set.

4. The method of claim 2, wherein, The tracking the obstacle points according to the track information to determine second coordinate information of each historical target obstacle point in the obstacle point set comprises: for each historical target obstacle point in the obstacle point set, determining position change information and angle change information of the vehicle according to the track information of the current frame and the track information corresponding to the historical target obstacle point; performing position coordinate correction according to the track information of the current frame, the position change information and the angle change information to determine the second coordinate information of the historical target obstacle point.

5. The method of claim 1, wherein, The method further comprises: controlling the vehicle to park in the target parking space.

6. The method of claim 1, wherein, The semantic information includes at least one of a parking line, a lane line, a drivable area and a non-drivable area, and the non-drivable area at least includes an area where an obstacle is located and an area where the vehicle is currently located.

7. A parking space determination apparatus characterized by comprising: Comprise: An information acquisition module configured to acquire an environment image, track information and ultrasonic information of a vehicle; An image processing module configured to perform semantic segmentation processing on the environment image to obtain a to-be-processed image containing semantic information; An obstacle point determination module configured to determine obstacle points according to the semantic information and the track information to obtain a set of obstacle points; A parking space determination module configured to perform parking space fitting calculation according to the set of obstacle points and the ultrasonic information to obtain a target parking space; The parking space determination module comprises: An angle point determination unit configured to determine a first parking space angle point and a parking space type according to the ultrasonic information; An angle point correction unit configured to determine an expansion area of the first parking space angle point according to a sum and a difference between a position coordinate of the first parking space angle point and a set expansion threshold value, and determine a second parking space angle point from the set of obstacle points that satisfies an angle point correction condition with the expansion area; A parking space fitting unit configured to fit a to-be-output parking space according to the second parking space angle point and the parking space type; A parking space determination unit configured to determine whether the to-be-output parking space satisfies a parking space parking-in condition, and if so, determine the to-be-output parking space as the target parking space.

8. A vehicle characterized by comprising: The vehicle comprises: At least one processor; and A memory connected in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the parking space determination method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the parking space determination method of any one of claims 1-6 when executed.

10. A computer program product, characterised in that, The computer program product comprises a computer program which, when executed by the processor, implements the parking space determination method according to any one of claims 1-6. The computer program product comprises a computer program which, when executed by the processor, implements the parking space determination method according to any one of claims 1-6.

Images