Parking space determination method, device, equipment, medium and product

Abstract

The invention discloses a parking space determination method and device, equipment, a medium and a product. The parking space determination method comprises the following steps: acquiring an environment image, track information and ultrasonic information of a vehicle; performing semantic segmentation processing on the environment image to obtain a to-be-processed image containing semantic information; determining obstacle points according to the semantic information and the track information to obtain an obstacle point set; and performing parking space fitting calculation according to the obstacle point set and the ultrasonic information to obtain a target parking space. According to the technical characteristics, the accuracy of parking space construction is improved, so that the smooth proceeding of subsequent automatic parking is guaranteed, and the use experience of a user is improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a parking space determination method, device, equipment, medium and product. Background Art

[0002] With the development of autonomous driving technology, automatic parking system has become an important research direction in the field of intelligent vehicles. In the automatic parking system, accurate detection of parking space is one of the prerequisites for realizing automatic parking of vehicles.

[0003] At present, for parking spaces without lines drawn in the space, parking space search mainly relies on ultrasonic detection of parking spaces. However, the accuracy of corner points calculated by ultrasonic detection is affected by the vehicle speed and the relative posture of the vehicle body and obstacles, which may lead to insufficient accuracy of the constructed ultrasonic parking space, resulting in poor parking effect or even parking failure. Summary of the invention

[0004] The present invention provides a parking space determination method, device, equipment, medium and product, which improve the accuracy of parking space construction, thereby ensuring the smooth progress of subsequent automatic parking and improving the user experience.

[0005] According to a first aspect of the present invention, a parking space determination method is provided, comprising:

[0006] Obtain environmental images, track information and ultrasonic information of the vehicle;

[0007] Performing semantic segmentation processing on the environment image to obtain an image to be processed containing semantic information;

[0008] Determine obstacle points according to the semantic information and the track information to obtain an obstacle point set;

[0009] A parking space fitting calculation is performed according to the obstacle point set and the ultrasonic information to obtain a target parking space.

[0010] According to a second aspect of the present invention, there is provided a parking space determination device, comprising:

[0011] An information acquisition module, used to acquire environmental images, track information and ultrasonic information of the vehicle;

[0012] An image processing module is used to perform semantic segmentation processing on the environment image to obtain an image to be processed containing semantic information;

[0013] An obstacle point determination module, used to determine obstacle points according to the semantic information and the track information to obtain an obstacle point set;

[0014] The parking space determination module is used to perform parking space fitting calculation according to the obstacle point set and the ultrasonic information to obtain a target parking space.

[0015] According to a third aspect of the present invention, there is provided a vehicle, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a parking space determination method described in any embodiment of the present invention.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a parking space determination method described in any embodiment of the present invention when executed.

[0020] According to a fifth aspect of the present invention, an embodiment of the present invention further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, a parking space determination method according to any embodiment of the present invention is implemented.

[0021] The technical solution of the embodiment of the present invention is to obtain the environmental image, track information and ultrasonic information of the vehicle; perform semantic segmentation processing on the environmental image to obtain an image to be processed containing semantic information; determine obstacle points based on the semantic information and the track information to obtain an obstacle point set; perform parking space fitting calculation based on the obstacle point set and the ultrasonic information to obtain a target parking space. The above technical features combine visual images with ultrasonic information, determine obstacle points through the image to be processed, track obstacle points through track information to obtain an obstacle point set, and correct the parking space corner points corresponding to the ultrasonic information based on the obtained obstacle point set, so as to achieve more accurate parking space corner point determination and fit a more accurate target parking space, thereby solving the problem of low accuracy of constructing parking spaces relying solely on ultrasonic waves, improving the accuracy of parking space fitting construction, and thus ensuring the smooth progress of subsequent automatic parking and improving user experience.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flow chart of a parking space determination method provided according to Embodiment 1 of the present invention;

[0025] Figure 2 is an example diagram of an image to be processed in a parking space determination method provided in Embodiment 1 of the present invention;

[0026] Figure 3 is a flow chart of a parking space determination method provided according to Embodiment 2 of the present invention;

[0027] Figure 4 is an example diagram of a current target obstacle point in a parking space determination method provided according to Embodiment 2 of the present invention;

[0028] Figure 5 is an example diagram of obstacle point tracking in a parking space determination method provided in Embodiment 2 of the present invention;

[0029] Figure 6 is a schematic diagram of the structure of a parking space determination device provided according to Embodiment 3 of the present invention;

[0030] Figure 7 It is a structural schematic diagram of a vehicle provided according to Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] The existing method of ultrasonic parking space detection is affected by the vehicle speed and driving angle, and the detected parking space is inaccurate. The deviation is too large, resulting in the interruption of parking due to obstacles, thus parking failure. Alternatively, the parking space output by ultrasonic wave is not accurate enough, resulting in the parking not being centered, affecting the user getting off the vehicle or affecting the users of the neighboring vehicles getting on and off the vehicle, seriously affecting the user experience. Therefore, the embodiment of the present invention provides a parking space determination method based on the combination of visual images and ultrasonic information to improve the accuracy of parking space determination.

[0035] Figure 1 A flowchart of a parking space determination method is provided for the first embodiment of the present invention. This embodiment is applicable to the case where a parking space is constructed based on visual images and ultrasonic information. The method can be executed by a parking space determination device, which can be implemented in the form of hardware and / or software, and can be integrated in a vehicle. Figure 1 As shown, the method includes:

[0036] S101, obtaining the vehicle's environmental image, track information, and ultrasonic information.

[0037] In this embodiment, the environmental image can be understood as an image that encompasses the surrounding environment of the vehicle, such as a bird's-eye view of the vehicle, which is collected and synthesized by a camera on the vehicle body. Dead Reckoning (DR) can be understood as the trajectory information of the vehicle during driving, that is, the real-time positioning information of the vehicle itself, which may include track coordinates. Ultrasonic information can be understood as information obtained after the vehicle sends ultrasonic detection to the surrounding area, such as the corner position information of the obstacle corner, which is collected and processed by radar or other ultrasonic monitoring devices.

[0038] Specifically, for a parking environment with obstacle corners, in each time frame during the vehicle's driving process, environmental information representing the vehicle's surrounding environment in the time frame, track information representing the vehicle's real-time position, and ultrasonic information representing the obstacle corners around the vehicle are obtained.

[0039] It can be understood that the obstacle corner point here is a corner point with a prominent position relationship relative to the flat ground, and its specific type is not limited. It can be any obstacle such as an ice cream bucket, a curb, a stone pier or a vehicle.

[0040] S102: Perform semantic segmentation processing on the environment image to obtain an image to be processed containing semantic information.

[0041] In this embodiment, the semantic information can be understood as a statement or representation of an object contained in the environment image. The image to be processed can be understood as an image in which the semantic information is superimposed on the environment image to visualize the semantic information.

[0042] The semantic information includes at least one of parking space lines, lane lines, drivable areas and non-drivable areas. The non-drivable areas include at least areas where obstacles are located and areas where the vehicle is currently located, and may also include curbs. Figure 2 is an example diagram of an image to be processed in a parking space determination method provided in Embodiment 1 of the present invention, such as Figure 2 As shown, in the image to be processed in the current frame, the purple lines represent lane lines, the red lines represent parking space lines, and the black areas represent non-drivable areas. In the non-drivable areas, the black area in the middle is the area where the vehicle itself is located, also the vehicle itself, and the black areas on the left and right are the areas where the obstacle vehicle is located, that is, the obstacle vehicle itself.

[0043] Specifically, semantic segmentation processing is performed on the environment image of each time frame, such as Mask Region-based Convolutional Neural Network (Mask R-CNN) or U-Net semantic segmentation methods, to identify lane lines, parking space lines, curbs, obstacle vehicles and the vehicle in the environment image, and obtain corresponding semantic information. The area where the curb, obstacle vehicles and the vehicle are located are combined as the vehicle's non-drivable area. Except for the non-drivable area, other areas are drivable areas. The semantic information is visualized on the environment image to obtain the image to be processed.

[0044] With the above technical features, the image to be processed obtained can accurately characterize the shape and position of the obstacle, providing an accurate reference for subsequent parking space determination.

[0045] S103: Determine obstacle points according to the semantic information and the track information to obtain an obstacle point set.

[0046] In this embodiment, the obstacle point set may be understood as a set of obstacle points that are closest to the vehicle during the vehicle's driving process.

[0047] Specifically, when the method starts to execute, an empty obstacle point set is established, and the obstacle point closest to the vehicle is determined according to the current time frame, that is, the position distance between the position coordinates of the obstacle (obstacle vehicle) corresponding to the semantic information of the current frame and the position coordinates of the vehicle itself, and the obstacle point is determined as the target obstacle point and added to the obstacle point set. It can be understood that the obstacle point set is updated in real time, and in the new time frame, the current target obstacle point corresponding to the frame and its coordinate information are added to the obstacle point set.

[0048] While the current target obstacle point and its coordinate information are added to the obstacle point set, the coordinate information of each historical target obstacle point in the obstacle point set in the time frame is updated based on the track information of the current frame and the previous frame, or based on the track information corresponding to the current frame and each frame in the historical time, to obtain a set of obstacle points with accurate relative positions in the current time frame.

[0049] Exemplarily, in the first frame, the current target obstacle point is determined according to the semantic information corresponding to the first frame of the image to be processed, an empty obstacle point set is constructed, and the current target obstacle point corresponding to the first frame and the coordinate information corresponding to the current target obstacle point are added to the obstacle point set. In the second frame, the current target obstacle point of the first frame is determined as the historical target obstacle point, the current target obstacle point is determined according to the semantic information corresponding to the second frame of the image to be processed, the current target obstacle point corresponding to the second frame and the coordinate information corresponding to the current target obstacle point are added to the obstacle point set, and the coordinate information of the first frame is updated based on the track information of the second frame and the track information of the first frame, so as to realize the update of the obstacle point set. In the third frame, the current target obstacle point of the second frame is determined as the historical target obstacle point, the current target obstacle point is determined according to the semantic information corresponding to the image to be processed in the third frame, the current target obstacle point corresponding to the third frame and the coordinate information corresponding to the current target obstacle point are added to the obstacle point set, and the coordinate information of the first frame and the second frame is updated based on the track information of the third frame and the track information of the second frame (or based on the track information of the third frame, the track information of the second frame and the track information of the first frame), so as to realize the update of the obstacle point set. Taking the first three frames as an example, the subsequent steps continuously repeat the process of adding the current target obstacle point and updating the historical target obstacle point.

[0050] S104: Perform parking space fitting calculation according to the obstacle point set and the ultrasonic information to obtain the target parking space.

[0051] In this embodiment, the target parking space can be understood as a fitted parking space available for the vehicle to park.

[0052] Specifically, the nearest obstacle corner point is determined based on the ultrasonic information, and the parking space type is determined based on the positional relationship of each obstacle corner point. The parking space type may include a side parking space and a reverse parking space, and may also include other types of parking spaces, which are not limited in this embodiment. Under the premise that the obstacle corner point may be erroneous or unstable, the position coordinates of each obstacle point in the obstacle point set are compared with the corner point position coordinates of the obstacle corner point, and the target obstacle point closest to the obstacle corner point is determined. The target obstacle point is used as the corrected obstacle corner point, and the parking space is fitted based on the corrected obstacle corner point and the parking space type. If the fitted parking space is consistent with the vehicle parameters of the vehicle itself (such as vehicle width and vehicle length, etc.), the fitted parking space is determined as the target parking space, and the target parking space is output and fed back to the vehicle control system or visualized on the driver side.

[0053] The parking space determination method provided in this embodiment improves the acquisition of the vehicle's environmental image, track information and ultrasonic information; performs semantic segmentation processing on the environmental image to obtain an image to be processed containing semantic information; determines obstacle points based on the semantic information and track information to obtain an obstacle point set; performs parking space fitting calculation based on the obstacle point set and ultrasonic information to obtain a target parking space. The above technical features combine visual images with ultrasonic information, determine obstacle points through the image to be processed, track obstacle points through track information to obtain an obstacle point set, and correct the parking space corner points corresponding to the ultrasonic information based on the obtained obstacle point set, so as to achieve more accurate parking space corner point determination and fit a more accurate target parking space, thereby solving the problem of low accuracy of constructing parking spaces relying solely on ultrasonic waves, improving the accuracy of parking space fitting construction, and thus ensuring the smooth progress of subsequent automatic parking and improving user experience.

[0054] As a first optional embodiment of this embodiment, the method further includes controlling the vehicle to park in a target parking space.

[0055] Specifically, after the target parking space is output to the vehicle control system, the vehicle control system generates a parking instruction, and controls the vehicle to automatically park in the target parking space based on the parking instruction. Alternatively, the target parking space is output to the driver end (i.e., the target parking space is visually presented), and when the driver sees the target parking space and triggers the parking operation, the parking instruction is generated in response to the triggering of the parking control corresponding to the parking operation, and the vehicle automatically parks in the target parking space based on the parking control.

[0056] Embodiment 2

[0057] Figure 3This is a flow chart of a parking space determination method provided in Embodiment 2 of the present invention. This embodiment is a further extension of the above embodiments and is applicable to the situation where parking space fitting is constructed based on visual images and ultrasonic information.

[0058] like Figure 3 As shown, the method includes:

[0059] S201, obtaining the vehicle's environmental image, track information, and ultrasonic information.

[0060] S202: Perform semantic segmentation processing on the environment image to obtain an image to be processed containing semantic information.

[0061] S203, traversing the semantic information of the image to be processed, determining the current target obstacle point of the vehicle according to the semantic information, and adding the current target obstacle point and the first coordinate information associated with the current target obstacle point to the obstacle point set.

[0062] In this embodiment, the current target obstacle point can be understood as the obstacle point closest to the vehicle in the current frame. The first coordinate information can be understood as information representing the position of the current target obstacle point and the position of the vehicle itself when the current target obstacle point is determined, including the obstacle point coordinates of the current target obstacle point in the current frame coordinate system and the track coordinates of the vehicle itself in the current frame. The first coordinate information is the initially stored information and has not been updated.

[0063] Specifically, the semantic information of the image to be processed is traversed, and each semantic information has corresponding coordinate information, such as the coordinate information of each obstacle point of the obstacle vehicle in the non-drivable area. Based on the position coordinates of each obstacle point corresponding to each semantic information and the position coordinates of the vehicle itself, the obstacle point closest to the vehicle itself is determined, and the obstacle point is determined as the current target obstacle point, and the position coordinates of the current target obstacle point and the track information of the vehicle itself collected in the current frame are determined as the first coordinate information and stored in the obstacle point set.

[0064] Figure 4 is an example diagram of a current target obstacle point in a parking space determination method provided by Embodiment 2 of the present invention, such as Figure 4 As shown, in the image to be processed, the position of the vehicle's rearview mirror is taken as the target position selected by the vehicle itself, a horizontal line is generated at the target position along the X-axis direction, and each semantic information in the image to be processed is traversed. The intersection position of the horizontal line and the obstacle vehicle is taken, that is, the position of the yellow dot. This position is closest to the target position of the vehicle on the horizontal line, and this position is determined as the current target obstacle point.

[0065] Optionally, determining the current target obstacle point of the vehicle according to the semantic information, and adding the current target obstacle point and the first coordinate information associated with the current target obstacle point to the obstacle point set includes:

[0066] S2031. Establish an image coordinate system relative to the image to be processed and a vehicle coordinate system relative to the vehicle.

[0067] In this embodiment, the image coordinate system C1 can be understood as an XY-axis rectangular coordinate system established with the corner position of the image to be processed as the origin. The origin can be the upper left corner or the lower left corner of the image to be processed. The established image coordinate system can be, for example, as follows: the upper left corner of the image to be processed is the origin, the X direction is rightward, and the Y direction is downward. The vehicle coordinate system C2 can be understood as an XY-axis rectangular coordinate system established with the center position of the vehicle as the origin. The origin can be the center of the rear axle, the center of the front axle, or the center of the entire vehicle. The established vehicle coordinate system can be, for example, as follows: the center of the entire vehicle is the origin, the X direction is rightward, and the Y direction is downward.

[0068] Specifically, the image to be processed contains a vehicle, so it can be considered that the image coordinate system of the image to be processed contains the vehicle coordinate system of the vehicle, and the origin of the vehicle coordinate system has a specific coordinate position in the image coordinate system. Therefore, each coordinate can be converted between the image coordinate system and the vehicle coordinate system.

[0069] S2032. In the image coordinate system, identify the coordinates of each obstacle point of the obstacle corresponding to the semantic information, and determine the current target obstacle point according to the distance between the coordinates of each obstacle point and the target position coordinates of the target position on the vehicle.

[0070] In this embodiment, the obstacle point coordinates can be understood as the coordinates of the edge position of the obstacle (such as the obstacle vehicle). The target position can be understood as a selected position on the vehicle, such as the position of the left and right rearview mirrors. The target position coordinates are the coordinates of the target position.

[0071] Specifically, the semantic information of the obstacle is visualized in the image to be processed. When the image coordinate system is constructed from the image to be processed, the obstacle has a specific coordinate area in the image coordinate system. Correspondingly, each obstacle point on the edge of the obstacle also has its specific position coordinates in the image coordinate system, that is, the obstacle point coordinates. Similarly, the target position on the vehicle also has specific position coordinates in the image coordinate system, that is, the target position coordinates (Xm, Ym). The coordinates of each obstacle point are traversed, and the obstacle point coordinates (Px, Py) with the smallest difference between the Y coordinate and Ym are determined from the coordinates of each obstacle point, and the obstacle point corresponding to (Px, Py) is used as the current target obstacle point.

[0072] S2033: Convert the first position coordinates of the current target obstacle point in the image coordinate system into second position coordinates in the vehicle coordinate system, and determine the second position coordinates and the track information of the current frame as the first coordinate information.

[0073] In this embodiment, the first position coordinates may be understood as the coordinate values ​​of the current target obstacle point in the image coordinate system, and the second position coordinates may be understood as the coordinate values ​​of the current target obstacle point in the vehicle coordinate system.

[0074] Specifically, since the vehicle coordinate system is included in the image coordinate system, based on the relative relationship between the origin of the vehicle coordinate system and the origin of the image coordinate system, the first position coordinate (Px, Py) of the current target obstacle point in the image coordinate system is transformed to obtain the second position coordinate (Cx, Cy) of the current target obstacle point in the vehicle coordinate system. The first position coordinate (Px, Py) and the second position coordinate (Cx, Cy) represent the same current target obstacle point. The coordinate values ​​presented are different only due to the change of the reference coordinate system. The second position coordinate of the current target obstacle point and the track information of the current frame are determined as the first coordinate information.

[0075] S2034. Associate the current target obstacle point with the first coordinate information and add them to the obstacle point set.

[0076] In this embodiment, the current target obstacle point is associated with the first coordinate information. Therefore, when the current target obstacle point is added to the obstacle point set, the first coordinate information associated with the current target obstacle point is also added to the obstacle point set.

[0077] S204: Track the obstacle points according to the track information, determine the second coordinate information of each historical target obstacle point in the obstacle point set, and update the obstacle point set according to the second coordinate information.

[0078] In this embodiment, the historical target obstacle point can be understood as the obstacle point closest to the vehicle in the past historical time frame. The second coordinate information can be understood as information representing the relative position of the historical target obstacle point to the vehicle in the current frame and the position of the vehicle itself when the historical target obstacle point is determined, including the obstacle point coordinates of the historical target obstacle point in the current frame coordinate system and the track coordinates of the vehicle itself in the historical frame. The second coordinate information is the position information calculated in the current frame and is updated information.

[0079] Specifically, for each historical target obstacle point in the obstacle point set, the attitude change amount between the current frame track information and the track information associated with each historical target obstacle point is determined, and the obstacle point coordinates of the historical target obstacle point in the current frame coordinate system are calculated based on the attitude change amount to achieve obstacle point tracking. The obstacle point coordinates and the track coordinates of the vehicle itself in the historical frame corresponding to the historical target obstacle point are used as the second coordinate information to replace the first coordinate information or the second coordinate information for the historical target obstacle point in the obstacle point set, thereby achieving the update of the obstacle point set.

[0080] It is understandable that in the process of obstacle point tracking, the track information of the vehicle itself remains unchanged for each historical target obstacle point in the corresponding historical frame, and obstacle point tracking and obstacle point set updating are achieved only by updating the obstacle point coordinates of the obstacle point.

[0081] Optionally, obstacle point tracking is performed according to the track information to determine the second coordinate information of each historical target obstacle point in the obstacle point set, including:

[0082] S2041. For each historical target obstacle point in the obstacle point set, determine the position change information and angle change information of the vehicle according to the track information of the current frame and the track information corresponding to the historical target obstacle point.

[0083] In this embodiment, the position change information can be understood as the change in the vehicle position corresponding to the current frame and the historical frame during the vehicle driving process, and the angle change information can be understood as the change in the vehicle steering angle corresponding to the current frame and the historical frame during the vehicle driving process.

[0084] Specifically, for a single historical target obstacle point in the obstacle point set, the track information associated with the historical target obstacle point is compared with the track information (curDR_X, curDR_Y) of the current frame, and the difference (X0, Y0) between the track coordinates corresponding to the two track information and the difference a between the vehicle driving angles corresponding to the two track information are taken.

[0085] S2042: Correct the position coordinates according to the track information, position change information and angle change information of the current frame to determine the second coordinate information of the historical target obstacle point.

[0086] In this embodiment, based on the track information (curDR_X, curDR_Y) of the current frame, the center point of the vehicle in the current frame is recorded. Let the rotation matrix be Let the translation matrix be Through Pn=R*T*P, the obstacle point coordinates Pn of the historical target obstacle point in the image coordinate system of the current frame are calculated to realize obstacle point tracking for the historical target obstacle point, and the obstacle point coordinates Pn replace the obstacle point coordinates originally associated with the historical target obstacle point. The corrected obstacle point coordinates Pn and the track information of the historical frame originally associated with the historical target obstacle point are combined to form the second coordinate information.

[0087] Figure 5 is an example diagram of obstacle point tracking in a parking space determination method provided in Embodiment 2 of the present invention, such as Figure 5As shown in the figure, the yellow dot at the top of the image is the current target obstacle point of the current frame, and the other yellow dots below it are the historical target obstacle points of each time frame. Each yellow dot and its corresponding coordinates are stored in the obstacle point set. Based on the vehicle's driving process, the target obstacle point is tracked through the vehicle's track information to achieve real-time update of the obstacle point coordinates. It can be understood that Figure 5 The middle green dot is the boundary point of the image to be processed. When the vehicle is moving forward, there is no intersection between the horizontal line of the vehicle's target position and the obstacle, and it extends directly to the edge of the image to be processed. The intersection of the target position and the boundary of the image to be processed is regarded as a non-obstacle point and is reflected in the form of a green dot.

[0088] S205: Determine the first parking space corner point and parking space type according to the ultrasonic information.

[0089] In this embodiment, the first parking space corner point can be understood as a physical obstacle point identified based on ultrasonic information. The parking space type can be understood as a parking type determined based on the first parking space corner point, including a reverse parking space and a side parking space.

[0090] Specifically, ultrasonic positioning is performed according to the ultrasonic information to determine the positions of the corner points of the physical obstacle, and the parking space type is determined to be a reverse parking space or a side parking space according to the relative distance or relative position relationship between the corner points.

[0091] S206: Correct the first parking space corner point according to the obstacle point set to obtain a second parking space corner point.

[0092] In this embodiment, the second parking space corner point can be understood as a corrected obstacle point, that is, an edge point of the parking space to be fitted.

[0093] Specifically, an obstacle point whose Y coordinate is closest to the Y coordinate of the first parking space corner point in the obstacle point set is selected, and the obstacle point is determined as the second parking space corner point, so as to correct the first parking space corner point and obtain a more accurate second parking space corner point. Alternatively, the first parking space corner point can be filtered and outliers can be filtered according to actual needs to obtain a more accurate second parking space corner point.

[0094] Optionally, the first parking space corner point is corrected according to the obstacle point set to obtain the second parking space corner point, including:

[0095] S2061: Determine an expansion area of ​​the first parking space corner point according to a sum and a difference between the position coordinates of the first parking space corner point and a set outward expansion threshold.

[0096] In this embodiment, the setting of the outer expansion threshold value can be understood as a pre-set floating value. The expansion area of ​​the first parking space corner point can be understood as the error range of the first parking space corner point.

[0097] Specifically, if there are two first parking corner points P1 (X1, Y1) and P2 (X2, Y2), take the sum and difference between the Y coordinate value of the first parking corner point and the set outward expansion threshold offestY, and correspondingly obtain the expansion area of ​​the first parking corner point P1 in the Y direction [Y1-offestY, Y1+offestY] and the expansion area of ​​P2 in the Y direction [Y2-offestY, Y2+offestY].

[0098] S2062: Determine, in the obstacle point set, an obstacle point whose distance from the extended area satisfies a corner point correction condition as a second parking space corner point.

[0099] In this embodiment, the corner point correction condition can be understood as a condition for selecting an obstacle point for correcting the corner point of the first parking space.

[0100] Specifically, the obstacle point with the smallest difference between the Y value of the obstacle point coordinate and the Y value included in the extended area in the obstacle point set is selected to determine that the obstacle point meets the corner point correction condition. Specifically, the obstacle point PN1 with the smallest difference between the Y value of the obstacle point coordinate and the Y value included in the extended area [Y1-offestY, Y1+offestY] of the first corner point coordinate P1, and the obstacle point PN2 with the smallest difference between the Y value of the obstacle point coordinate and the Y value included in the extended area [Y2-offestY, Y2+offestY] of the first corner point coordinate P2 are selected, and the obstacle point PN1 is determined as the second parking space corner point corrected relative to the first parking space corner point P1, and the obstacle point PN2 is determined as the second parking space corner point corrected relative to the first parking space corner point P2.

[0101] S207: Fitting a parking space to be output according to the second parking space corner point and the parking space type.

[0102] In this embodiment, the parking space to be output can be understood as a parking space formed by fitting.

[0103] Specifically, the position coordinates of the corner point of the second parking space are filled into the corner point position of the contour corresponding to the parking space type, and the parking space to be output is formed by fitting.

[0104] S208: Determine whether the parking space to be output meets the parking space parking condition. If so, determine the parking space to be output as the target parking space.

[0105] In this embodiment, the parking space parking condition can be understood as a condition for determining whether the vehicle can be safely parked in the parking space to be output.

[0106] Specifically, the vehicle parameter information (such as vehicle width and vehicle length) of the vehicle itself is compared with the fitted parking space to be output. If the vehicle width is greater than the width of the parking space to be output and / or the vehicle length is greater than the length of the parking space to be output, it is determined that the parking space parking condition is not met at present. At this time, no reaction is made, the vehicle continues to move forward, and continues to collect new environmental images, track information and ultrasonic information to determine the next parking space; if the vehicle width is less than the width of the parking space to be output and the vehicle length is less than the length of the parking space to be output, it is determined that the parking space parking condition is met at present, and the parking space to be output is determined as the target parking space.

[0107] The present embodiment provides a parking space determination method, which obtains an environmental image, track information and ultrasonic information of a vehicle; performs semantic segmentation processing on the environmental image to obtain an image to be processed containing semantic information; traverses the semantic information of the image to be processed, determines the current target obstacle point of the vehicle according to the semantic information, and adds the current target obstacle point and the first coordinate information associated with the current target obstacle point to an obstacle point set; tracks the obstacle points according to the track information, determines the second coordinate information of each historical target obstacle point in the obstacle point set, and updates the obstacle point set according to the second coordinate information; determines a first parking space corner point and a parking space type according to the ultrasonic information; corrects the first parking space corner point according to the obstacle point set to obtain a second parking space corner point; fits a parking space to be output according to the second parking space corner point and the parking space type; determines whether the parking space to be output meets the parking space parking condition, and if so, determines the parking space to be output as the target parking space. The above technical solution introduces image semantic segmentation technology, updates the obstacle point set in real time based on visual images and track information, and corrects the obstacle edge corner point positions obtained by ultrasonic detection based on the obstacle point set to obtain more accurate parking space corner points, thereby achieving more accurate fitting of the target parking space and improving the accuracy of parking space fitting construction, thereby ensuring the safety and success rate of subsequent automatic parking and improving user experience.

[0108] Embodiment 3

[0109] Figure 6 Schematic diagram of the structure of a parking space determination device provided according to Embodiment 3 of the present invention. Figure 6 As shown, the device comprises:

[0110] The information acquisition module 31 is used to acquire the vehicle's environmental image, track information and ultrasonic information;

[0111] An image processing module 32 is used to perform semantic segmentation processing on the environment image to obtain an image to be processed containing semantic information;

[0112] An obstacle point determination module 33, used to determine obstacle points according to the semantic information and the track information to obtain an obstacle point set;

[0113] The parking space determination module 34 is used to perform parking space fitting calculation according to the obstacle point set and the ultrasonic information to obtain a target parking space.

[0114] The parking space determination device provided in this embodiment combines visual images and ultrasonic information, determines obstacle points through images to be processed, tracks obstacle points through track information, obtains an obstacle point set, and corrects the parking space corner points corresponding to the ultrasonic information based on the obtained obstacle point set, thereby achieving more accurate parking space corner point determination and fitting a more accurate target parking space, solving the problem of low accuracy of parking space construction relying solely on ultrasonic waves, improving the accuracy of parking space fitting construction, and thus ensuring the smooth progress of subsequent automatic parking and improving user experience.

[0115] Optionally, the obstacle point determination module 33 includes:

[0116] a current obstacle point adding unit, configured to traverse the semantic information of the image to be processed, determine the current target obstacle point of the vehicle according to the semantic information, and add the current target obstacle point and the first coordinate information associated with the current target obstacle point to the obstacle point set;

[0117] The historical obstacle point updating unit is used to track the obstacle points according to the track information, determine the second coordinate information of each historical target obstacle point in the obstacle point set, and update the obstacle point set according to the second coordinate information.

[0118] Optionally, add a unit to the current obstacle point, specifically used for:

[0119] Establishing an image coordinate system relative to the image to be processed and a vehicle coordinate system relative to the vehicle;

[0120] In the image coordinate system, identifying the coordinates of each obstacle point of the obstacle corresponding to the semantic information, and determining the current target obstacle point according to the distance between each obstacle point coordinate and the target position coordinate of the target position on the vehicle;

[0121] Converting the first position coordinates of the current target obstacle point in the image coordinate system into second position coordinates in the vehicle coordinate system, and determining the second position coordinates and the track information of the current frame as first coordinate information;

[0122] The current target obstacle point and the first coordinate information are associated and added to the obstacle point set.

[0123] Optional, historical obstacle point update unit, specifically used for:

[0124] For each historical target obstacle point in the obstacle point set, determining the position change information and the angle change information of the vehicle according to the track information of the current frame and the track information corresponding to the historical target obstacle point;

[0125] The position coordinates are corrected according to the track information, the position change information and the angle change information of the current frame to determine the second coordinate information of the historical target obstacle point.

[0126] Optionally, the parking space determination module 34 includes:

[0127] A corner point determination unit, configured to determine a first parking space corner point and a parking space type according to the ultrasonic information;

[0128] A corner point correction unit, configured to correct the first parking space corner point according to the obstacle point set to obtain a second parking space corner point;

[0129] A parking space fitting unit, configured to fit a parking space to be output according to the second parking space corner point and the parking space type;

[0130] The parking space determination unit is used to determine whether the parking space to be output meets the parking space parking condition, and if so, determine the parking space to be output as the target parking space.

[0131] Optional, corner correction unit, specifically used for:

[0132] Determine an expansion area of ​​the first parking space corner point according to a sum and a difference between the position coordinates of the first parking space corner point and a set outward expansion threshold;

[0133] An obstacle point in the obstacle point set whose distance from the extended area satisfies a corner point correction condition is determined as a second parking space corner point.

[0134] Optionally, the device further includes: a parking module, configured to control the vehicle to park in the target parking space.

[0135] Optionally, the semantic information includes at least one of parking space lines, lane lines, drivable areas and non-drivable areas, and the non-drivable areas include at least areas where obstacles are located and areas where the vehicle is currently located.

[0136] The parking space determination device provided in the embodiment of the present invention can execute the parking space determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0137] Embodiment 4

[0138] Figure 74 is a schematic diagram of a vehicle 40 provided according to Embodiment 4 of the present invention. The vehicle is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The vehicle may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0139] like Figure 7 As shown, the vehicle 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the vehicle 40 can also be stored. The processor 41, ROM 42 and RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0140] A number of components in the vehicle 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the vehicle 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0141] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as a parking space determination method.

[0142] In some embodiments, the parking space determination method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the vehicle 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the parking space determination method described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to execute the parking space determination method in any other appropriate manner (e.g., by means of firmware).

[0143] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0145] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] To provide interaction with a user, the systems and techniques described herein can be implemented on a vehicle having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the vehicle. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0147] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0148] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0149] In one embodiment, the embodiment of the present invention further includes a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the parking space determination method of any embodiment of the present invention is implemented.

[0150] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0151] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0152] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A parking space determination method, characterized in that: include: Obtain environmental images, track information and ultrasonic information of the vehicle; Performing semantic segmentation processing on the environment image to obtain an image to be processed containing semantic information; Determine obstacle points according to the semantic information and the track information to obtain an obstacle point set; A parking space fitting calculation is performed according to the obstacle point set and the ultrasonic information to obtain a target parking space.

2. The method according to claim 1, characterized in that The step of determining obstacle points according to the semantic information and the track information to obtain an obstacle point set includes: Traversing the semantic information of the image to be processed, determining the current target obstacle point of the vehicle according to the semantic information, and adding the current target obstacle point and the first coordinate information associated with the current target obstacle point to the obstacle point set; Obstacle point tracking is performed according to the track information, second coordinate information of each historical target obstacle point in the obstacle point set is determined, and the obstacle point set is updated according to the second coordinate information.

3. The method according to claim 2, characterized in that The determining the current target obstacle point of the vehicle according to the semantic information, and adding the current target obstacle point and the first coordinate information associated with the current target obstacle point to the obstacle point set, includes: Establishing an image coordinate system relative to the image to be processed and a vehicle coordinate system relative to the vehicle; In the image coordinate system, identifying the coordinates of each obstacle point of the obstacle corresponding to the semantic information, and determining the current target obstacle point according to the distance between each obstacle point coordinate and the target position coordinate of the target position on the vehicle; Converting the first position coordinates of the current target obstacle point in the image coordinate system into second position coordinates in the vehicle coordinate system, and determining the second position coordinates and the track information of the current frame as first coordinate information; The current target obstacle point and the first coordinate information are associated and added to the obstacle point set.

4. The method according to claim 2, characterized in that: Tracking the obstacle points according to the track information to determine the second coordinate information of each historical target obstacle point in the obstacle point set includes: For each historical target obstacle point in the obstacle point set, determining the position change information and the angle change information of the vehicle according to the track information of the current frame and the track information corresponding to the historical target obstacle point; The position coordinates are corrected according to the track information, the position change information and the angle change information of the current frame to determine the second coordinate information of the historical target obstacle point.

5. The method according to claim 1, characterized in that The performing parking space fitting calculation according to the obstacle point set and the ultrasonic information to obtain the target parking space includes: Determine a first parking space corner point and a parking space type according to the ultrasonic information; Correcting the first parking space corner point according to the obstacle point set to obtain a second parking space corner point; According to the second parking space corner point and the parking space type, fitting is performed to form a parking space to be output; It is determined whether the parking space to be output meets the parking space parking condition. If so, the parking space to be output is determined as the target parking space.

6. The method according to claim 5, characterized in that The step of correcting the first parking space corner point according to the obstacle point set to obtain a second parking space corner point includes: Determine an expansion area of ​​the first parking space corner point according to a sum and a difference between the position coordinates of the first parking space corner point and a set outward expansion threshold; An obstacle point in the obstacle point set whose distance from the extended area satisfies a corner point correction condition is determined as a second parking space corner point.

7. The method according to claim 1, characterized in that Also includes: The vehicle is controlled to park in the target parking space.

8. The method according to claim 1, characterized in that The semantic information includes at least one of parking space lines, lane lines, drivable areas and non-drivable areas, and the non-drivable areas at least include areas where obstacles are located and areas where the vehicle is currently located.

9. A parking space determination device, characterized in that: include: An information acquisition module, used to acquire environmental images, track information and ultrasonic information of the vehicle; An image processing module is used to perform semantic segmentation processing on the environment image to obtain an image to be processed containing semantic information; An obstacle point determination module, used to determine obstacle points according to the semantic information and the track information to obtain an obstacle point set; The parking space determination module is used to perform parking space fitting calculation according to the obstacle point set and the ultrasonic information to obtain a target parking space.

10. A vehicle, characterized in that: The vehicle comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the parking space determination method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the parking space determination method according to any one of claims 1 to 8 when executed.

12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the parking space determination method according to any one of claims 1 to 8 is implemented.

Images