Method for determining ground contact box of obstacle, and controller, vehicle, storage medium and program

Abstract

A method for determining a ground contact box of an obstacle, and a controller, a vehicle, a storage medium and a program. The method comprises: determining ground contact point information of a target obstacle from an image acquired on the basis of a vehicle-mounted photographic apparatus in a vehicle; and on the basis of the ground contact point information, determining a closed ground contact box of the target obstacle in an AVM image.

Description

Obstacle ground box determination method, controller, vehicle, storage medium and program

[0001] Cross Reference to Related Applications

[0002] The present disclosure claims priority to a Chinese patent application No. 202410736596.5, filed on June 7, 2024, and entitled “Obstacle ground box determination method, controller, vehicle, storage medium and program”, the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to the field of vehicles, and in particular, to an obstacle ground box determination method, controller, vehicle, storage medium and program. BACKGROUND

[0004] In the related art, an image obtained by a vehicle-mounted photographing device can be mapped into an AVM map, thereby assisting a driver in driving a vehicle.

[0005] However, since a field of view of the vehicle-mounted photographing device is large and a display size of the AVM map is limited, only part of display content in the image can be mapped into the AVM map when the image obtained by the vehicle-mounted photographing device is mapped into the AVM map. Thus, there is a problem that an obstacle in the image is not fully displayed in the AVM map, and further, ground information of the obstacle not fully displayed in the AVM map cannot be determined, which can easily cause driving safety.

[0006] SUMMARY

[0007] An object of the present disclosure is to provide an obstacle ground box determination method, controller, vehicle, storage medium and program to solve the above technical problems.

[0008] To achieve the above object, a first aspect of the present disclosure provides an obstacle ground box determination method, the method comprising:

[0009] determining, in an image obtained by a vehicle-mounted photographing device of a vehicle, ground point information of a target obstacle, the target obstacle being an obstacle not fully displayed in an AVM map of the vehicle after the image is mapped into the AVM map;

[0010] determining, according to the ground point information, a closed ground box of the target obstacle in the AVM map.

[0011] A second aspect of the present disclosure provides a controller, the controller comprising:

[0012] a processor;

[0013] a memory for storing processor-executable instructions;

[0014] The processor is configured to perform the steps of any of the methods of the first aspect.

[0015] The third aspect of the present disclosure provides a vehicle comprising the controller of the second aspect, or

[0016] The vehicle comprises a processor and a memory for storing processor-executable instructions, wherein the processor is configured to perform the steps of any of the methods of the first aspect.

[0017] The fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of any of the methods of the first aspect.

[0018] The fifth aspect of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of any of the methods of the first aspect.

[0019] Through the above technical solution, the grounding point information of the target obstacle can be determined based on the image obtained by the vehicle-mounted shooting device, and the closed grounding frame of the target obstacle in the AVM map can be determined according to the grounding point information, wherein the target obstacle is an obstacle that is not fully displayed in the AVM map. Thus, the closed grounding frame of the displayed part of the target obstacle in the AVM map can be determined based on the image obtained by the vehicle-mounted shooting device. That is to say, the present disclosure can determine the grounding information of the obstacle that is not fully displayed in the AVM map, so that the grounding information in the AVM map is more abundant and complete, and the driving safety and / or parking safety is improved.

[0020] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and / or additional aspects and advantages of the present disclosure will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0022] FIG. 1 is a contrast schematic diagram of a target obstacle before and after mapping according to an exemplary embodiment of the present disclosure;

[0023] FIG. 2 is a flowchart of a method for determining an obstacle grounding frame according to an exemplary embodiment of the present disclosure;

[0024] FIG. 3 is a determination schematic diagram of a valid area and an invalid area according to an exemplary embodiment of the present disclosure;

[0025] FIG. 4 is a grounding frame of a target obstacle displayed in an AVM map based on the method of the present disclosure according to an exemplary embodiment of the present disclosure;

[0026] FIG. 5 is a schematic diagram illustrating a determination of a cross-border edge of a target obstacle, according to an example embodiment of the present disclosure;

[0027] FIG. 6 is a flowchart illustrating another method of determining an obstacle grounding box, according to an example embodiment of the present disclosure;

[0028] FIG. 7 is a structural block diagram of an apparatus for determining an obstacle grounding box, according to an example embodiment of the present disclosure;

[0029] FIG. 8 is a functional block diagram illustrating a vehicle, according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The specific embodiments of the present disclosure will be described below in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and explanatory and are not intended to limit the present disclosure.

[0031] It should be understood that each step recited in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0032] The term "comprising" and variations thereof as used herein are open-ended, and mean "including but not limited to". The term "based on" means "based, at least in part, on". The term "one embodiment" means "at least one embodiment". The term "another embodiment" means "at least one additional embodiment". The term "some embodiments" means "at least some embodiments". Related definitions are given throughout the detailed description.

[0033] It should be noted that the terms "first", "second", and the like used in the present disclosure are merely used to distinguish different devices, modules or units, and do not imply the order or interdependence of the functions performed by these devices, modules or units.

[0034] It should be noted that the terms "one", "multiple" used in the present disclosure are illustrative and not limiting, and those skilled in the art should understand that "one or more" should be understood unless otherwise explicitly indicated in the context.

[0035] As the background art, due to the large field of view angle of the vehicle-mounted shooting device and the limited display size of the AVM map, only part of the display content in the image obtained by the vehicle-mounted shooting device can be mapped into the AVM map when the image is mapped into the AVM map. Thus, after the image is mapped into the AVM map, the problem of incomplete display of the obstacle in the image in the AVM map exists, and the ground information of the obstacle with incomplete display in the AVM map cannot be determined, so that the ground information of the obstacle in the AVM map is incomplete, and driving safety is easily caused.

[0036] For example, in a parking scenario, the automatic auxiliary system of the vehicle can collect a fisheye image by using a fisheye camera, identify the obstacle in the fisheye image, and map the identified obstacle into an AVM map, so as to plan a path according to the display of the obstacle in the AVM map. However, due to the mapping process, only part of the content in the fisheye image can be mapped into the AVM map, which leads to the problem of incomplete display of the obstacle in the AVM map, as shown in FIG. 1. Thus, the closed ground frame of the obstacle displayed in the AVM map cannot be determined, so that the ground information of the obstacle in the AVM map is incomplete, and the ground area of the obstacle displayed in the AVM map cannot be accurately determined, which leads to the problem of parking safety when the automatic auxiliary system plans a parking path.

[0037] Therefore, the present disclosure provides a method for determining an obstacle ground frame, a controller, a vehicle, a storage medium, and a program to solve the above technical problems.

[0038] The embodiments of the present disclosure are further explained and described below with reference to the accompanying drawings.

[0039] FIG. 2 is a flowchart of a method for determining an obstacle ground frame according to an example embodiment of the present disclosure. Referring to FIG. 2, the method can include the following steps:

[0040] S201: In an image obtained by a vehicle-mounted shooting device of a vehicle, determine the ground point information of a target obstacle, the target obstacle being an obstacle with incomplete display in an AVM map after the image is mapped into the AVM map.

[0041] The target obstacle can be all obstacles with incomplete display in the AVM map, or obstacles with incomplete display in the AVM map and close to the vehicle. The obstacle can be a wheel stop, a cone barrel, a pedestrian, a ground lock, a column, and / or a remaining vehicle, etc., which is determined according to actual conditions, and the embodiments of the present disclosure do not make any limitation thereto.

[0042] The grounding point information is used to represent contact information of the target obstacle with the ground, and can be determined according to actual conditions, and the embodiments of the present disclosure do not make any limitation in this regard. For example, the grounding point information can include a number of grounding points, a position of the grounding points, distribution information of the grounding points, a number of grounding lines, a connection mode between the grounding lines, or a position of the grounding lines, and the like.

[0043] The image obtained by the vehicle-mounted shooting device can be one or multiple, and the embodiments of the present disclosure do not make any limitation in this regard. In order to more accurately determine the grounding point information of the target obstacle, the image obtained by the vehicle-mounted shooting device can be multiple, and the image contents corresponding to each image are different. For example, the image can be collected by the fisheye camera arranged at the front, rear, left front, left rear, right front and / or right rear of the vehicle. That is, in a possible manner, the image can be obtained by collecting the image around the vehicle through the fisheye camera arranged at the preset position of the vehicle, so that the grounding point information of the target obstacle can be accurately determined according to the images at different positions.

[0044] After the image is obtained by the vehicle-mounted shooting device, the image can be analyzed and processed to obtain the grounding point information of the target obstacle.

[0045] In a possible manner, determining the grounding point information of the target obstacle in the image can include:

[0046] The image is input into a target detection model to obtain the grounding point information of the target obstacle, wherein the target detection model is trained by taking a sample image as input and taking the grounding point information of a sample obstacle as output.

[0047] The detection model can be a detection model in the related art, or a detection model improved on the basis of the detection model in the related art, and the embodiments of the present disclosure do not make any limitation in this regard. For example, the detection model can be an R-CNN (Region-based Convolutional Neural Networks) series model, an SSD (Single Shot MultiBox Detector) model, or a YOLO (You Only Look Once) series model.

[0048] The labels can be determined according to specific conditions of the sample grounding point information, and the embodiments of the present disclosure do not make any limitation in this regard. For example, when the sample grounding point information includes a grounding box of the target obstacle in contact with the ground and an edge of the target obstacle in contact with the ground, the labels can include a first label and a second label, where the first label is used to indicate the sample grounding box in the sample image, and the second label is used to indicate the sample edge in the sample image. Accordingly, after the sample image is input into the detection model, the predicted grounding box and the predicted edge of the sample image can be obtained, so that the first loss function value can be determined according to the predicted grounding box and the first label, the second loss function value can be determined according to the predicted edge and the second label, and the parameters of the detection model can be updated according to the first loss function value and the second loss function value until a preset number of iterations is reached or the accuracy of the detection model reaches a preset value.

[0049] After the detection model is determined, in order to determine the grounding point information of the target obstacle based on the image based on the detection model, the detection model needs to be trained based on the training data to obtain a target detection model.

[0050] In a possible manner, the target detection model can be obtained by training the detection model in the following manner:

[0051] obtaining a sample image and a label, where the label is used to indicate the grounding point information of a sample obstacle in the sample image; inputting the sample image into the detection model to obtain predicted grounding point information of the sample obstacle, and determining a loss function value according to the predicted grounding point information and the label, and updating the parameters of the detection model according to the loss function value until a preset number of iterations is reached or the accuracy of the detection model reaches a preset value.

[0052] Thus, the grounding point information of the target obstacle can be obtained by the above manner.

[0053] S202: determining a closed grounding box of the target obstacle in the AVM map according to the grounding point information.

[0054] In a possible manner, determining the closed grounding box of the target obstacle in the AVM map according to the grounding point information can include:

[0055] determining a critical grounding point and an effective grounding point of the target obstacle according to the grounding point information, the critical grounding point being a grounding point located at an edge of an effective region, the effective grounding point being a grounding point located in the effective region except the critical grounding point, the effective region being a region in the image in which the image content can be mapped into the AVM map; and determining the closed grounding box of the target obstacle in the AVM map according to the critical grounding point and the effective grounding point.

[0056] It should be understood that the image content in the image is generally expressed in the form of pixels. Thus, the effective area can also refer to an area in the image to which the pixels can be mapped into the AVM graph. That is, the pixels in the area can be displayed in the AVM graph after coordinate transformation. For example, as shown in FIG. 3, after the quadrilateral ABCD in the image is mapped into the AVM graph, only the BE edge, the BC edge and the CF edge are displayed in the AVM graph, and thus the area in which the BE edge, the BC edge and the CF edge are located is the effective area of the image.

[0057] After the effective area is determined, the critical grounding point and the effective grounding point of the target obstacle can be determined based on the effective area and the grounding point information.

[0058] For example, after the grounding point information is obtained, the grounding frame or the grounding line at which the target obstacle contacts the ground can be determined according to the grounding point information, and then the critical grounding point and the effective grounding point of the target obstacle can be determined according to the distribution of the grounding frame or the grounding line in the effective area. Subsequently, the effective grounding frame of the target obstacle in the image is obtained by connecting the critical grounding point and the effective grounding point, and the closed grounding frame of the target obstacle in the AVM graph is obtained by mapping the effective grounding frame into the AVM graph based on the mapping relationship between the image and the AVM graph. That is, in a possible manner, the closed grounding frame of the target obstacle in the AVM graph can be determined according to the critical grounding point and the effective grounding point, which can include:

[0059] connecting the critical grounding point and the effective grounding point to form a starting grounding frame in the image; and mapping the starting grounding frame into the AVM graph according to a preset coordinate mapping relationship to obtain the closed grounding frame of the target obstacle in the AVM graph.

[0060] For example, as shown in FIG. 4, the quadrilateral ABCD in the image is used to represent the grounding frame of the target obstacle, since the effective area displays the BE grounding line, the BC grounding line and the CF grounding line, and the grounding point E and the grounding point F are located at the edges of the effective area, the critical grounding point (the grounding point E and the grounding point F) and the effective grounding point (the grounding point B and the grounding point C) of the target obstacle can be obtained. Then, the grounding point E, the grounding point F, the grounding point B and the grounding point C can be connected to obtain the starting grounding frame BCEF, and then the starting grounding frame BCEF is mapped into the AVM graph based on the mapping relationship between the image and the AVM graph to obtain the closed grounding frame B’C’F’E’ of the target obstacle in the AVM graph.

[0061] In a possible manner, the critical grounding point of the target obstacle can be determined according to the grounding point information, which can include:

[0062] According to the grounding point information, a grounding box of the target obstacle is determined; and according to the intersection line between the invalid region and the valid region and the grounding box, a critical grounding point of the target obstacle is determined, wherein the invalid region is a region in the image in which image content cannot be mapped into the AVM map.

[0063] As described above, the image content in the image is generally expressed in the form of pixels. Thus, the invalid region can also refer to a region in the image in which pixel points cannot be mapped into the AVM map. That is, the pixel points in the region cannot be displayed in the AVM map after coordinate transformation. For example, continuing to refer to FIG. 3, since the quadrilateral ABCD in the image is mapped into the AVM map, the EA edge, the AD edge and the DF edge are not displayed, and thus the region in which the EA edge, the AD edge and the DF edge are located is the invalid region of the image.

[0064] It should be understood that, since the critical grounding point is a grounding point located at the edge of the valid region, the critical grounding points can be connected after the valid region is determined by mapping the pixel points, so as to obtain the intersection line between the valid region and the invalid region. For example, continuing to refer to FIG. 3, after the grounding point E and the grounding point F are obtained, the intersection line between the valid region and the invalid region, that is, the straight line MN, can be obtained by connecting EF and extending to the edge of the image.

[0065] It should be understood that the intersection line between the valid region and the invalid region obtained here is only illustrative and does not limit the scheme. In a possible manner, a coordinate information correspondence table can also be determined in advance according to the mapping relationship between the image and the AVM map, wherein the coordinate information correspondence table is used to indicate whether a pixel point in the image is located in the valid region. Then, target pixel points in the image located in the valid region and close to the pixel points in the invalid region are determined based on the coordinate information correspondence table, and finally the intersection line between the valid region and the invalid region is obtained by connecting the target pixel points.

[0066] After the intersection line between the invalid region and the valid region is determined, the critical grounding point of the target obstacle can be determined according to the distribution of the grounding box of the target obstacle and the intersection line.

[0067] In a possible manner, determining the critical grounding point of the target obstacle according to the intersection line between the invalid region and the valid region and the grounding box can include:

[0068] The intersection point of the intersection line and the grounding box is determined as the critical grounding point of the target obstacle.

[0069] For example, continuing to refer to FIG. 3, if the quadrilateral ABCD represents the grounding box of the target obstacle, the intersection points (the intersection point E and the intersection point F) of the grounding box ABCD and the straight line MN can be determined as the critical grounding points.

[0070] In a possible implementation, determining the critical contact point of the target obstacle according to the contact point information can include:

[0071] According to the contact point information, a cross-border edge of the target obstacle is determined, the cross-border edge being an edge of the target obstacle that is not fully displayed in the AVM map after the image is mapped to the AVM map; and according to the cross-border edge, the critical contact point of the target obstacle is determined.

[0072] In this case, determining the cross-border edge of the target obstacle can include determining an edge (i.e., an edge) of the target obstacle that is in contact with the ground, mapping the edge to the AVM map according to the mapping relationship between the image and the AVM map, and determining whether the edge is a cross-border edge according to the display of the edge in the AVM map. Alternatively, the effective area and the invalid area of the image can be determined in advance according to the mapping relationship between the image and the AVM map; then, after the image is obtained, the contact point of the target obstacle in contact with the ground in the image is determined, and the cross-border edge is determined according to the distribution of the contact point in the effective area and the invalid area.

[0073] For example, as shown in FIG. 5, when the four contact points of the target obstacle are all in the effective area or the four contact points are all in the invalid area, it is determined that there is no cross-border edge in the target obstacle; otherwise, there is a cross-border edge in the target obstacle.

[0074] In a possible implementation, the contact point information includes a first contact point coordinate of an edge of the target obstacle in contact with the ground and an end point of the edge, and determining the cross-border edge of the target obstacle around the vehicle according to the contact point information can include:

[0075] According to the first contact point coordinate, a first identifier for indicating whether the end point of the edge is located in the effective area is determined; and when the first identifiers of the end points corresponding to the edge of the target obstacle in contact with the ground are different, the edge is determined as a cross-border edge.

[0076] For example, the edge of the target obstacle in contact with the ground includes an edge ab, and the edge ab includes an end point a and an end point b; if the first identifier of the end point a and the first identifier of the end point b are different, the edge ab is determined as a cross-border edge. If the first identifier of the end point a and the first identifier of the end point b are the same, the edge ab is determined as a non-cross-border edge.

[0077] In a possible implementation, in order to quickly determine the first identifier of the end point and improve the determination efficiency of the closed contact frame, determining the first identifier for indicating whether the end point of the edge is located in the effective area according to the first contact point coordinate can include:

[0078] According to the first touch point coordinate and a preset coordinate information correspondence table, a first identifier for indicating whether the end point of the edge is located in the effective area is determined, wherein the coordinate information correspondence table is used to indicate whether a pixel point in the image is located in the effective area.

[0079] For example, the touch point coordinate of the end point 1 is (u1, v1), and the pixel point corresponding to (u1, v1) is pixel point 1. If the coordinate information correspondence table indicates that the pixel point 1 is located in the effective area, the first identifier for indicating that the end point 1 is located in the effective area is determined.

[0080] For example, the touch point coordinate of the end point 2 is (u2, v2), and the pixel point corresponding to (u2, v2) is pixel point 2. If the coordinate information correspondence table indicates that the pixel point 2 is not located in the effective area, the first identifier for indicating that the end point 2 is not located in the effective area is determined.

[0081] It should be understood that the first identifier for indicating whether the end point of the edge is located in the effective area is determined according to the first touch point coordinate only as an illustrative description, and does not constitute a limitation on the scheme. In a possible manner, the pixel point corresponding to the first touch point coordinate can be mapped into the AVM graph according to the mapping relationship between the image and the AVM graph, and then the first identifier can be determined according to the display of the pixel point in the AVM graph. For example, if the pixel point can be displayed in the AVM graph, the first identifier for indicating that the end point of the edge is located in the effective area is determined. If the pixel point cannot be displayed in the AVM graph, the first identifier for indicating that the end point of the edge is not located in the effective area is determined. Alternatively, the target coordinate of the pixel point corresponding to the first touch point coordinate in the AVM graph can be determined according to the mapping relationship between the image and the AVM graph, and then the first identifier can be determined according to the target coordinate and the coordinate range of the AVM graph. For example, if the target coordinate is located in the coordinate range of the AVM graph, the first identifier for indicating that the end point of the edge is located in the effective area is determined. If the target coordinate is not located in the coordinate range of the AVM graph, the first identifier for indicating that the end point of the edge is not located in the effective area is determined.

[0082] In a possible manner, the coordinate information correspondence table can be obtained in the following manner:

[0083] A first coordinate of a pixel point in the image is determined, and a second coordinate of the pixel point in the AVM graph is determined based on the first coordinate, a distance corresponding to the pixel point, and a preset coordinate mapping relationship, wherein the coordinate mapping relationship is used to represent the mapping relationship between the coordinate of the pixel point in the image and the coordinate of the pixel point in the AVM graph. According to the image size of the AVM graph and the second coordinate, a target identifier for indicating whether the pixel point corresponding to the second coordinate is located in the effective area is determined. According to the correspondence relationship among the pixel point, the first coordinate, and the target identifier, the coordinate information correspondence table is determined.

[0084] It should be understood that the distance between pixels is determined based on the actual situation, and the embodiments disclosed herein do not impose any restrictions on this.

[0085] Additionally, it should be understood that, based on the image size and second coordinates of the AVM image, determining whether the pixel corresponding to the second coordinate is located within the valid region can be done by: first determining the coordinate range of the AVM image based on its image size; then determining the target identifier based on the second coordinate and the coordinate range of the AVM image.

[0086] For example, if the coordinates of a pixel in the image are (u, v), and the distance to the pixel is K meters, then by using the coordinate mapping relationship and the distance to the pixel, the coordinates of the pixel in the AVM image can be obtained as (X, v). w / K,Y w / K), because the Z of the grounding point w =0, therefore it can be ignored. If the AVM graph has an image length of M and an image width of N, then we can determine the AVM graph based on the image length M, image width N, and (x... w / K,Y w / K), determine (X) w / K,Y w Whether (X) is within the coordinate range of the AVM graph, if (X) w / K,Y w / K) Within the coordinate range of the AVM graph, determine the target identifier used to characterize the pixel point as being located within the effective region; if (X w / K,Y w If / K) is not within the coordinate range of the AVM graph, determine the target identifier used to characterize the pixel that is not located within the valid area.

[0087] Among the possible methods, the coordinate mapping relationship can be obtained in the following ways:

[0088] The camera used to acquire the image is jointly calibrated to obtain the camera intrinsic and extrinsic parameters with the target position as the origin. The first product of the coordinate matrix of the pixel in the image and the preset camera depth is determined, and the second product of the camera intrinsic and extrinsic parameter matrices is determined. The ratio of the first product to the second product is determined as the coordinate mapping relationship. The coordinate matrix is ​​determined according to the coordinate of the pixel, the camera intrinsic matrix is ​​determined according to the camera intrinsic parameters, and the camera extrinsic parameter matrix is ​​determined according to the camera extrinsic parameters.

[0089] The preset camera depth and target position can be set according to actual conditions, and this embodiment does not impose any limitations on them. For example, the target position can be the center of the rear axle of the vehicle.

[0090] After the vehicle rear axle center is determined, the camera can be jointly calibrated with the vehicle rear axle center as the origin to obtain the camera intrinsic parameter and the camera extrinsic parameter, and then the coordinate mapping relationship is determined.

[0091] For example, the coordinate matrix of the pixel point in the image can be set as: The camera intrinsic parameter matrix can be set as: The camera extrinsic parameter matrix can be set as: Therefore, the coordinate matrix of the pixel point in the image in the AVM image can be obtained by the following formula: That is, the coordinate mapping relationship between the coordinate of the pixel point in the image and the coordinate of the pixel point in the AVM image can be obtained by the following formula:

[0092] Wherein, f x , f y , u0 and v0 are camera intrinsic parameters, specifically, f x and f y represent focal length, u0 and v0 represent principal point coordinates, R and T are camera extrinsic parameters, representing camera pose, Z c represents camera depth, u and v represent the coordinates of the pixel point in the image, X w , Y w and Z w represent the coordinates of the pixel point in the AVM image.

[0093] It should be understood that since the AVM image can simultaneously display images of different directions of the vehicle, and the display positions of the images of different directions in the AVM image are different, the coordinate mapping relationship between the images taken by the vehicle-mounted shooting devices of different directions and the AVM image is different. Therefore, when determining the coordinate mapping relationship between the image and the AVM image, the above steps can be performed for each image to obtain the coordinate mapping relationship between each image and the AVM image.

[0094] In a possible manner, determining the critical contact point of the target obstacle according to the cross-border edge can include:

[0095] determining a second identifier for representing whether the contact point in the cross-border edge is located in the effective area; when the second identifiers between the two adjacent contact points in the cross-border edge are different, the corresponding second identifier in the two adjacent contact points is used to represent the target contact point located in the effective area, and the critical contact point of the target obstacle is determined.

[0096] For example, the cross-border edge includes the grounding point a, the grounding point b, the grounding point c, the grounding point d and the grounding point e, and the second identifier of the grounding point a, the grounding point b and the grounding point c is used to represent that the grounding point is located in the effective area, and the second identifier of the grounding point d and the grounding point e is used to represent that the grounding point is not located in the effective area. Since only the second identifier of the grounding point c is different from the second identifier of the grounding point d among the adjacent grounding points, and since the second identifier of the grounding point c is used to represent that the grounding point c is located in the effective area and the second identifier of the grounding point d is used to represent that the grounding point d is not located in the effective area, the grounding point c can be determined as the critical grounding point located at the edge of the effective area in the cross-border edge.

[0097] In a possible manner, when the second identifiers between two adjacent grounding points in the cross-border edge are different, the corresponding second identifier of the two adjacent grounding points is used to represent the target grounding point located in the effective area, and the critical grounding point located at the edge of the effective area in the cross-border edge is determined.

[0098] The corresponding any one end point of the cross-border edge is taken as an initial first to-be-processed grounding point, and the following process is executed in a loop: when the first target grounding point adjacent to the first to-be-processed grounding point in the cross-border edge has the same second identifier as the first to-be-processed grounding point, the first target grounding point is taken as a new first to-be-processed grounding point, until the first target grounding point has a different second identifier from the first to-be-processed grounding point, and when the second identifiers of the first target grounding point and the first to-be-processed grounding point are different, the corresponding second identifier of the first target grounding point and the first to-be-processed grounding point is used to represent the grounding point located in the effective area, and the critical grounding point located at the edge of the effective area in the cross-border edge is determined.

[0099] For example, the cross-border edge includes the grounding point a, the grounding point b, the grounding point c, the grounding point d and the grounding point e arranged in sequence. In determining the critical grounding point, the grounding point a can be taken as an initial first to-be-processed grounding point, and it is determined whether the second identifier between the grounding point a and the grounding point b is the same; if the second identifier between the grounding point a and the grounding point b is different, the critical grounding point is determined according to the second identifier of the grounding point a and the second identifier of the grounding point b. Specifically, if the second identifier of the grounding point a represents that the grounding point a is located in the effective area, the grounding point a is determined as the critical grounding point; if the second identifier of the grounding point a represents that the grounding point a is not located in the effective area, the grounding point b is determined as the critical grounding point. If the second identifier between the grounding point a and the grounding point b is the same, the grounding point b is taken as a new first to-be-processed grounding point, and the above steps are repeatedly executed until the critical grounding point located at the edge of the effective area in the cross-border edge is obtained.

[0100] In a possible manner, the second identifier used to represent whether the grounding point in the cross-border edge is located in the effective area can include:

[0101] determining a second contact point coordinate of the contact point in the cross-border edge; and determining, according to the second contact point coordinate and a preset coordinate information correspondence table, a second identifier for representing whether the contact point in the cross-border edge is located in the effective region, wherein the coordinate information correspondence table is used to indicate whether a pixel point in the image is located in the effective region.

[0102] The manner of determining, according to the second contact point coordinate and the coordinate information correspondence table, the second identifier for representing whether the contact point in the cross-border edge is located in the effective region can refer to the manner of determining, according to the first contact point coordinate and the coordinate information correspondence table, the first identifier for representing whether the end point of the edge is located in the effective region, which will not be described herein again.

[0103] In a possible manner, the second contact point coordinate of the contact point in the cross-border edge can be determined in the following manner:

[0104] The following process is executed in a loop manner: taking any one end point of the cross-border edge as an initial second to-be-processed contact point, determining, according to the contact point coordinate of the second to-be-processed contact point, a preset horizontal axis interval value and a preset vertical axis interval value, a second contact point coordinate of a second target contact point adjacent to the second to-be-processed contact point in the cross-border edge, taking the second target contact point as a new second to-be-processed contact point, and repeating the above process until the adjacent contact point of the other end point of the cross-border edge is taken as the second to-be-processed contact point, wherein the horizontal axis interval value is used to represent a horizontal axis coordinate interval value between adjacent contact points in the cross-border edge, and the vertical axis interval value is used to represent a vertical axis coordinate interval value between adjacent contact points in the cross-border edge.

[0105] For example, the cross-border edge includes sequentially arranged contact points a, b, c, d and e. When determining the contact point coordinates of the contact points b, c and d, the contact point a can be taken as an initial second to-be-processed contact point, and the contact point coordinate (u a , v a ) of the contact point b is determined according to the contact point coordinate (u b , v b ) of the contact point a, a preset horizontal axis interval value dx and a preset vertical axis interval value dy, that is, u b =u a +dx, v b =v a +dy. Then the contact point b is taken as a new second to-be-processed contact point, and the above steps are repeated until the contact point d is taken as the second to-be-processed contact point.

[0106] In a possible manner, the horizontal axis interval value and the vertical axis interval value can be determined in the following manner:

[0107] According to the ground contact coordinates of each end point corresponding to the cross-border edge, a horizontal axis coordinate difference value and a vertical axis coordinate difference value between each end point corresponding to the cross-border edge are determined; when the absolute value of the horizontal axis coordinate difference value is greater than the absolute value of the vertical axis coordinate difference value, the preset coordinate interval value is determined as the horizontal axis interval value, and the vertical axis interval value is determined according to the coordinate interval value and a preset calculation formula; when the absolute value of the horizontal axis coordinate difference value is greater than the absolute value of the vertical axis coordinate difference value, the preset coordinate interval value is determined as the vertical axis interval value, and the horizontal axis interval value is determined according to the coordinate interval value and the preset calculation formula.

[0108] It should be understood that the coordinate interval value can be set according to actual conditions, and the embodiments of the present disclosure do not make any limitation on this. For example, the coordinate interval value can be set to 1 nm, 5 nm or 1 mm, etc.

[0109] In addition, it should be understood that the preset calculation formula can be determined according to actual conditions, and the embodiments of the present disclosure do not make any limitation on this. For example, the preset calculation formula can be determined according to the ground contact coordinates of the two end points of the cross-border edge and / or the shape of the edge.

[0110] For example, when the shape of the edge is a straight line, the initial calculation formula can be set as y=kx+b, and then the slope k and the constant b are determined according to the two end points of the edge. Then the preset calculation formula is determined according to the slope k and the constant b. At the same time, since the pixel points are all integers, and the ground contact points in the cross-border edge to be determined are all the points closest to the straight line, the constant b can be set to 0, so as to determine the pixel point coordinates along the straight line, that is, the preset calculation formula can be set as y=kx at this time. If the cross-border edge includes end point 1 and end point 2, wherein the ground contact coordinates of end point 1 are (u1, v1), and the ground contact coordinates of end point 2 are (u2, v2). The horizontal axis coordinate difference value Δx between end point 1 and end point 2 is Δx=|u1-u2|, and the vertical axis coordinate difference value Δy between end point 1 and end point 2 is Δy=|v1-v2|. If Δx>Δy, it means that the maximum difference value of the horizontal axis is greater than that of the vertical axis, and the horizontal axis direction is the main direction of the step, so the preset coordinate interval value can be determined as the horizontal axis interval value, for example, 1 is determined as the horizontal axis interval value. At the same time, the vertical axis interval value is determined according to the horizontal axis interval value and the preset calculation formula, that is, k is determined as the vertical axis interval value. If Δy>Δx, it means that the maximum difference value of the vertical axis is greater than that of the horizontal axis, and the vertical axis direction is the main direction of the step, so the preset coordinate interval value can be determined as the vertical axis interval value, for example, 1 is determined as the vertical axis interval value. At the same time, the horizontal axis interval value is determined according to the vertical axis interval value and the preset calculation formula, that is, 1 / k is determined as the horizontal axis interval value.

[0111] In order to facilitate understanding of the obstacle ground contact frame determination method of the present disclosure, an implementation manner of the present disclosure is described below in combination with each step:

[0112] For example, as shown in FIG. 6, first, an image collected by a vehicle-mounted shooting device can be obtained, and the image is input into a target detection model to obtain the edge of the target obstacle in contact with the ground and the first grounding point coordinates of the endpoints of the edge. Second, the first grounding point coordinates of the endpoints and a preset coordinate information correspondence table are matched to determine the first identifier for representing whether the endpoints are located in the effective area, and the first identifier between the endpoints is matched to determine the cross-border edge. Third, the first grounding point coordinates of the two endpoints of the cross-border edge are matched to determine the second grounding point coordinates of each grounding point in the cross-border edge, and the second grounding point coordinates are matched with the preset coordinate information correspondence table to determine the second identifier for representing whether the grounding points in the cross-border edge are located in the effective area. Fourth, the second identifiers of adjacent grounding points in the cross-border edge are matched to determine the critical grounding points, and the second identifiers of each grounding point in the cross-border edge are matched to determine the effective grounding points. Finally, the initial grounding box is obtained by connecting the critical grounding points and the effective grounding points, and the initial grounding box is mapped into the AVM map based on the mapping relationship between the image and the AVM map to obtain the closed grounding box of the target obstacle in the AVM map.

[0113] In summary, through the above technical solutions, the grounding point information of the target obstacle can be determined, and the closed grounding box of the target obstacle in the AVM map can be determined based on the grounding point information, where the target obstacle is an obstacle that is not fully displayed in the AVM map. Therefore, the closed grounding box of the displayed part of the target obstacle in the AVM map can be determined based on the image obtained by the vehicle-mounted shooting device. That is, the present disclosure can determine the grounding information of the obstacle that is not fully displayed in the AVM map, so that the grounding information in the AVM map is more abundant and complete, and the driving safety and / or parking safety is improved.

[0114] Based on the same concept, the present disclosure also provides an obstacle grounding box determination device, as shown in FIG. 7, the obstacle grounding box determination device 700 can include:

[0115] A first determination module 701 is configured to determine the grounding point information of a target obstacle in an image obtained by a vehicle-mounted shooting device of a vehicle, where the target obstacle is an obstacle that is not fully displayed in an AVM map of the vehicle after the image is mapped to the AVM map of the vehicle.

[0116] A second determination module 702 is configured to determine a closed grounding box of the target obstacle in the AVM map based on the grounding point information.

[0117] By means of the above-obtained obstacle grounding box determination apparatus 700, the grounding point information of the target obstacle can be determined based on the image obtained by the vehicle-mounted photographing apparatus, and the closed grounding box of the target obstacle in the AVM map can be determined according to the grounding point information, where the target obstacle is an obstacle that is not fully displayed in the AVM map. Thus, the closed grounding box of the displayed part of the target obstacle in the AVM map can be determined based on the image obtained by the vehicle-mounted photographing apparatus. That is to say, the present disclosure can determine the grounding information of the obstacle that is not fully displayed in the AVM map, so that the grounding information in the AVM map is more abundant and complete, and the driving safety and / or parking safety is improved.

[0118] In a possible implementation, the second determination module 702 can include:

[0119] A first determination sub-module, configured to determine, according to the grounding point information, a critical grounding point and an effective grounding point of the target obstacle, the critical grounding point being a grounding point located at an edge of an effective region, and the effective grounding point being a grounding point located in the effective region except the critical grounding point, and the effective region being a region in the image in which the image content can be mapped to the AVM map of the vehicle;

[0120] A second determination sub-module, configured to determine, according to the critical grounding point and the effective grounding point, the closed grounding box of the target obstacle in the AVM map.

[0121] In a possible implementation, the first determination sub-module can include:

[0122] A first determination unit, configured to determine, according to the grounding point information, the grounding box of the target obstacle;

[0123] A second determination unit, configured to determine, according to a boundary line between the invalid region and the effective region and the grounding box, the critical grounding point of the target obstacle, where the invalid region is a region in the image in which the image content cannot be mapped to the AVM map.

[0124] In a possible implementation, the second determination unit can be configured to determine, as the critical grounding point of the target obstacle, an intersection point of the boundary line and the grounding box.

[0125] In a possible implementation, the first determination sub-module can include:

[0126] A third determination unit, configured to determine, according to the grounding point information, a cross-border edge of the target obstacle, the cross-border edge being an edge of the target obstacle that is not fully displayed in the AVM map after the image is mapped to the AVM map;

[0127] A fourth determination unit, configured to determine, according to the cross-border edge, the critical grounding point of the target obstacle.

[0128] In a possible implementation, the ground contact point information comprises first ground contact point coordinates of an edge of the target obstacle in contact with the ground and end points of the edge, and correspondingly, the third determining unit can comprise:

[0129] The first determining subunit is configured to determine, according to the first ground contact point coordinates, a first identifier used to represent whether the end points of the edge are located in the effective area, the effective area being an area in the image to which a pixel point can be mapped in the AVM map.

[0130] The second determining subunit is configured to determine, when the first identifiers between the respective end points corresponding to the edge of the target obstacle in contact with the ground are different, that the edge is a cross-border edge of the target obstacle.

[0131] In a possible implementation, the first determining subunit can be configured to determine, according to the first ground contact point coordinates and a preset coordinate information reference table, a first identifier used to represent whether the end points of the edge are located in the effective area, wherein the coordinate information reference table is used to indicate whether a pixel point in the image is located in the effective area.

[0132] In a possible implementation, the fourth determining unit can comprise:

[0133] The third determining subunit is configured to determine a second identifier used to represent whether a ground contact point in the cross-border edge is located in the effective area.

[0134] The fourth determining subunit is configured to determine, when the second identifiers between two adjacent ground contact points in the cross-border edge are different, that the second identifiers corresponding to the two adjacent ground contact points are used to represent target ground contact points located in the effective area, and to determine the target ground contact points as critical ground contact points of the target obstacle.

[0135] In a possible implementation, the fourth determining subunit can be configured to take any one of the end points corresponding to the cross-border edge as an initial first ground contact point to be processed, and to perform the following process in a loop: when a first target ground contact point adjacent to the first ground contact point to be processed in the cross-border edge has the same second identifier as the first ground contact point to be processed, taking the first target ground contact point as a new first ground contact point to be processed, until the second identifier between the first target ground contact point and the first ground contact point to be processed is different, and when the second identifiers corresponding to the first target ground contact point and the first ground contact point to be processed are different, taking the second identifiers to represent ground contact points located in the effective area, and determining the ground contact points as critical ground contact points of the cross-border edge located in the edge of the effective area.

[0136] In a possible implementation, the third determining subunit can comprise:

[0137] The first determining component is configured to determine second ground contact point coordinates of the ground contact points in the cross-border edge.

[0138] a second determining component configured to determine a second identifier for indicating whether the contact point in the cross-border edge is located in the effective region according to the second contact point coordinate and a preset coordinate information correspondence table, wherein the coordinate information correspondence table is configured to indicate whether a pixel point in the image is located in the effective region.

[0139] In a possible implementation, the second determining component can be configured to: take any one of the end points corresponding to the cross-border edge as an initial second contact point to be processed, and perform the following process in a loop: determine a second contact point coordinate of a second target contact point adjacent to the second contact point to be processed according to the contact point coordinate of the second contact point to be processed, a preset horizontal axis interval value and a preset vertical axis interval value, take the second target contact point as a new second contact point to be processed, and repeat the process until the adjacent contact point of the other end point corresponding to the cross-border edge is taken as the second contact point to be processed, wherein the horizontal axis interval value is configured to indicate a horizontal axis coordinate interval value between adjacent contact points in the cross-border edge, and the vertical axis interval value is configured to indicate a vertical axis coordinate interval value between adjacent contact points in the cross-border edge.

[0140] In a possible implementation, the horizontal axis interval value and the vertical axis interval value can be determined in the following manner:

[0141] determine a horizontal axis coordinate difference value and a vertical axis coordinate difference value between the end points corresponding to the cross-border edge according to the contact point coordinates of the end points corresponding to the cross-border edge;

[0142] when the absolute value of the horizontal axis coordinate difference value is greater than the absolute value of the vertical axis coordinate difference value, determine the preset coordinate interval value as the horizontal axis interval value, and determine the vertical axis interval value according to the coordinate interval value and a preset calculation formula;

[0143] when the absolute value of the horizontal axis coordinate difference value is greater than the absolute value of the vertical axis coordinate difference value, determine the preset coordinate interval value as the vertical axis interval value, and determine the horizontal axis interval value according to the coordinate interval value and a preset calculation formula.

[0144] In a possible implementation, the coordinate information correspondence table can be obtained in the following manner:

[0145] determine a first coordinate of a pixel point in the image, and determine a second coordinate of the pixel point in the AVM graph based on the first coordinate, a distance corresponding to the pixel point and a preset coordinate mapping relationship, wherein the coordinate mapping relationship is configured to indicate a mapping relationship between the coordinate of the pixel point in the image and the coordinate of the pixel point in the AVM graph;

[0146] determine a target identifier for indicating whether the pixel point corresponding to the second coordinate is located in the effective region according to the image size of the AVM graph and the second coordinate;

[0147] determine the coordinate information correspondence table according to a corresponding relationship among the pixel point, the first coordinate and the target identifier.

[0148] In a possible manner, the coordinate mapping relationship can be obtained in the following manner:

[0149] jointly calibrate the camera used to acquire the image to obtain camera intrinsic parameters and camera extrinsic parameters with the target position as the origin;

[0150] determine a first product of a coordinate matrix of a pixel point in the image and a preset camera depth, and determine a second product of a camera intrinsic parameter matrix and a camera extrinsic parameter matrix, and determine a ratio of the first product to the second product as the coordinate mapping relationship, wherein the coordinate matrix is determined according to the coordinates of the pixel point, the camera intrinsic parameter matrix is determined according to the camera intrinsic parameters, and the camera extrinsic parameter matrix is determined according to the camera extrinsic parameters.

[0151] In a possible manner, the second determination sub-module can include:

[0152] a connection unit, configured to connect the critical grounding point and the effective grounding point to form a starting grounding frame in the image;

[0153] a mapping unit, configured to map the starting grounding frame to an AVM graph according to a preset coordinate mapping relationship to obtain a closed grounding frame of the target obstacle in the AVM graph.

[0154] In a possible manner, the first determination module 701 can be configured to input the image into a target detection model to obtain the grounding point information of the target obstacle, wherein the target detection model is trained by taking a sample image as input and taking grounding point information of a sample obstacle as output.

[0155] In a possible manner, the image can be acquired in the following manner:

[0156] by an fisheye camera arranged at a preset position of the vehicle.

[0157] Based on the same concept, the embodiments of the present disclosure further provide a controller, which can include:

[0158] a processor;

[0159] a memory for storing processor-executable instructions;

[0160] The processor can be configured to perform the steps of any of the above obstacle grounding frame determination methods.

[0161] Based on the same concept, the embodiments of the present disclosure further provide a vehicle, which includes the above controller, or

[0162] The vehicle includes a processor and a memory for storing processor-executable instructions, wherein the processor can be configured to perform the steps of any of the above obstacle grounding frame determination methods.

[0163] Based on the same idea, the embodiments of the present disclosure further provide a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of any of the above obstacle grounding box determination methods.

[0164] Based on the same idea, the embodiments of the present disclosure further provide a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of any of the above obstacle grounding box determination methods.

[0165] Please refer to FIG. 8, which is a functional block diagram of a vehicle according to an exemplary embodiment. The vehicle 800 can include various subsystems, such as an infotainment system 810, a perception system 820, a decision control system 830, a drive system 840, and a computing platform 850. The vehicle 800 can include more or fewer subsystems, and each subsystem can include multiple components. In addition, each subsystem and each component of the vehicle 800 can be interconnected by wired or wireless means.

[0166] In some embodiments, the infotainment system 810 can include a communication system, an entertainment system, a navigation system, and the like.

[0167] The perception system 820 can include several sensors for sensing information about the environment around the vehicle 800. For example, the perception system 820 can include a global positioning system (which can be a GPS system, a Beidou system, or other positioning systems), an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0168] The decision control system 830 can include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0169] The drive system 840 can include components that provide power motion for the vehicle 800. In one embodiment, the drive system 840 can include an engine, an energy source, a transmission system, and wheels. The engine can be one or a combination of an internal combustion engine, an electric motor, an air compression engine. The engine can convert the energy provided by the energy source into mechanical energy.

[0170] Part or all of the functions of the vehicle 800 are controlled by the computing platform 850. The computing platform 850 can include at least one processor 851 and a memory 852, and the processor 851 can execute instructions 853 stored in the memory 852.

[0171] The processor 851 can be any conventional processor, such as a commercially available CPU. The processor can also include a Graphics Process Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0172] The memory 852 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0173] In addition to the instructions 853, the memory 852 can also store data, such as road maps, route information, the position, direction, speed, and the like of the vehicle. The data stored in the memory 852 can be used by the computing platform 850.

[0174] In the embodiments of the present disclosure, the processor 851 can execute the instructions 853 to complete all or part of the steps of the above-described obstacle grounding box determination method.

[0175] The preferred embodiments of the present disclosure are described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details in the above-described embodiments. Within the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

[0176] In addition, it should be noted that each specific technical feature described in the above specific embodiments can be combined in any appropriate manner without contradiction, and in order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

Claims

A method for determining an obstacle grounding box, characterized in that, The method comprises: In an image acquired by a vehicle-based on-vehicle shooting device, determining ground contact information of a target obstacle, the target obstacle being an obstacle that is not fully displayed in an AVM map after the image is mapped to the AVM map of the vehicle; According to the ground contact information, determining a closed ground contact box of the target obstacle in the AVM map. The obstacle ground box determination method according to claim 1, wherein The step of determining the closed ground contact box of the target obstacle in the AVM map according to the ground contact information comprises: According to the ground contact information, determining a critical ground contact point and an effective ground contact point of the target obstacle, the critical ground contact point being a ground contact point located at the edge of an effective area, and the effective ground contact point being a ground contact point located in the effective area except the critical ground contact point, the effective area being an area in the image whose image content can be mapped to the AVM map; According to the critical ground contact point and the effective ground contact point, determining the closed ground contact box of the target obstacle in the AVM map. The obstacle ground box determination method according to claim 1 or 2, wherein, The step of determining the critical ground contact point of the target obstacle according to the ground contact information comprises: According to the ground contact information, determining a ground contact box of the target obstacle; According to the intersection line between an invalid area and the effective area and the ground contact box, determining the critical ground contact point of the target obstacle, wherein the invalid area is an area in the image whose image content cannot be mapped to the AVM map. The obstacle ground box determination method according to claim 3, wherein The step of determining the critical ground contact point of the target obstacle according to the intersection line between the invalid area and the effective area and the ground contact box comprises: Determining the intersection point of the intersection line and the ground contact box as the critical ground contact point of the target obstacle. The obstacle ground box determination method according to claim 2, wherein The step of determining the critical ground contact point of the target obstacle according to the ground contact information comprises: According to the ground contact information, determining a cross-border edge of the target obstacle, the cross-border edge being an edge of the target obstacle that is not fully displayed in the AVM map after the image is mapped to the AVM map; According to the cross-border edge, determining the critical ground contact point of the target obstacle. The obstacle ground box determination method according to claim 5, wherein The ground contact information comprises a first ground contact point coordinate of an edge of the target obstacle in contact with the ground and an end point of the edge, and the step of determining the cross-border edge of the target obstacle according to the ground contact information comprises: According to the first ground contact point coordinate, determining a first identifier for indicating whether the end point of the edge is located in the effective area; When the first identifiers of the respective end points corresponding to the edge of the target obstacle in contact with the ground are different, determining the edge as the cross-border edge of the target obstacle. The obstacle ground box determination method according to claim 6, wherein The step of determining the first identifier for indicating whether the end point of the edge is located in the effective area according to the first ground contact point coordinate comprises: According to the first ground contact point coordinate and a preset coordinate information reference table, determining the first identifier for indicating whether the end point of the edge is located in the effective area, wherein the coordinate information reference table is used to indicate whether a pixel point in the image is located in the effective area. The obstacle ground box determination method according to any one of claims 5-7, characterized in that, The step of determining the critical ground contact point of the target obstacle according to the cross-border edge comprises: determining a second identifier used for representing whether a contact point in the cross-border edge is located in the effective area; when the second identifiers between two adjacent contact points in the cross-border edge are different, using the second identifiers corresponding to the target contact points in the two adjacent contact points for representing whether the contact points are located in the effective area, and determining the target contact points as the critical contact points of the target obstacle. The obstacle ground box determination method according to claim 8, wherein The process when the second identifiers between two adjacent contact points in the cross-border edge are different, using the second identifiers corresponding to the target contact points in the two adjacent contact points for representing whether the contact points are located in the effective area, and determining the target contact points as the critical contact points of the cross-border edge located at the edge of the effective area, comprises: taking any one of the end points corresponding to the cross-border edge as an initial first to-be-processed contact point, and cyclically executing the following process: when the second identifiers between the first target contact point adjacent to the first to-be-processed contact point and the first to-be-processed contact point are the same, taking the first target contact point as a new first to-be-processed contact point, until the second identifiers between the first target contact point and the first to-be-processed contact point are different, and when the second identifiers between the first target contact point and the first to-be-processed contact point are different, using the second identifiers corresponding to the first target contact point and the first to-be-processed contact point for representing whether the contact points are located in the effective area, and determining the first target contact point and the first to-be-processed contact point as the critical contact points of the cross-border edge located at the edge of the effective area. The process of determining the second identifier used for representing whether a contact point in the cross-border edge is located in the effective area, comprises: The obstacle ground box determination method according to claim 8 or 9, characterized in that, determining second contact point coordinates of a contact point in the cross-border edge; determining the second identifier used for representing whether a contact point in the cross-border edge is located in the effective area according to the second contact point coordinates and a preset coordinate information reference table, wherein the coordinate information reference table is used for indicating whether a pixel point in the image is located in the effective area. The process of determining the second contact point coordinates of a contact point in the cross-border edge, comprises: The obstacle ground box determination method according to claim 10, wherein taking any one of the end points corresponding to the cross-border edge as an initial second to-be-processed contact point, and cyclically executing the following process: determining second contact point coordinates of a second target contact point adjacent to the second to-be-processed contact point in the cross-border edge according to the contact point coordinates of the second to-be-processed contact point, a preset horizontal axis interval value and a preset vertical axis interval value, and taking the second target contact point as a new second to-be-processed contact point, until taking an adjacent contact point of another end point corresponding to the cross-border edge as the second to-be-processed contact point, wherein the horizontal axis interval value is used for representing a horizontal axis coordinate interval value between adjacent contact points in the cross-border edge, and the vertical axis interval value is used for representing a vertical axis coordinate interval value between adjacent contact points in the cross-border edge. The horizontal axis interval value and the vertical axis interval value are determined by the following process: The obstacle ground box determination method according to claim 11, wherein determining horizontal axis coordinate difference values and vertical axis coordinate difference values between the end points corresponding to the cross-border edge according to the contact point coordinates of the end points corresponding to the cross-border edge; ​ determining the preset coordinate interval value as the horizontal axis interval value when the absolute value of the horizontal axis coordinate difference value is greater than the absolute value of the vertical axis coordinate difference value, and determining the vertical axis interval value according to the coordinate interval value and the preset calculation formula; determining the preset coordinate interval value as the vertical axis interval value when the absolute value of the horizontal axis coordinate difference value is greater than the absolute value of the vertical axis coordinate difference value, and determining the horizontal axis interval value according to the coordinate interval value and the preset calculation formula. The obstacle ground box determination method according to any one of claims 10 to 12, characterized in that, The coordinate information correspondence table is obtained by the following method: determining a first coordinate of a pixel point in the image in the image, and determining a second coordinate of the pixel point in the AVM graph based on the first coordinate, a distance corresponding to the pixel point, and a preset coordinate mapping relationship, wherein the coordinate mapping relationship is used to represent a mapping relationship between a coordinate of a pixel point in the image and a coordinate of the pixel point in the AVM graph; determining whether a pixel point corresponding to the second coordinate is located in a target mark of the effective area according to an image size of the AVM graph and the second coordinate; determining a coordinate information correspondence table according to a corresponding relationship among the pixel point, the first coordinate, and the target mark. The obstacle ground box determination method according to claim 13, wherein The coordinate mapping relationship is obtained by the following method: jointly calibrating a camera used to acquire an image to obtain a camera intrinsic parameter and a camera extrinsic parameter with a target position as an origin; determining a first product of a coordinate matrix of a pixel point in the image and a preset camera depth, and determining a second product of a camera intrinsic parameter matrix and a camera extrinsic parameter matrix, and determining a coordinate mapping relationship as a ratio of the first product to the second product, wherein the coordinate matrix is determined according to a coordinate of the pixel point, the camera intrinsic parameter matrix is determined according to the camera intrinsic parameter, and the camera extrinsic parameter matrix is determined according to the camera extrinsic parameter. The obstacle ground box determination method according to any one of claims 2 to 14, characterized in that, The determining the closed grounding frame of the target obstacle in the AVM graph according to the critical grounding point and the effective grounding point comprises: connecting the critical grounding point and the effective grounding point to form a starting grounding frame in the image; mapping the starting grounding frame into the AVM graph according to a preset coordinate mapping relationship to obtain the closed grounding frame of the target obstacle in the AVM graph. The determining the grounding point information of the target obstacle in the image acquired by a vehicle-based vehicle-mounted shooting device comprises: The obstacle ground box determination method according to any one of claims 1 to 15, wherein, inputting the image into a target detection model to obtain the grounding point information of the target obstacle, wherein the target detection model is obtained by training a detection model with a sample image as input and grounding point information of a sample obstacle as output. The image is acquired by the following method: The obstacle ground box determination method according to any one of claims 1 to 16, wherein, The image is acquired by an fisheye camera arranged at a preset position of the vehicle. The controller comprises: A controller characterized by comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the steps of the method of any one of claims 1-17. including the controller of claim 18, or A vehicle characterized by comprising: ​ The vehicle comprises a processor and a memory for storing processor-executable instructions, wherein the processor is configured to perform the steps of the method of any one of claims 1-17. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, which when executed by a processor, implements the steps of the method of any one of claims 1-17. A computer program product comprising a computer program, characterized in that The computer program, which when executed by a processor, implements the steps of the method of any one of claims 1-17.

Images