Detection frame determination method and device, vehicle and storage medium

Abstract

The invention relates to a detection frame determination method and device, a vehicle and a storage medium, and relates to the technical field of automatic driving, and the method comprises the steps: determining a detection frame and a fitting frame corresponding to an obstacle in a to-be-processed image, the detection frame and the fitting frame are used for positioning the obstacle, the detection frame is a detection frame outputted by an obstacle detection model, and the fitting frame is used for positioning the obstacle; the fitting frame is obtained by fitting according to the size of the obstacle; and correcting the detection frame by using the fitting frame to obtain a target detection frame. By using the detection frame determination method and device, the vehicle and the storage medium provided by the invention, a more accurate target detection frame can be obtained.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to a detection frame determination method, device, vehicle, and storage medium. Background Art

[0002] In the field of autonomous driving, it is necessary to acquire images through an image acquisition device and mark the detection frames in the images. The detection frames can be used to locate obstacles in the images, distinguish multiple obstacles in the images, classify different obstacles in the images, and track obstacles.

[0003] In the related art, an obstacle detection model is used to simulate and obtain a detection frame corresponding to an obstacle in an image. However, the accuracy of the detection frame obtained by the obstacle detection model is low. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a detection frame determination method, device, vehicle and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for determining a detection frame is provided, comprising:

[0006] Determine a detection frame and a fitting frame corresponding to an obstacle in the image to be processed, wherein the detection frame and the fitting frame are used to locate the obstacle, the detection frame is a detection frame output by an obstacle detection model, and the fitting frame is a fitting frame obtained by fitting according to the size of the obstacle;

[0007] The detection frame is corrected using the fitting frame to obtain a target detection frame.

[0008] Optionally, determining a fitting frame corresponding to an obstacle in the image to be processed includes:

[0009] Determine a grounding point of the obstacle, where the grounding point is a contact point between the obstacle and the ground;

[0010] The fitting frame is obtained according to the grounding point.

[0011] Optionally, determining the grounding point of the obstacle includes:

[0012] The image to be processed is used as an input parameter of a ground point detection model to obtain the ground point.

[0013] Optionally, the grounding point detection model is trained by the following steps:

[0014] Screening out, from a plurality of obstacles in the image samples acquired by the image acquisition device, a target obstacle whose distance from the image acquisition device is less than a first preset distance;

[0015] The image samples are used as training samples, and the grounding point of each target obstacle, the position corresponding to the grounding point, and the category of the grounding point are used as labels to train the first initial model to obtain the grounding point detection model.

[0016] Optionally, the obstacle includes a first obstacle; and obtaining the fitting frame according to the grounding point includes:

[0017] According to two end points of the plurality of grounding points of the first obstacle, fitting to obtain the length of the first obstacle;

[0018] According to a first grounding point among the plurality of grounding points, the distance between the first grounding point and the long side of the first obstacle being less than a second preset distance, fitting to obtain the width of the first obstacle;

[0019] A fitting frame corresponding to the first obstacle is obtained by fitting according to the length and width of the first obstacle and the standard height of the first obstacle.

[0020] Optionally, the obstacle includes a second obstacle; and obtaining the fitting frame according to the grounding point includes:

[0021] Obtaining a center point of the second obstacle according to positions of a plurality of grounding points of the second obstacle;

[0022] A fitting frame corresponding to the second obstacle is obtained by fitting according to the central point, the standard length of the second obstacle, the standard width of the second obstacle, and the standard height of the second obstacle.

[0023] Optionally, the detection frame includes the size of the obstacle; and determining the detection frame corresponding to the obstacle in the image to be processed includes:

[0024] Determining a size range of the obstacle and a standard size of the obstacle;

[0025] When the predicted size of the obstacle output by the obstacle detection model is outside the size range, the predicted size is updated to the standard size.

[0026] Optionally, the size of the obstacle includes a predicted width of the obstacle, and the size range includes a width range; when the predicted size of the obstacle output by the obstacle detection model is outside the size range, updating the predicted size to the standard size includes:

[0027] When the predicted width of the obstacle output by the obstacle detection model is outside the width range, the width of the obstacle is updated to a standard width.

[0028] Optionally, the size of the obstacle includes a predicted length of the obstacle, and the size range includes a length range; when the predicted size of the obstacle output by the obstacle detection model is outside the size range, updating the predicted size to the standard size includes:

[0029] According to the predicted width, fitting is performed to obtain the predicted length;

[0030] When the predicted length is outside the length range, the length of the obstacle is updated to a standard length.

[0031] Optionally, the size includes a predicted height of the obstacle, and the size range includes a height range; when the predicted size of the obstacle output by the obstacle detection model is outside the size range, updating the predicted size to the standard size includes:

[0032] According to the predicted width, fitting to obtain the predicted height;

[0033] When the predicted height is outside the height range, the height of the obstacle is updated to a standard height.

[0034] Optionally, the using the fitting frame to correct the detection frame to obtain the target detection frame includes:

[0035] The target detection frame is obtained according to the detection frame parameters of the detection frame, the first weights corresponding to the detection frame parameters, the fitting frame parameters of the fitting frame, and the second weights corresponding to the fitting frame parameters.

[0036] Optionally, the using the fitting frame to correct the detection frame to obtain the target detection frame includes:

[0037] Filtering out the detection frame and the fitting frame corresponding to the maximum matching value from the target matching values ​​between at least one of the detection frames and at least one of the fitting frames;

[0038] The fitting frame corresponding to the maximum matching value is used to correct the detection frame corresponding to the maximum matching value to obtain the target detection frame.

[0039] Optionally, the method further comprises:

[0040] Matching the detection frame with the fitting frame based on a plurality of different matching methods to obtain a plurality of matching values;

[0041] A plurality of the matching values ​​are fused to obtain a target matching value between the detection frame and the fitting frame.

[0042] According to a second aspect of an embodiment of the present disclosure, a detection frame determination device is provided, including:

[0043] a determination module configured to determine a detection frame and a fitting frame corresponding to an obstacle in the image to be processed, wherein the detection frame and the fitting frame are used to locate the obstacle, the detection frame is a detection frame output by an obstacle detection model, and the fitting frame is a fitting frame obtained by fitting according to the size of the obstacle;

[0044] The correction module is configured to use the fitting frame to correct the detection frame to obtain a target detection frame.

[0045] According to a third aspect of an embodiment of the present disclosure, a detection frame determination device is provided, including:

[0046] processor;

[0047] a memory for storing processor-executable instructions;

[0048] Wherein, the processor is configured to:

[0049] Execute the steps of the detection frame determination method provided in the first aspect of the embodiment of the present disclosure.

[0050] According to a fourth aspect of an embodiment of the present disclosure, a vehicle is provided, on which the detection frame determination device provided by the third aspect of the embodiment of the present disclosure is configured, or the vehicle is connected to the detection frame determination device provided by the third aspect of the embodiment of the present disclosure.

[0051] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the detection frame determination method provided by the first aspect of the embodiment of the present disclosure are implemented.

[0052] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0053] Through the above technical solution, the detection frame can be corrected using an accurate fitting frame. In this way, the obtained target detection frame incorporates the information of the fitting frame, and the target detection frame can be more accurate than the detection frame output by the obstacle detection model.

[0054] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0056] Figure 1 The figure is a flowchart of a method for determining a detection frame according to an exemplary embodiment.

[0057] Figure 2 It is a schematic diagram showing an image to be processed of the back side of a vehicle A in front photographed by a host vehicle according to an exemplary embodiment.

[0058] Figure 3 It is a schematic diagram showing a method of screening target obstacles whose distance from a vehicle is less than a first preset distance according to an exemplary embodiment.

[0059] Figure 4 The diagram is a logical diagram of updating the size of an obstacle according to an exemplary embodiment.

[0060] Figure 5 The figure is a schematic diagram showing a method of obtaining a detection frame and a fitting frame corresponding to a maximum matching value according to an exemplary embodiment.

[0061] Figure 6 The figure is a block diagram of a detection frame determination device according to an exemplary embodiment.

[0062] Figure 7 is a block diagram of a vehicle according to an exemplary embodiment.

[0063] Figure 8 The figure is a block diagram of a detection frame determination device according to an exemplary embodiment. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0065] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the owner of the corresponding device.

[0066] In the related art, the image acquisition device will input the collected image to be processed into the obstacle detection model, and the radar device will also input the collected point cloud data into the obstacle detection model, and simulate the obstacle detection model to obtain the detection frame corresponding to the obstacle in the image to be processed. However, due to the limitation of the collection field of view of the image acquisition device and the radar device, or the interference of other obstructions, the image acquisition device and the radar device can usually only collect data on one side of the obstacle. The above-mentioned image acquisition device can be a camera, and the radar device can be a laser radar device.

[0067] See also Figure 2 As shown in the figure, if there is an obstacle, vehicle A, in front of the vehicle, the image acquisition device on the vehicle can only capture the back image of vehicle A, and the radar device can only detect the distance between the back of vehicle A and the vehicle. The image acquisition device and the radar device cannot capture all the information of vehicle A, such as the side or front information of vehicle A. If the back information of vehicle A is input into the obstacle detection model, the obstacle detection model can only simulate the back information of the detection frame corresponding to the vehicle A, and the detection frame is a 3D frame with six faces. The obstacle detection model can only guess the information of the other five faces of the simulated detection frame based on the back information of the detection frame. The detection frame simulated in this way is naturally inaccurate.

[0068] Figure 1 is a flow chart of a method for determining a detection frame according to an exemplary embodiment. Figure 1 As shown, the following steps are included.

[0069] In step S11, a detection frame and a fitting frame corresponding to the obstacle in the image to be processed are determined.

[0070] Both the detection frame and the fitting frame are used to locate and identify obstacles. The detection frame and the fitting frame are 3D frames. Usually, the detection frame and the fitting frame are cubes, both of which have six faces. The difference is that the detection frame is the detection frame output by the obstacle detection model, while the fitting frame is the fitting frame obtained by fitting according to the accurate size of the obstacle.

[0071] Determining the detection frame corresponding to the obstacle includes: determining the detection frame, the detection frame position (the detection frame position includes the three-dimensional coordinates of each vertex of the detection frame and the three-dimensional coordinates of the center point of the detection frame), the detection frame category (the detection frame category is the obstacle category) and the detection frame size (the detection frame size includes length, width and height), etc. Determining the fitting frame corresponding to the obstacle includes: determining the fitting frame, determining the fitting frame position, the fitting frame category and the fitting frame size, etc.

[0072] For the detection frame, the image acquisition device can first collect image samples, and use the point cloud data collected by the radar device at the same time as the same set of data, and use the point cloud data at the same time to annotate the detection frame envelope, detection frame position, detection frame category and detection frame size corresponding to each obstacle in the image sample; then input the annotated image sample into the second initial model, train the second initial model, and obtain an obstacle detection model. The obstacle detection model mainly identifies obstacles, and identifies the category, position and size of obstacles. After receiving the current image to be processed, the trained obstacle detection model can output the detection frame size, detection frame position, detection frame category, detection frame heading angle and confidence score corresponding to each obstacle in the image to be processed. The confidence score means that for each obstacle detection frame, the obstacle detection model can output the confidence that the category of the detection frame is a preset category. For example, the obstacle detection model outputs that the confidence that the detection frame category is a person is 70%, the confidence that it is a car is 50%, the confidence that it is a pillar is 30%, and so on. Ultimately, the preset category corresponding to the maximum confidence will be used as the detection frame category.

[0073] Point cloud data is a collection of discrete three-dimensional point coordinate data. The radar device transmits a light beam and receives the light beam reflected by the obstacle to determine the distance and direction of the obstacle compared to the radar, thereby obtaining the three-dimensional position information of the obstacle. Point cloud data includes the location information, shape information, color information, reflection intensity information and classification label information of the obstacle.

[0074] Using point cloud data to annotate the detection frame of each obstacle in the image sample includes: (1) selecting the obstacle; (2) point cloud segmentation, using the point cloud segmentation algorithm to extract the points related to the obstacle from the entire point cloud data as the point cloud data of the obstacle; (3) feature extraction, performing feature extraction on the point cloud data of the obstacle to describe the shape and structure of the obstacle; (4) annotating the detection frame, drawing the detection frame corresponding to the obstacle based on the shape information and position information contained in the point cloud data of the obstacle, and annotating the category information, structure information and other information in the point cloud data as the category information and structure information corresponding to the detection frame.

[0075] For the fitting frame, the accurate grounding point of the obstacle can be determined first, and the width and length of the obstacle can be fitted according to the grounding point. Then, the center point of the obstacle can be fitted using the width and length of the obstacle. Finally, the fitting frame corresponding to the obstacle can be fitted according to the center point, width, length and standard height of the obstacle. The fitting frame includes the position of the fitting frame (the position of the fitting frame includes the three-dimensional coordinates of the center point of the fitting frame and the three-dimensional coordinates of each vertex) and the size of the fitting frame.

[0076] In the disclosed embodiment, the size of the obstacle is the size of the fitting frame, that is, the length, width and height of the obstacle are the length, width and height of the fitting frame, so the size of the obstacle will be referred to as the size of the fitting frame in some descriptions.

[0077] In step S12, the detection frame is corrected using the fitting frame to obtain a target detection frame.

[0078] Correcting the detection frame using the fitting frame may include any of the following solutions: updating the size of the detection frame to the size of the fitting frame, and weighted averaging the fitting frame parameters of the fitting frame and the detection frame parameters of the detection frame. The fitting frame parameters include information such as the position of the fitting frame and the size of the fitting frame, and the detection frame parameters include information such as the position of the detection frame and the size of the detection frame.

[0079] After obtaining the target detection frame, the target detection frame is integrated with the fitting frame parameters of the fitting frame, so the obtained target detection frame is more accurate than the detection frame output by the original obstacle detection model.

[0080] The target detection frame can be used to distinguish multiple obstacles, locate obstacles, classify different obstacles, exclude non-target areas in the image that are not in the target detection frame, track obstacles, and other target tasks. Based on the high accuracy of the obtained target detection frame, the accuracy in achieving these target tasks is also high.

[0081] Through the above technical solution, the detection frame can be corrected using an accurate fitting frame. In this way, the obtained target detection frame incorporates the information of the fitting frame and can be more accurate than the detection frame output by the obstacle detection model.

[0082] The optional embodiments and specific embodiments involved in the above steps S11 to S12 will be introduced below.

[0083] In general, the grounding point of the obstacle can be determined, and a fitting frame can be obtained by fitting according to the grounding point. Different fitting methods are used to fit different types of obstacles to obtain the fitting frame. The following will take the two types of obstacles, the first obstacle and the second obstacle, as examples to explain the steps of obtaining the fitting frame.

[0084] The first obstacle may be an obstacle of the same obstacle type such as a vehicle but with size deviation. For the first obstacle, a fitting frame may be obtained by fitting the following steps:

[0085] Step S21: Determine a grounding point of the first obstacle, where the grounding point is a contact point between the obstacle and the ground.

[0086] The image to be processed may be used as an input parameter of a ground point detection model to obtain a ground point, a category of the ground point, and a confidence score that the category of the ground point is a preset category.

[0087] The grounding point detection model can be trained by the following method: from multiple obstacles in the image samples obtained by the image acquisition device, screen out the target obstacles whose distance from the image acquisition device is less than a first preset distance; use the image samples as training samples, and use the grounding point of each target obstacle, the position corresponding to the grounding point, and the category of the grounding point as labels to train the first initial model to obtain the grounding point detection model.

[0088] For example, see Figure 3 As shown, taking the first preset distance of 5m as an example, the point cloud data corresponding to each frame of image samples can be projected onto the image samples first; then the target obstacles with a distance of less than 5m from the vehicle in each frame of image are selected, for example Figure 3 In the figure, the distance between vehicle A and the vehicle is less than 5m, while the distances between vehicle B, vehicle C and the vehicle are all greater than or equal to 5m, so vehicle A is taken as the target obstacle; then the grounding line between the target obstacle and the ground is marked. For example, when the image sample of the vehicle A in front is obtained from the perspective of the vehicle, the line between vehicle A and the ground in the image sample is the grounding line; then multiple grounding points on the grounding line are evenly collected, each with its own position and category; then the image samples are used as training samples, and each grounding point, the position of the grounding point, and the category of the grounding point are used as labels to train the first initial model and obtain the grounding point detection model. The grounding point detection model is used to identify the grounding points in the image and identify the category and position of the grounding points.

[0089] The above example needs further explanation:

[0090] (1) The reason for projecting point cloud data onto image samples is that after the point cloud data such as the position, color and shape of the ground and each obstacle are projected onto the image samples through the external parameters and internal parameters of the image acquisition device, the ground and each obstacle will have their own shapes and colors, so that the grounding wire between the obstacle and the ground can be identified.

[0091] (2) The reason for selecting the target obstacle whose distance from the vehicle is less than the first preset distance in each frame of the image sample is that the closer the target obstacle is to the vehicle in the image sample, the higher the image clarity, the larger the display area of ​​the target obstacle, and the richer the picture details, while the farther the target obstacle is from the vehicle, the lower the image clarity, and the target obstacle farther from the vehicle will be blocked by other obstacles. Therefore, if the obstacle farther from the vehicle in the image sample is used as a training sample to train the first initial model, the grounding point detection model obtained by training will not be very accurate. Therefore, the target obstacle with higher clarity and not blocked by other obstacles can be used as a training sample to train the first initial model, so that the grounding point detection model obtained can more accurately predict the grounding point.

[0092] (3) Since the grounding points are multiple points uniformly collected from the grounding line, the category of the grounding line is the category of the grounding point, and the category of the grounding line is the category of the obstacle, and is also the category of the detection box or the fitting box.

[0093] Step S22: fitting the length of the first obstacle according to two end points of the plurality of grounding points of the first obstacle.

[0094] First, the grounding points whose confidence is greater than the confidence threshold are filtered out from the multiple grounding points output by the grounding point detection model to obtain multiple grounding points whose confidence is greater than the confidence threshold; then, from the multiple grounding points whose confidence is greater than the confidence threshold, multiple grounding points whose distance from the vehicle is less than a first preset distance (for example, 5m) are screened out; then, the multiple grounding points whose distance from the vehicle is less than the first preset distance are clustered to obtain multiple grounding points corresponding to first obstacles, and the multiple first obstacles may be first obstacles of the same type but different targets (for example, different vehicles under the same vehicle type), or may be first obstacles of different types.

[0095] For example, all the grounding points in the image to be processed output by the grounding point detection model can be traversed, and a first grounding point set of categories being the head and tail of the vehicle and a second grounding point set of categories being the side of the vehicle can be saved respectively; then all the grounding points in the second grounding point set of the side of the vehicle are clustered, and a grounding point is randomly selected first, and then a grounding point whose distance from the grounding point is less than a preset distance is found, and the randomly selected grounding point and the grounding point whose distance from the grounding point is less than the preset distance are taken as grounding points corresponding to a first obstacle, and the clustering step is repeatedly performed, and finally all the grounding points are divided into grounding points corresponding to different first obstacles.

[0096] After all the grounding points in the second grounding point set are divided into grounding points corresponding to different first obstacles, for each grounding point of the first obstacle, the multiple grounding points corresponding to the first obstacle can be sorted according to the size of the y coordinates, and then the first grounding point and the last grounding point after the sorting are used as the two endpoints of the multiple grounding points of the first obstacle; then the line between the y coordinates of the two endpoints is used as the long side of the first obstacle, and the difference between the y coordinates of the two endpoints is used as the length of the first obstacle, and the length is used to measure the size of the long side.

[0097] For example, the first obstacle is a vehicle, and the three-dimensional coordinates (x, y, z) of the multiple grounding points in the second grounding point set are (0, 2, 0), (0, 1, 0), (0, 3, 0), and (0, 4, 0), respectively. Then, by sorting the y coordinates from small to large, we can get (0, 1, 0), (0, 2, 0), (0, 3, 0), and (0, 4, 0), and the two endpoints are (0, 1, 0) and (0, 4, 0), respectively. Subtracting the y coordinates of the two endpoints, we can get that the length of the vehicle is 3.

[0098] After obtaining the long side of the first obstacle, the heading angle of the first obstacle can be calculated according to the two endpoints of the long side of the first obstacle. For example, the slope of the straight line formed by (0, 4, 0) and (0, 1, 0) in the above example is used as the heading angle, and the x coordinates of (0, 4, 0) and (0, 1, 0) are the same, which means that the first obstacle is traveling in a straight line.

[0099] Step S23: fitting the width of the first obstacle according to a first grounding point among the plurality of grounding points, the distance between the first grounding point and the long side of the first obstacle being less than a second preset distance.

[0100] All the grounding points in the first grounding point set at the front and rear of the vehicle can be traversed, and the first grounding point whose distance to the long side of the first obstacle is less than the second preset distance is first found; then the multiple first grounding points are sorted according to the size of the x-coordinates, and then the first grounding point and the last grounding point after sorting are used as two endpoints; then the line between the x-coordinates of the two endpoints is used as the wide side of the first obstacle, and the difference between the x-coordinates of the two endpoints is used as the width of the first obstacle, and the width is used to measure the size of the wide side.

[0101] For example, the first obstacle is a vehicle, and the three-dimensional coordinates (x, y, z) of the multiple grounding points in the first grounding point set are (2, 0, 0), (1, 0, 0), and (3, 0, 0), respectively. Then, by sorting them from small to large according to the x-coordinate, we can get (1, 0, 0), (2, 0, 0), and (3, 0, 0), and the two endpoints are (1, 0, 0) and (3, 0, 0), respectively. Subtracting the x-coordinates of the two endpoints, we can get the width of the vehicle is 2.

[0102] In the above process, since there may be adjacent first obstacles in the captured image to be processed, such as adjacent vehicles A and vehicle B, if the method in step S23 is adopted, although the first grounding point in the wide side of vehicle A whose distance from the long side of vehicle A is less than the second preset distance can be determined, the grounding point in the wide side of the adjacent vehicle B whose distance from the long side of vehicle A is less than the second preset distance will also be regarded as the first grounding point. In this case, the grounding point in the wide side of vehicle B will be misjudged as the grounding point in the wide side of vehicle A.

[0103] Based on this, Hungarian matching can also be used to find multiple grounding points that best match the long side of the first obstacle from multiple first grounding points, that is, find the grounding points on the wide side connected to the long side of the first obstacle from multiple first grounding points, and use these grounding points as the multiple grounding points that best match the long side of the first obstacle.

[0104] For example, Hungarian matching can find the first grounding point on the broad side connected to the long side of vehicle A, and filter out the first connection point on the broad side of vehicle B, so that the obtained first connection point is located on the broad side of vehicle A.

[0105] Of course, in the above steps S23 and S24, the wide side of the first obstacle may be fitted first, and then the long side of the first obstacle may be fitted using the wide side of the first obstacle. The specific order is not limited.

[0106] Step S24: fitting a fitting frame corresponding to the first obstacle according to the length and width of the first obstacle and the standard height of the first obstacle.

[0107] The standard height may be the average height of multiple first obstacles of the same type obtained after investigation. The first obstacles of the same type may include: cars of the same type, trucks of the same type, and SUVs (sport utility vehicles) of the same type. For example, after investigation, the heights of multiple cars are obtained, and then the average height of the multiple cars is used as the standard height of the cars; for another example, after investigation, the heights of multiple trucks are obtained, and then the average height of the multiple trucks is used as the standard height of the trucks.

[0108] After obtaining the length, width and height of the first obstacle, the three-dimensional coordinates of the center point of the first obstacle can be obtained according to the length, width and standard height of the first obstacle, where the x coordinate of the center point is half of the length of the first obstacle, the y coordinate is half of the width of the first obstacle, and the z coordinate is half of the height of the first obstacle; finally, the fitting frame of the first obstacle is drawn according to the center coordinates of the first obstacle, the length, width and height of the first obstacle.

[0109] In addition to counting the standard height of the first obstacle, the standard length and standard width of the first obstacle may also be counted. The standard length of the first obstacle is the average length of the first obstacles of the same type, and the standard width of the first obstacle is the average width of the first obstacles of the same type.

[0110] For the first obstacle, since the sizes of the first obstacles of the same type may be different, the standard length of the first obstacle may have a small deviation from the actual length of the first obstacle. Similarly, the standard width of the first obstacle may also have a small deviation from the actual width of the first obstacle. For example, the standard length of many trucks is 12 meters, but for some trucks, the actual length of the current truck is 13 meters, so there is a 1-meter deviation from the standard length.

[0111] Therefore, the width and length of the first obstacle can be fitted first, so that the obtained width and length of the first obstacle are closer to the current actual width and length of the first obstacle. For example, the image to be processed of the current truck can be received through the ground point detection model, and the ground point detection model outputs multiple ground points of the side of the current truck (long side), and then the length of the line connecting the two endpoints of the multiple ground points is used as the actual length of the current truck; then the actual length of the current truck and the multiple ground points of the front and rear of the current truck (wide side) are used to obtain the actual width of the current truck, and finally the fitting frame obtained by fitting the actual length and actual width of the current truck is more accurate.

[0112] For the first obstacle, the length range, width range and height range of the fitting frame corresponding to the first obstacle can also be set. After fitting the length, width and height of the fitting frame corresponding to the first obstacle, if the length of the fitting frame is outside the length range, the standard length of the first obstacle is used as the length of the fitting frame; if the width of the fitting frame is outside the width range, the standard width of the first obstacle is used as the width of the fitting frame; if the height of the fitting frame is outside the height range, the standard height of the first obstacle is used as the height of the fitting frame.

[0113] The length of the fitting box is used for detailed description. If the length of the fitting box is outside the length range, it means that the length of the fitting box obtained by fitting is inaccurate, which can further indicate that the grounding point output by the grounding point detection model has an error. At this time, the standard length of the first obstacle can be used as the length of the fitting box, so that the length of the fitting box can be more accurate. Of course, the width and height of the fitting box are the same, which will not be repeated here.

[0114] Through the above technical solution, the grounding point of the first obstacle in the image to be processed can be outputted through the grounding point detection model, and then the length of the first obstacle can be obtained according to the two endpoints of the grounding point of the long side of the first obstacle; then the grounding point on the short side that is closer to the long side of the first obstacle is determined, and then the width of the first obstacle is obtained according to the two endpoints of the grounding point of the short side of the first obstacle; finally, according to the length and width of the first obstacle and the standard height of the first obstacle, a fitting frame that is closer to the current first obstacle is fitted.

[0115] Since the grounding point detection model can receive the image to be processed in real time, if the long side and the wide side of the first obstacle are displayed in the image to be processed, the image to be processed can be input into the grounding point detection model. The grounding point detection model can output multiple grounding points of the long side and the wide side of the first obstacle according to the image to be processed. Finally, these grounding points are used to fit the actual length and width of the first obstacle, and further obtain an accurate fitting frame of the first obstacle.

[0116] Of course, when the length of the first obstacle is not within the length range, or the width of the first obstacle is not within the width range, it means that the grounding point output by the grounding point detection model is wrong, or only the long side or the wide side exists in the image to be processed received by the grounding point monitoring model, resulting in an error in the calculated length or width. At this time, the standard length and standard width of the first obstacle can be used as the current length and width of the first obstacle, so that when there is an error in the grounding point output by the grounding point detection model, the error is avoided from affecting the calculated length and width of the first obstacle, the size error of the first obstacle is reduced, and the fault tolerance of the grounding point detection model is improved; and because the standard length and standard width of the first obstacle are slightly different from the actual length and width of the first obstacle, the accuracy of the fitting frame of the first obstacle is relatively high, and naturally, the accuracy of the fitting frame obtained by fitting with the standard length and standard width is also high.

[0117] The second obstacle can be a non-vehicle, such as a cone, a wall, a pillar, or other obstacles of the same obstacle type and size. For the second obstacle, the fitting frame can be obtained by the following steps:

[0118] Step S31: Determine a grounding point of the second obstacle, where the grounding point is a contact point between the obstacle and the ground.

[0119] The principle of this step is the same as that of the above step S21, and will not be repeated here.

[0120] Step S32: Obtaining the center point of the second obstacle according to the positions of the multiple grounding points of the second obstacle.

[0121] The coordinates of the multiple grounding points of the second obstacle may be averaged to obtain the center point of the second obstacle. For example, if the multiple grounding points of the second obstacle are (2, 1, 0) and (2, 1, 0), the center point of the second obstacle is (2, 1, 0).

[0122] It can be understood that, since the grounding point is located on the ground, the z coordinate of the grounding point is 0, and naturally, the center point of the second obstacle obtained in step S32 is also 0.

[0123] Step S33: fitting a fitting frame corresponding to the second obstacle according to the central point, the standard length of the second obstacle, the standard width of the second obstacle, and the standard height of the second obstacle.

[0124] The z coordinate of the center point of the second obstacle can be replaced by half of the standard height of the second obstacle to obtain the updated center point of the second obstacle. For example, if the standard height of the second obstacle is 5, the center point of the second obstacle finally obtained is (2, 1, 2.5).

[0125] Since the sizes of the second obstacles are basically the same, the average length of multiple second obstacles of the same type can be counted as the standard length of the second obstacle, the average width of multiple second obstacles of the same type can be counted as the standard width of the second obstacle, and the average height of multiple second obstacles of the same type can be counted as the standard height of the second obstacle, without fitting the length and width of the second obstacle.

[0126] For example, the sizes of cones in various cities are the same. In this case, the average length of multiple cones can be used as the standard length of the cones, the average width of multiple cones can be used as the standard width of the cones, and the average height of multiple cones can be used as the standard height of the cones.

[0127] Through the above technical solution, the grounding point of the second obstacle in the image to be processed can be output by the grounding point detection model, and then the average value of the grounding points of the second obstacle can be used as the center point of the second obstacle. Finally, the fitting box corresponding to the second obstacle can be drawn according to the center point, standard length, standard width and standard height of the second obstacle.

[0128] Figure 4 A schematic diagram of obtaining the length, width and height of the detection frame is shown, which specifically includes the following steps:

[0129] In step S41 , it is determined whether the predicted width of the obstacle output by the obstacle detection model is outside the width range.

[0130] Since the predicted width of the obstacle output by the obstacle detection model may have errors, it is necessary to determine whether the predicted width is outside the width range.

[0131] In step S42 - 1 , when the predicted width of the obstacle output by the obstacle detection model is within the width range, the width of the obstacle is updated to the predicted width.

[0132] When the predicted width of the obstacle output by the obstacle detection model is within the width range, it means that the predicted width of the obstacle output by the obstacle detection model is accurate, and the predicted width can be used as the width of the obstacle.

[0133] The standard length, standard width and standard height of the obstacle can also be obtained by statistics, and the predicted width can be used for linear interpolation to fit the predicted length and predicted height of the obstacle. For example, if the standard length, standard width and standard height of the obstacle are 4, 2, and 2 respectively, and the predicted width is 1, then the linear relationship between the predicted width and the standard width is 0.5 determined by linear interpolation. Using this linear relationship, the fitted standard length is 2 and the standard height is 1.

[0134] Of course, the least square method can also be used to fit the predicted length and predicted height using the predicted width, standard length, standard width and standard height, which will not be repeated here.

[0135] After the obstacle detection model obtains the image to be processed, the image to be processed usually shows the back of the obstacle, that is, the width of the obstacle. Therefore, the obstacle detection model can predict the predicted width of the obstacle based on the image to be processed, but cannot predict the predicted length and predicted height of the obstacle well. Therefore, a linear fitting method can be used to obtain the accurate predicted width and linearly interpolate the standard length and standard height to fit the predicted length and predicted height of the obstacle. The predicted length and predicted height obtained in this way are more in line with the current length and height of the obstacle.

[0136] In step S42 - 2 , when the predicted width of the obstacle output by the obstacle detection model is outside the width range, the width of the obstacle is updated to a standard width.

[0137] Of course, when the predicted length obtained by fitting is outside the length range, the length of the obstacle is updated to the standard length. When the predicted length obtained by fitting is outside the length range, the length of the obstacle is updated to the standard length.

[0138] When the predicted width of an obstacle output by the obstacle detection model is outside the width range, it means that the prediction of the obstacle detection model is incorrect. In this case, the standard width of the obstacle can be used as the width of the obstacle. Similarly, when the predicted length of the obstacle obtained by fitting is outside the length range, it means that the predicted length obtained by fitting is incorrect. In this case, the standard length of the obstacle can be used as the length of the obstacle. When the predicted height of the obstacle obtained by fitting is outside the height range, it means that the predicted height obtained by fitting is incorrect. In this case, the standard height of the obstacle can be used as the height of the obstacle.

[0139] Through the above technical solution, the obstacle detection model can be used to output the predicted width of the obstacle, and then the predicted width of the obstacle can be used to fit the predicted length and predicted height of the obstacle, so that the predicted length, predicted width and predicted height of the obstacle can be closer to the actual size of the obstacle, and the prediction box obtained according to the predicted length, predicted width and predicted height is also closer to the actual size of the obstacle; when the predicted width of the obstacle is outside the width range, or the predicted length is outside the length range, or the predicted height is outside the predicted height range, it means that the predicted value output by the obstacle detection model is wrong. At this time, the size of the obstacle can be updated to the standard length, standard width and standard height, thereby reducing the size error and improving the fault tolerance of the obstacle detection model.

[0140] It can be understood that when there is a fitting frame, that is, the obstacle in the image to be processed received by the ground point detection model shows the long side and the wide side, then the grounding points of the long side and the wide side can be obtained, and the fitting frame can be fitted. The fitting frame is then used to correct the detection frame to make the obtained target detection frame more accurate; when there is no fitted fitting frame, that is, the obstacle in the image to be processed received by the ground point detection model shows the long side or the wide side, then only the grounding points of the long side or the wide side can be obtained. In this case, there is no fitting frame, and the corrected detection frame can be used. Since the detection frame has been corrected by steps S41, S42-1, and S42-2, the accuracy of the detection frame is higher than that of the original detection frame.

[0141] Figure 5 A schematic diagram of using a fitting frame to correct a detection frame is shown, and the correction method specifically includes:

[0142] In step S51, the detection frame and the fitting frame are matched based on a plurality of different matching methods to obtain a plurality of matching values.

[0143] The different matching methods include the following:

[0144] (1) IOU matching under BEV (bird’s eye view): First, the volume intersection of the overlapping part between the detection box and the fitting box can be calculated; then the volume union between the detection box and the fitting box can be calculated. When calculating the volume union, the volume of the detection box and the fitting box can be added, and the volume intersection can be subtracted to obtain the volume union; finally, the matching value (IOU) is calculated, matching value = volume intersection / volume union. The larger the matching value, the greater the similarity between the detection box and the fitting box, and the greater the degree of overlap.

[0145] (2) Distance matching between the detection frame and the fitting frame: The Euclidean distance between the detection frame and the fitting frame can be calculated. If the distance is less than a threshold, the two are considered to be matched successfully. Then, the matching value is calculated based on the Euclidean distance between the detection frame and the fitting frame. The smaller the Euclidean distance, the larger the matching value, and the greater the similarity between the detection frame and the fitting frame.

[0146] (3) IOU matching on 2D images can be understood as IOU matching without a bird's-eye view: similar to the principle of the above method (1), it will not be repeated here, and the corresponding matching value under this method can be obtained.

[0147] (4) Similarity matching of 2D images: The pixel levels of the crop images corresponding to the detection box and the fitting box can be compared, and the mean square error, structural similarity index, normalized cross-correlation and other indicators between the pixel levels of the two crop images can be used to determine whether the two crop images are similar; 2. The crop images are converted into feature vectors, and then the distance or similarity measure between the two feature vectors is compared, such as Euclidean distance, cosine similarity, etc. The smaller the distance and the larger the cosine similarity, the more similar the two crop images are; 3. The deep learning model is used to learn the similarity between the two crop images. The similarity represents the matching value between the detection box and the fitting box. The crop image refers to the region of interest or the target object cropped out from the image to be processed.

[0148] After matching the detection frame with the fitting frame using multiple methods, matching values ​​corresponding to the multiple methods can be obtained.

[0149] In step S52, a plurality of the matching values ​​are fused to obtain a target matching value between the detection frame and the fitting frame.

[0150] You can set the weight corresponding to each matching method, and then multiply the matching value obtained by each method by the weight corresponding to each method to get the target value, and finally add multiple target values ​​to get the target matching value.

[0151] For example, the weights corresponding to the matching methods (1), (2), (3), and (4) in the above step S51 can be set to 0.25, 0.25, 0.25, and 0.25 respectively, and the matching values ​​can be set to 5, 10, 10, and 5 respectively. Then the final target matching value is equal to 5*0.25+10*0.25+10*0.25+5*0.25.

[0152] In step S53, the detection frame and the fitting frame corresponding to the maximum matching value are screened out from the target matching values ​​between at least one of the detection frames and at least one of the fitting frames.

[0153] See also Figure 5 As shown, each obstacle in the image to be processed has its own corresponding detection frame and fitting frame, see Figure 5 As shown in FIG. 1 , if the detection frame has detection frames 1 to 4 and the fitting frame has fitting frames 5 to 8, the fitting frame corresponding to each detection frame and the target matching value between the detection frame and the fitting frame can be obtained through the above multiple matching methods. Figure 5 As shown in FIG. 1 , a detection frame and a fitting frame connected by a straight line represent that the two match each other; a detection frame and a fitting frame not connected by a straight line represent that the two do not match each other.

[0154] See also Figure 5 As shown, the fitting frames that match the detection frame 1 are fitting frames 5 and 7, the fitting frames that match the detection frame 2 are fitting frames 5, the fitting frames that match the detection frame 3 are fitting frames 5 and 6, and the fitting frames that match the detection frame 4 are fitting frames 7 and 8. At this time, Hungarian matching can be used for multiple detection frames and multiple fitting frames to obtain the detection frame and fitting frame corresponding to the maximum matching value. For example, see Figure 5 As shown, it can be obtained that detection frame 1 and fitting frame 7 have the maximum matching value, detection frame 2 and detection frame 5 have the maximum matching value, detection frame 3 and detection frame 6 have the maximum matching value, and detection frame 4 and detection frame 8 have the maximum matching value.

[0155] In step S54, the fitting frame corresponding to the maximum matching value is used to correct the detection frame corresponding to the maximum matching value to obtain the target detection frame.

[0156] The detection frame and fitting frame corresponding to the maximum matching value indicate that the obstacle represented by the two is the same. In this case, the fitting frame representing the same obstacle can be used to correct the detection frame to obtain the target detection frame.

[0157] During correction, the target detection frame can be obtained according to the detection frame parameters of the detection frame, the first weights corresponding to the detection frame parameters, the fitting frame parameters of the fitting frame, and the second weights corresponding to the fitting frame parameters.

[0158] The detection frame parameters include the position and size of the detection frame, and the fitting frame parameters include the position and size of the fitting frame. Therefore, the obtained target detection frame includes the target detection frame parameters of the target detection frame, and the target detection frame parameters include the position and size of the target detection frame. It can be understood that the position includes but is not limited to the vertex coordinates and center coordinates of the detection frame or fitting frame, and the size includes but is not limited to the length, width and height of the detection frame or fitting frame.

[0159] When calculating the target detection box parameters, the detection box parameters may be multiplied by the first weight, and the fitting box parameters may be added to the second weight to obtain the target detection box parameters.

[0160] For example, if the width of the detection box is 5, the first weight is 0.2, the width of the fitting box is 6, and the second weight is 0.8, then the width of the final target detection box is 5*0.2+6*0.8, which is equal to 5.8.

[0161] When the first weight is 0 and the second weight is 1, it can be regarded as replacing the detection box parameters of the detection box with the fitting box parameters of the fitting box.

[0162] When the first weight is 0.5 and the second weight is also 0.5, it can be regarded as a weighted average of the detection box parameters of the detection box and the fitting box parameters of the fitting box.

[0163] Through the above technical solution, a more accurate fitting frame can be used to correct the detection frame, so that the obtained target detection frame is obtained by integrating the fitting frame parameters of the fitting frame, and the naturally obtained target detection frame is more accurate.

[0164] Figure 6 is a block diagram of a detection frame determination device according to an exemplary embodiment. Figure 6 The detection frame determination device 600 includes a determination module 610 and a correction module 620.

[0165] The determination module 610 is configured to determine a detection frame and a fitting frame corresponding to an obstacle in the image to be processed, wherein the detection frame and the fitting frame are used to locate the obstacle, the detection frame is a detection frame output by an obstacle detection model, and the fitting frame is a fitting frame obtained by fitting according to the size of the obstacle;

[0166] The correction module 620 is configured to use the fitting frame to correct the detection frame to obtain a target detection frame.

[0167] Optionally, the determination module 610 includes:

[0168] A grounding point determination submodule is configured to determine a grounding point of the obstacle, where the grounding point is a contact point between the obstacle and the ground;

[0169] The fitting submodule is configured to obtain the fitting frame according to the grounding point.

[0170] Optionally, the grounding point determination submodule includes:

[0171] The prediction submodule is configured to use the image to be processed as an input parameter of a ground point detection model to obtain the ground point.

[0172] Optionally, the detection frame determining device 600 includes:

[0173] A first screening module is configured to screen out a target obstacle whose distance from the image acquisition device is less than a first preset distance from a plurality of obstacles in the image sample acquired by the image acquisition device;

[0174] The training module is configured to use the image samples as training samples, the grounding point of each target obstacle, the position corresponding to the grounding point, and the category of the grounding point as labels, train the first initial model, and obtain the grounding point detection model.

[0175] Optionally, the obstacle includes a first obstacle; and the fitting submodule includes:

[0176] A first fitting submodule is configured to obtain the length of the first obstacle by fitting two end points among the plurality of grounding points of the first obstacle;

[0177] A second fitting submodule is configured to obtain the width of the first obstacle by fitting according to a first grounding point among the plurality of grounding points, the distance between the first grounding point and the long side of the first obstacle being less than a second preset distance;

[0178] The first fitting frame fitting submodule is configured to fit a fitting frame corresponding to the first obstacle according to the length and width of the first obstacle and the standard height of the first obstacle.

[0179] Optionally, the obstacle includes a second obstacle; and the fitting submodule includes:

[0180] a third fitting submodule, configured to obtain a center point of the second obstacle according to positions of a plurality of grounding points of the second obstacle;

[0181] The second fitting frame fitting submodule is configured to fit a fitting frame corresponding to the second obstacle according to the center point, the standard length of the second obstacle, the standard width of the second obstacle, and the standard height of the second obstacle.

[0182] Optionally, the detection frame includes the size of the obstacle; the determination module 610 includes:

[0183] a size determination submodule, configured to determine a size range of the obstacle and a standard size of the obstacle;

[0184] The first updating submodule is configured to update the predicted size of the obstacle output by the obstacle detection model to the standard size when the predicted size is outside the size range.

[0185] Optionally, the size of the obstacle includes a predicted width of the obstacle, and the size range includes a width range; the determination module 610 includes:

[0186] The second updating submodule is configured to update the width of the obstacle to a standard width when the predicted width of the obstacle output by the obstacle detection model is outside the width range.

[0187] Optionally, the size of the obstacle includes a predicted length of the obstacle, and the size range includes a length range; the determination module 610 includes:

[0188] A fourth fitting submodule is configured to obtain the predicted length by fitting according to the predicted width;

[0189] The third updating submodule is configured to update the length of the obstacle to a standard length when the predicted length is outside the length range.

[0190] Optionally, the size includes a predicted height of the obstacle, and the size range includes a height range; the determination module 610 includes:

[0191] a fifth fitting submodule, configured to obtain the predicted height by fitting according to the predicted width;

[0192] The fourth updating submodule is configured to update the height of the obstacle to a standard height when the predicted height is outside the height range.

[0193] Optionally, the correction module 620 includes:

[0194] The first correction submodule is configured to obtain the target detection frame according to the detection frame parameters of the detection frame, the first weights corresponding to the detection frame parameters, the fitting frame parameters of the fitting frame, and the second weights corresponding to the fitting frame parameters.

[0195] Optionally, the correction module 620 includes:

[0196] A second screening submodule is configured to screen out the detection frame and the fitting frame corresponding to the maximum matching value from the target matching values ​​between at least one of the detection frames and at least one of the fitting frames;

[0197] The second correction submodule is configured to use the fitting frame corresponding to the maximum matching value to correct the detection frame corresponding to the maximum matching value to obtain the target detection frame.

[0198] Optionally, the detection frame determining device 600 includes:

[0199] A matching submodule is configured to match the detection frame with the fitting frame based on a plurality of different matching modes to obtain a plurality of matching values;

[0200] The fusion submodule is configured to fuse the multiple matching values ​​to obtain a target matching value between the detection box and the fitting box.

[0201] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0202] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and when the program instructions are executed by a processor, the steps of the detection frame determination method provided by the present disclosure are implemented.

[0203] Figure 7 is a block diagram of a vehicle 700 according to an exemplary embodiment. For example, the vehicle 700 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. The vehicle 700 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0204] Reference Figure 7 , the vehicle 700 may include various subsystems, for example, an infotainment system 710, a perception system 720, a decision control system 730, a drive system 740, and a computing platform 750. The vehicle 700 may also include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and each component of the vehicle 700 may be interconnected by wire or wireless means.

[0205] In some embodiments, the infotainment system 710 may include a communication system, an entertainment system, and a navigation system, etc.

[0206] The perception system 720 may include several sensors for sensing information about the environment around the vehicle 700. For example, the perception system 720 may include a global positioning system (the global positioning system may be a GPS system, or a Beidou system or other positioning systems), an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0207] The decision control system 730 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0208] The drive system 740 may include components that provide power movement for the vehicle 700. In one embodiment, the drive system 740 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, and an air compression engine. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0209] Some or all functions of the vehicle 700 are controlled by a computing platform 750. The computing platform 750 may include at least one processor 751 and a memory 752, and the processor 751 may execute instructions 753 stored in the memory 752.

[0210] The processor 751 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0211] The memory 752 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0212] In addition to instructions 753 , memory 752 may also store data, such as road maps, route information, vehicle location, direction, speed, etc. The data stored in memory 752 may be used by computing platform 750 .

[0213] In the embodiment of the present disclosure, the processor 751 may execute instruction 753 to complete all or part of the steps of the above-mentioned detection frame determination method.

[0214] Figure 8 8 is a block diagram of a device 800 for determining a detection frame according to an exemplary embodiment. The detection frame determining device 800 may be a system-on-a-chip (SOC). The detection frame determining device 800 may be configured on a Figure 7 The vehicle 700 shown may also be connected to the vehicle 700 .

[0215] Reference Figure 8 The apparatus 800 includes a processing component 822, which further includes one or more processors, and a memory resource represented by a memory 832 for storing instructions executable by the processing component 822, such as an application. The application stored in the memory 832 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 822 is configured to execute instructions to perform the above-mentioned detection frame determination method.

[0216] The device 800 may also include a power supply component 826 configured to perform power management of the device 800, a wired or wireless network interface 850 configured to connect the device 800 to a network, and an input / output interface 858. The device 800 may operate based on an operating system stored in the memory 832.

[0217] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above detection frame determination method when executed by the programmable device.

[0218] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0219] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for determining a detection frame, characterized in that: include: Determine a detection frame and a fitting frame corresponding to an obstacle in the image to be processed, wherein the detection frame and the fitting frame are used to locate the obstacle, the detection frame is a detection frame output by an obstacle detection model, and the fitting frame is a fitting frame obtained by fitting according to the size of the obstacle; The detection frame is corrected using the fitting frame to obtain a target detection frame.

2. The method according to claim 1, characterized in that The step of determining a fitting frame corresponding to an obstacle in the image to be processed includes: Determine a grounding point of the obstacle, where the grounding point is a contact point between the obstacle and the ground; The fitting frame is obtained according to the grounding point.

3. The method according to claim 2, characterized in that The determining the grounding point of the obstacle includes: The image to be processed is used as an input parameter of a ground point detection model to obtain the ground point.

4. The method according to claim 3, characterized in that The grounding point detection model is trained by the following steps: Screening out, from a plurality of obstacles in the image samples acquired by the image acquisition device, a target obstacle whose distance from the image acquisition device is less than a first preset distance; The image samples are used as training samples, and the grounding point of each target obstacle, the position corresponding to the grounding point, and the category of the grounding point are used as labels to train the first initial model to obtain the grounding point detection model.

5. The method according to claim 2, characterized in that: The obstacle includes a first obstacle; and obtaining the fitting frame according to the grounding point includes: According to two end points of the plurality of grounding points of the first obstacle, fitting to obtain the length of the first obstacle; According to a first grounding point among the plurality of grounding points, the distance between the first grounding point and the long side of the first obstacle being less than a second preset distance, fitting to obtain the width of the first obstacle; A fitting frame corresponding to the first obstacle is obtained by fitting according to the length and width of the first obstacle and the standard height of the first obstacle.

6. The method according to claim 2, characterized in that The obstacle includes a second obstacle; and obtaining the fitting frame according to the grounding point includes: Obtaining a center point of the second obstacle according to positions of a plurality of grounding points of the second obstacle; A fitting frame corresponding to the second obstacle is obtained by fitting according to the central point, the standard length of the second obstacle, the standard width of the second obstacle, and the standard height of the second obstacle.

7. The method according to claim 1, characterized in that The detection frame includes the size of the obstacle; The step of determining a detection frame corresponding to an obstacle in the image to be processed includes: Determining a size range of the obstacle and a standard size of the obstacle; When the predicted size of the obstacle output by the obstacle detection model is outside the size range, the predicted size is updated to the standard size.

8. The method according to claim 7, characterized in that The size of the obstacle includes a predicted width of the obstacle, and the size range includes a width range; when the predicted size of the obstacle output by the obstacle detection model is outside the size range, updating the predicted size to the standard size includes: When the predicted width of the obstacle output by the obstacle detection model is outside the width range, the width of the obstacle is updated to a standard width.

9. The method according to claim 8, characterized in that The size of the obstacle includes a predicted length of the obstacle, and the size range includes a length range; when the predicted size of the obstacle output by the obstacle detection model is outside the size range, updating the predicted size to the standard size includes: According to the predicted width, fitting is performed to obtain the predicted length; When the predicted length is outside the length range, the length of the obstacle is updated to a standard length.

10. The method according to claim 8, characterized in that The size includes a predicted height of the obstacle, and the size range includes a height range; when the predicted size of the obstacle output by the obstacle detection model is outside the size range, updating the predicted size to the standard size includes: According to the predicted width, fitting to obtain the predicted height; When the predicted height is outside the height range, the height of the obstacle is updated to a standard height.

11. The method according to claim 1, characterized in that: The step of correcting the detection frame by using the fitting frame to obtain the target detection frame includes: The target detection frame is obtained according to the detection frame parameters of the detection frame, the first weights corresponding to the detection frame parameters, the fitting frame parameters of the fitting frame, and the second weights corresponding to the fitting frame parameters.

12. The method according to claim 1, characterized in that The step of correcting the detection frame by using the fitting frame to obtain the target detection frame includes: Filtering out the detection frame and the fitting frame corresponding to the maximum matching value from the target matching values ​​between at least one of the detection frames and at least one of the fitting frames; The fitting frame corresponding to the maximum matching value is used to correct the detection frame corresponding to the maximum matching value to obtain the target detection frame.

13. The method according to claim 12, characterized in that The method further comprises: Matching the detection frame with the fitting frame based on a plurality of different matching methods to obtain a plurality of matching values; A plurality of the matching values ​​are fused to obtain a target matching value between the detection frame and the fitting frame.

14. A detection frame determination device, characterized in that: include: A determination module is configured to determine a detection frame and a fitting frame corresponding to an obstacle in the image to be processed, wherein the detection frame and the fitting frame are used to locate the obstacle, the detection frame is a detection frame output by an obstacle detection model, and the fitting frame is a fitting frame obtained by fitting according to the size of the obstacle; The correction module is configured to use the fitting frame to correct the detection frame to obtain a target detection frame.

15. A detection frame determination device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Execute the steps of the method according to any one of claims 1 to 13.

16. A vehicle, characterized in that: The vehicle is provided with the detection frame determination device according to claim 15, or the vehicle is connected to the detection frame determination device according to claim 15.

17. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images