Travelable area boundary point processing method, device, equipment, medium and product

Abstract

The embodiment of the invention provides a travelable area boundary point processing method and device, equipment, a medium and a product. According to the method, after a complete boundary point set and a partial boundary point set of a current frame of a vehicle body environment image are obtained, fan-shaped division is carried out on the current frame of the vehicle body environment image by taking an image center point as a vertex, and a plurality of first grids are obtained; and according to each first grid, carrying out screening and fusion processing on the complete boundary point set and the partial boundary point set to obtain a target boundary point. According to the scheme, the complete boundary point set and the partial boundary point set are subjected to screening and fusion processing by using the grids, so that the number of obtained target boundary points is large, the requirement of generating a drivable area can be met, the information of the target boundary points comprises image coordinates and categories, and the integrity of boundary point data is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a method, device, equipment, medium and product for processing boundary points of a drivable area. Background Art

[0002] With the continuous development of intelligent driving technology, vehicles can perform automatic driving or path planning based on the determined drivable area. The vehicle can detect the vehicle body environment image, which is a top view with the vehicle as the center point. After detecting the boundary points of the drivable area, the valid points are connected in sequence to form a drivable area.

[0003] In the prior art, the detection of boundary points in the drivable area is usually performed by using a deep neural network model to detect the vehicle body environment image. In one case, a complete boundary point set can be obtained, which is a set consisting of all boundary points in the drivable area, where the information of the boundary points only includes the image coordinates. In another case, a partial boundary point set can be obtained, which is a set consisting of some boundary points in the drivable area, where the information of the boundary points includes the image coordinates and categories. In both cases, boundary point data will be missing.

[0004] Therefore, there is an urgent need for a drivable area boundary point processing method that can process the obtained boundary points and improve the integrity of the boundary point data. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment, medium and product for processing boundary points of a drivable area, which are used to process the obtained boundary points and improve the integrity of the boundary point data.

[0006] In a first aspect, an embodiment of the present application provides a method for processing boundary points of a drivable area, comprising:

[0007] Acquire a complete boundary point set and a partial boundary point set of the vehicle body environment image of the current frame, wherein the information of each boundary point in the complete boundary point set includes image coordinates, and the information of each boundary point in the partial boundary point set includes image coordinates and categories;

[0008] Taking the center point of the image as a vertex, the vehicle body environment image of the current frame is divided into sectors to obtain a plurality of first grids;

[0009] According to each of the first grids, the complete boundary point set and the partial boundary point set are screened and fused to obtain target boundary points, and information of each target boundary point includes image coordinates and categories.

[0010] In a possible implementation, the screening and fusion processing of the complete boundary point set and the partial boundary point set according to each of the first grids to obtain the target boundary point includes:

[0011] According to each of the first grids, the complete boundary point set is screened to obtain a first boundary point;

[0012] According to each of the first grids, the partial boundary point set is screened to obtain second boundary points;

[0013] According to each of the first boundary points, each of the second boundary points, and each of the first grids, in a possible implementation manner, screening the complete boundary point set according to each of the first grids to obtain the first boundary point includes:

[0014] For each of the first grids, the following processing is performed:

[0015] Using the boundary points in the complete boundary point set belonging to the first grid as third boundary points;

[0016] The boundary point with the smallest distance from the center point among all the third boundary points is used as the first boundary point.

[0017] In a possible implementation manner, the screening of the partial boundary point set according to each of the first grids to obtain the second boundary point includes:

[0018] For each of the first grids, the following processing is performed:

[0019] The boundary points of the partial boundary points that belong to the first grid are concentrated as fourth boundary points;

[0020] The boundary point with the smallest distance from the center point among all the fourth boundary points is used as the second boundary point.

[0021] In a possible implementation manner, performing fusion processing according to each of the first boundary points, each of the second boundary points, and each of the first grids to obtain the target boundary point includes:

[0022] For each of the first grids, the following processing is performed:

[0023] If the first grid only includes the first boundary point, taking the first boundary point as the target boundary point, and determining the category of the target boundary point to be a preset category;

[0024] If the first grid only includes the second boundary point, taking the second boundary point as the target boundary point;

[0025] If the first grid includes the first boundary point and the second boundary point, a target boundary point is generated according to the first boundary point and the second boundary point, and the category of the target boundary point is determined to be the category of the second boundary point.

[0026] In a possible implementation manner, generating a target boundary point according to the first boundary point and the second boundary point includes:

[0027] If the distance between the first boundary point and the second boundary point is greater than a preset distance threshold, taking the boundary point between the first boundary point and the second boundary point that is closer to the center point as the target boundary point;

[0028] If the distance between the first boundary point and the second boundary point is less than or equal to the preset distance threshold, the midpoint between the first boundary point and the second boundary point is used as the target boundary point.

[0029] In a possible implementation manner, before obtaining the complete boundary point set and the partial boundary point set of the vehicle body environment image of the current frame, the method further includes:

[0030] Using a preset number of frames of historical images before the current frame of the vehicle body environment image and the current frame of the vehicle body environment image as images to be processed;

[0031] For each of the images to be processed, taking the center point of the image as a vertex, the image to be processed is divided into sectors to obtain a plurality of second grids corresponding to the image to be processed, wherein the angle of each of the second grids is greater than the angle of the first grid;

[0032] For each image to be processed, screening an initial part of the boundary point set of the image to be processed according to a plurality of second grids corresponding to the image to be processed, to obtain the boundary points to be mapped of the image to be processed;

[0033] The boundary points to be mapped of each of the images to be processed are mapped to the vehicle body environment image of the current frame to obtain the partial boundary point set.

[0034] In a second aspect, an embodiment of the present application provides a drivable area boundary point processing device, comprising:

[0035] An acquisition module, used to acquire a complete boundary point set and a partial boundary point set of the vehicle body environment image of the current frame, wherein the information of each boundary point in the complete boundary point set includes image coordinates, and the information of each boundary point in the partial boundary point set includes image coordinates and categories;

[0036] Processing modules for:

[0037] Taking the center point of the image as a vertex, the vehicle body environment image of the current frame is divided into sectors to obtain a plurality of first grids;

[0038] According to each of the first grids, the complete boundary point set and the partial boundary point set are screened and fused to obtain target boundary points, and information of each target boundary point includes image coordinates and categories.

[0039] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0040] Processor, memory, communication interface;

[0041] The memory is used to store executable instructions of the processor;

[0042] Wherein, the processor is configured to execute the drivable area boundary point processing method described in any one of the first aspects by executing the executable instructions.

[0043] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for processing boundary points of a drivable area as described in any one of the first aspects is implemented.

[0044] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the method for processing boundary points of a drivable area as described in any one of the first aspects.

[0045] The method, device, equipment, medium and product for processing boundary points of a drivable area provided in the embodiment of the present application obtain the complete boundary point set and partial boundary point set of the current frame vehicle environment image, and then divide the current frame vehicle environment image into sectors with the center point of the image as the vertex to obtain multiple first grids; then, according to each first grid, the complete boundary point set and the partial boundary point set are screened and fused to obtain the target boundary point. This solution uses grids to screen and fuse the complete boundary point set and the partial boundary point set, so that the number of target boundary points obtained is large, which can meet the requirements of generating a drivable area, and the information of the target boundary point includes image coordinates and categories, which improves the integrity of the boundary point data. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] Figure 1a A flowchart of a method for processing boundary points of a drivable area provided in this application;

[0048] Figure 1b A schematic diagram of the vehicle body environment image of the current frame provided by this application;

[0049] Figure 1c A schematic diagram of the distribution of the complete set of boundary points provided for this application;

[0050] Figure 1d A schematic diagram of the distribution of effective points provided for this application;

[0051] Figure 1e A schematic diagram of the distribution of some boundary point sets provided for this application;

[0052] Figure 1f A schematic diagram of the first grid provided for this application;

[0053] Figure 1g A schematic diagram of the process of determining the first boundary point provided for this application;

[0054] Figure 2a A flowchart of a second embodiment of a method for processing boundary points of a drivable area provided in the present application;

[0055] Figure 2b Schematic diagram 1 of the process of determining the target boundary point provided for this application;

[0056] Figure 2c Schematic diagram 2 of the process of determining the target boundary point provided for this application;

[0057] Figure 3 A flowchart of Embodiment 3 of the method for processing boundary points of a drivable area provided in this application;

[0058] Figure 4 A schematic diagram of the structure of an embodiment of a drivable area boundary point processing device provided in the present application;

[0059] Figure 5 A schematic diagram of the structure of an electronic device provided in this application.

[0060] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0061] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0062] With the continuous development of intelligent driving technology, vehicles can perform automatic driving or path planning based on the drivable area in the vehicle environment image. The vehicle environment image is a top view with the vehicle located at the center of the image, which can be obtained by performing inverse perspective transformation based on images taken by multiple surround view cameras.

[0063] The vehicle can detect the body environment image, and after detecting the boundary points of the drivable area, the boundary points include valid points and invalid points. The valid points are connected in sequence to form the drivable area, and the invalid points cannot be used to form the drivable area.

[0064] In the prior art, the detection of boundary points of the drivable area is usually performed by using a deep neural network model to detect the vehicle body environment image. In one case, a complete boundary point set can be obtained, which is a set of all boundary points of the drivable area, where the information of the boundary points only includes the image coordinates. The drivable area can be generated based on the complete boundary point set, but there is no category of the boundary points.

[0065] In another case, a partial boundary point set can be obtained, which is a set of partial boundary points of the drivable area, wherein the information of the boundary points includes image coordinates and categories. Since the number of boundary points in the partial boundary point set is small, the drivable area cannot be formed. The categories of boundary points can be curbs, fences, vehicles, pedestrians, walls, etc.

[0066] In both cases, boundary point data will be missing. Therefore, a drivable area boundary point processing method is urgently needed to process the obtained boundary points and improve the integrity of the boundary point data.

[0067] In view of the problems existing in the prior art, the inventors found in the process of studying the method for processing the boundary points of the drivable area that there are a large number of boundary points in the complete boundary point set, which can generate a drivable area. The boundary point information in some boundary points includes categories. Therefore, in order to be able to generate a drivable area with boundary points of categories in the future and reduce the amount of calculation for generating the drivable area, the front frame vehicle body environment image can be divided into sectors to obtain multiple first grids, and then the complete boundary point set and the partial boundary point set are screened and fused according to each first grid to obtain the target boundary point. The information of each target boundary point includes image coordinates and categories. Based on the above-mentioned inventive concept, the drivable area boundary point processing scheme in this application is designed.

[0068] The executor of the method for processing boundary points of the drivable area in the present application may be a vehicle-mounted terminal, or a computer, a server, a controller, etc. The present application does not limit it and the following description will be made using a vehicle-mounted terminal as an example.

[0069] The following is an example of an application scenario of the method for processing boundary points of a drivable area provided in this application.

[0070] For example, in this application scenario, when the vehicle is about to park, the vehicle terminal takes an image through a surround view camera, and after inverse perspective transformation, obtains the current frame vehicle environment image. Then, the current frame vehicle environment image is detected using a deep neural network model to obtain a complete boundary point set and a partial boundary point set. The information of each boundary point in the complete boundary point set includes image coordinates, and the information of each boundary point in the partial boundary point set includes image coordinates and categories.

[0071] The vehicle-mounted terminal divides the vehicle body environment image of the current frame into sectors with the center point of the image as the vertex to obtain a plurality of first grids.

[0072] Then, according to each first grid, the complete boundary point set and the partial boundary point set are screened and fused to obtain target boundary points, and the information of each target boundary point includes image coordinates and categories.

[0073] The vehicle-mounted terminal can then generate a drivable area based on the image coordinates of the target boundary points, and control the vehicle to drive in the drivable area based on the category of the target boundary points to complete parking.

[0074] It should be noted that the above scenario is only an example of an application scenario provided by an embodiment of the present application. The embodiment of the present application does not limit the actual form of the various devices included in the scenario, nor does it limit the interaction method between the devices. In the specific application of the solution, it can be set according to actual needs.

[0075] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0076] Figure 1a This is a flow chart of the first embodiment of the method for processing boundary points of the drivable area provided in the present application. In the present embodiment, the vehicle terminal divides the vehicle environment image of the current frame into sectors to obtain the first grid, and then screens and fuses the complete boundary point set and the partial boundary point set according to the first grid to obtain the target boundary point. The method in this embodiment can be implemented by software, hardware, or a combination of software and hardware. Figure 1a As shown, the method for processing the boundary points of the drivable area specifically includes the following steps:

[0077] S101: Acquire a complete boundary point set and a partial boundary point set of a vehicle body environment image of a current frame.

[0078] In this step, in order to improve the integrity of the boundary point data, it is necessary to first obtain the complete boundary point set and the partial boundary point set of the current frame vehicle environment image. The information of each boundary point in the complete boundary point set includes image coordinates, and the information of each boundary point in the partial boundary point set includes image coordinates and categories.

[0079] For example, Figure 1b This is a schematic diagram of the current frame vehicle environment image provided by this application, such as Figure 1b As shown, the vehicle is located at the center of the vehicle body environment image in the current frame, there are walls on the left and rear of the vehicle, there are two stone pillars in front of the vehicle, and there is a vehicle in front of the left of the vehicle.

[0080] For example, in Figure 1b On the basis of Figure 1c A schematic diagram of the distribution of the complete boundary point set provided for this application, such as Figure 1c As shown, the boundary points in the complete boundary point set are the location points of the wall, stone pillars and vehicles. The boundary point information only has image coordinates and no category. The boundary points in the complete boundary point set include valid points and invalid points. For example, Figure 1c On the basis of Figure 1d The distribution diagram of the effective points provided for this application is as follows: Figure 1d As shown in the figure, the black dots are valid points. By connecting the valid points in sequence, the drivable area can be obtained, but there is no category information of the boundary points.

[0081] For example, in Figure 1b On the basis of Figure 1e A schematic diagram of the distribution of some boundary point sets provided for this application, such as Figure 1eAs shown in the figure, the boundary points in some boundary point sets are the locations of stone pillars and vehicles. The boundary point information includes image coordinates and categories. The category of the boundary point in front of the vehicle is stone pillar, and the category of the boundary point in front of the left side of the vehicle is vehicle. The boundary points in some boundary point sets are valid and invalid. Since the number of boundary points is small, they cannot form a drivable area.

[0082] It should be noted that the complete boundary point set and partial boundary point set of the current frame vehicle environment image can be obtained by detecting the current frame vehicle environment image using a deep neural network model, or can be obtained by detecting multiple surround view camera images respectively using a deep neural network model to obtain the complete boundary point set and the partial boundary point set to be spliced. The complete boundary point set to be spliced ​​is spliced ​​to obtain the complete boundary point set, and the partial boundary point set to be spliced ​​is spliced ​​to obtain the partial boundary point set.

[0083] S102: Taking the center point of the image as a vertex, the vehicle body environment image of the current frame is divided into sectors to obtain a plurality of first grids.

[0084] In this step, after the vehicle-mounted terminal obtains the complete boundary point set and partial boundary point set of the current frame vehicle body environment image, in order to filter out the valid points in the complete boundary point set and the partial boundary point set, and in order to obtain the category of the boundary points, it is necessary to divide the current frame vehicle body environment image into sectors with the image center point as the vertex to obtain multiple first grids.

[0085] For example, Figure 1f The first grid schematic diagram provided for this application is as follows: Figure 1f As shown in the figure, each sector area is a first grid, the vertex of each first grid is the center point of the image, which is also the center point of the vehicle, and the angle of each first grid is the same.

[0086] It should be noted that when the first grid is obtained by sector division, the division can be performed according to a preset first angle, that is, the angle of each first grid is the preset first angle. The preset first angle can be 10 degrees, 20 degrees, 30 degrees, etc.

[0087] S103: According to each first grid, the complete boundary point set and the partial boundary point set are screened and merged to obtain target boundary points.

[0088] In this step, after obtaining the first grid, the vehicle terminal screens and fuses the complete boundary point set and the partial boundary point set according to each first grid to obtain the target boundary point, and the information of each target boundary point includes image coordinates and category.

[0089] Specifically, according to each first grid, the complete boundary point set is screened to obtain the first boundary point.

[0090] That is, the following processing is performed for each first grid:

[0091] The boundary points in the complete boundary point set belonging to the first grid are used as the third boundary points;

[0092] The boundary point with the smallest distance from the center point among all the third boundary points is used as the first boundary point.

[0093] For example, Figure 1g A schematic diagram of the process of determining the first boundary point provided for this application is as follows: Figure 1g As shown, after obtaining the third boundary point, the vehicle-mounted terminal calculates the distance between each third boundary point and the center point. Then, for each third boundary point, it is determined whether the third boundary point is the boundary point with the smallest distance from the center point among all the third boundary points; if the third boundary point is the boundary point with the smallest distance from the center point among all the third boundary points, the third boundary point is used as the first boundary point; if the third boundary point is not the boundary point with the smallest distance from the center point among all the third boundary points, the third boundary point is discarded.

[0094] According to each first grid, a part of the boundary point set is screened to obtain second boundary points.

[0095] That is, the following processing is performed for each first grid:

[0096] The boundary points of the first grid in the partial boundary points collection are used as the fourth boundary points;

[0097] The boundary point with the smallest distance from the center point among all the fourth boundary points is used as the second boundary point.

[0098] After the first boundary point and the second boundary point are obtained, a fusion process is performed according to each first boundary point, each second boundary point and each first grid to obtain a target boundary point.

[0099] The method for processing boundary points of a drivable area provided in this embodiment obtains the complete boundary point set and the partial boundary point set of the current frame vehicle environment image, and then divides the current frame vehicle environment image into sectors with the center point of the image as the vertex to obtain multiple first grids; then, according to each first grid, the complete boundary point set and the partial boundary point set are screened and fused to obtain the target boundary point. This solution uses grids to screen and fuse the complete boundary point set and the partial boundary point set, so that the number of target boundary points obtained is large, which can meet the requirements of generating a drivable area, and the information of the target boundary point includes image coordinates and categories, which improves the integrity of the boundary point data.

[0100] Figure 2aThis is a flow chart of the second embodiment of the method for processing the boundary points of the drivable area provided by the present application. Based on the above embodiment, the present embodiment of the present application describes the situation in which the vehicle terminal performs fusion processing according to the first boundary point, the second boundary point and the first grid to obtain the target boundary point. Figure 2a As shown, the following steps are performed on each first grid:

[0101] S201: Determine whether the first grid includes the first boundary point; if the first grid includes the first boundary point, execute steps S202 to S204; if the first grid does not include the first boundary point, execute steps S205 to S207.

[0102] In this step, after the vehicle-mounted terminal obtains the first boundary point and the second boundary point, in order to determine the target boundary point, it is necessary to first determine whether the first grid includes the first boundary point.

[0103] S202: Determine whether the first grid includes the second boundary point; if the first grid includes the second boundary point, execute step S203; if the first grid does not include the second boundary point, execute step S204.

[0104] In this step, if the vehicle-mounted terminal determines that the first grid includes the first boundary point, it is still necessary to continue to determine whether the first grid includes the second boundary point.

[0105] S203: Generate a target boundary point according to the first boundary point and the second boundary point, and determine the category of the target boundary point to be the category of the second boundary point.

[0106] In this step, if the vehicle-mounted terminal determines that the first grid includes the second boundary point, it means that the first grid includes both the first boundary point and the second boundary point, and the number of the first boundary point and the second boundary point included is one. In a grid, at most only one boundary point is retained as a valid point, that is, the target boundary point, and the drivable area can be generated, so the target boundary point is generated according to the first boundary point and the second boundary point, and the category of the target boundary point is determined to be the category of the second boundary point.

[0107] Specifically, if the distance between the first boundary point and the second boundary point is greater than the preset distance threshold, it means that the distance between the two boundary points is far, and the boundary point closer to the center point is a valid point, then the boundary point closer to the center point between the first boundary point and the second boundary point is used as the target boundary point.

[0108] For example, Figure 2b A schematic diagram of the process of determining the target boundary point provided for this application is shown in FIG. Figure 2bAs shown, the first grid includes both the first boundary point and the second boundary point, point a is the first boundary point, point b is the second boundary point, the distance between the first boundary point and the second boundary point is greater than the preset distance threshold, point b is closer to the center point, and point b is taken as the target boundary point.

[0109] If the distance between the first boundary point and the second boundary point is less than or equal to the preset distance threshold, it means that the distance between the two boundary points is close. In order to make the drivable area smoother, the midpoint between the first boundary point and the second boundary point is used as the target boundary point.

[0110] For example, Figure 2c The second schematic diagram of the process of determining the target boundary point provided for this application is as follows: Figure 2c As shown, the first grid includes both the first boundary point and the second boundary point, point a is the first boundary point, point b is the second boundary point, point c is the midpoint of the first boundary point and the second boundary point, the distance between the first boundary point and the second boundary point is less than the preset distance threshold, and point c is taken as the target boundary point.

[0111] It should be noted that the preset distance threshold may be 3 pixels, 4 pixels, 10 pixels, etc. The embodiment of the present application does not limit the preset distance threshold, and it may be determined according to actual conditions.

[0112] It should be noted that, when the distance between the first boundary point and the second boundary point is less than or equal to the preset distance threshold, the first boundary point or the second boundary point can also be used as the target boundary point.

[0113] S204: Taking the first boundary point as a target boundary point, and determining the category of the target boundary point as a preset category.

[0114] In this step, if the vehicle terminal determines that the first grid does not include the second boundary point, it means that the first grid only includes the first boundary point, and the number of the first boundary points included is one, then the first boundary point is used as the target boundary point, and the category of the target boundary point is determined to be the preset category.

[0115] It should be noted that the preset category can be unknown, wall, curb, etc.

[0116] S205: Determine whether the first grid includes the second boundary point; if the first grid includes the second boundary point, execute step S206; if the first grid does not include the second boundary point, execute step S207.

[0117] In this step, if the vehicle-mounted terminal determines that the first grid does not include the first boundary point, it is still necessary to continue to determine whether the first grid includes the second boundary point.

[0118] S206: Taking the second boundary point as the target boundary point.

[0119] In this step, if the vehicle terminal determines that the first grid includes a second boundary point, it means that the first grid only includes a second boundary point and the number of second boundary points included is one, then the second boundary point is used as the target boundary point, and the category of the target boundary point is the category of the second boundary point.

[0120] S207: End the processing of the first grid.

[0121] In this step, if the vehicle-mounted terminal determines that the first grid does not include the second boundary point, it means that the first grid does not include either the first boundary point or the second boundary point, so the processing of the first grid is terminated.

[0122] It should be noted that this embodiment can be processed in an image coordinate system, or in a vehicle coordinate system or a world coordinate system.

[0123] It should be noted that after the vehicle terminal obtains the target boundary point, since there is at most one target boundary point in each first grid, the target boundary points in the first grid can be connected in sequence in a clockwise or counterclockwise order to obtain a drivable area. The target boundary points in the first grid can also be sorted in a clockwise or counterclockwise order to obtain an ordered sequence of target boundary points.

[0124] The method for processing the boundary points of the drivable area provided in this embodiment determines the target boundary points according to whether the first boundary points and / or the second boundary points exist in the first grid, which not only improves the position accuracy of the target boundary points, but also determines the category of the target boundary points. In addition, the number of boundary points is reduced by screening, and the amount of calculation for generating the drivable area is also reduced.

[0125] Figure 3 This is a flow chart of the third embodiment of the method for processing boundary points of the drivable area provided by the present application. Based on the above embodiment, the present embodiment of the present application describes the situation where the vehicle terminal generates a partial boundary point set based on the historical image and the current frame vehicle body environment image. Figure 3 As shown, the method for processing the boundary points of the drivable area specifically includes the following steps:

[0126] S301: taking a preset number of frames of historical images before a current frame of vehicle body environment image and the current frame of vehicle body environment image as images to be processed.

[0127] In this step, in order to improve the accuracy of the target boundary points, the historical images and the current frame vehicle environment image can be used to generate a partial boundary point set. The vehicle terminal uses the preset number of historical images before the current frame vehicle environment image and the current frame vehicle environment image as the images to be processed.

[0128] It should be noted that the preset number can be 2, 5, 7, 10, etc. The embodiment of the present application does not limit the preset number and can be determined according to actual conditions.

[0129] S302: For each image to be processed, the image to be processed is divided into sectors with the center point of the image as a vertex to obtain a plurality of second grids corresponding to the image to be processed.

[0130] In this step, after the vehicle terminal obtains the image to be processed, in order to remove invalid points in the initial part of the boundary point set, it is necessary to divide each image to be processed into sectors with the image center point as the vertex to obtain multiple second grids corresponding to the image to be processed.

[0131] It should be noted that when the second grid is obtained by sector division, it can be divided according to the preset second angle, that is, the angle of each second grid is the preset second angle. The angle of each second grid is greater than the angle of the first grid, that is, the preset second angle is greater than the preset first angle, so the number of second grids is less than the number of first grids, and the number of invalid points that can be removed is greater. The preset second angle can be 45 degrees, 60 degrees, 90 degrees, etc.

[0132] S303: For each image to be processed, according to a plurality of second grids corresponding to the image to be processed, an initial part of the boundary point set of the image to be processed is screened to obtain the boundary points to be mapped of the image to be processed.

[0133] In this step, after the vehicle-mounted terminal obtains multiple second grids corresponding to each image to be processed, for each image to be processed, according to the multiple second grids corresponding to the image to be processed, the initial part of the boundary point set of the image to be processed is screened and processed, and invalid points are removed to obtain the boundary points to be mapped of the image to be processed.

[0134] Specifically, the following processing is performed on each second grid corresponding to each image to be processed:

[0135] The boundary points of the initial part of boundary points belonging to the second grid are concentrated as fifth boundary points;

[0136] The boundary point with the smallest distance from the center point among all the fifth boundary points is used as the boundary point to be mapped.

[0137] It should be noted that the initial partial boundary point set is a boundary point set obtained by detecting the processed image using a deep neural network model, wherein the information of each boundary point includes image coordinates and categories.

[0138] S304: Mapping the boundary points to be mapped of each image to be processed to the vehicle body environment image of the current frame to obtain a partial boundary point set.

[0139] In this step, after the vehicle terminal obtains the boundary points to be mapped of each image to be processed, in order to generate a partial boundary point set, the boundary points to be mapped of each image to be processed are mapped to the current frame vehicle body environment image to obtain a partial boundary point set.

[0140] It should be noted that the boundary points to be mapped of each image to be processed can be mapped to the world coordinate system, or mapped to the same vehicle coordinate system, and then mapped to the image coordinate system corresponding to the current frame vehicle environment image to obtain a partial boundary point set.

[0141] It should be noted that the initial partial boundary point set of the vehicle body environment image of the current frame may be used as the partial boundary point set.

[0142] The method for processing boundary points of the drivable area provided in this embodiment can improve the accuracy of some boundary point sets and thus improve the accuracy of target boundary points by removing invalid points from the initial boundary points of the historical image and the current frame vehicle environment image through a second grid and then mapping them to the current frame vehicle environment image.

[0143] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0144] Figure 4 This is a schematic diagram of the structure of an embodiment of the drivable area boundary point processing device provided in this application. Figure 4 As shown, the drivable area boundary point processing device 40 includes:

[0145] An acquisition module 41 is used to acquire a complete boundary point set and a partial boundary point set of a vehicle body environment image of a current frame, wherein the information of each boundary point in the complete boundary point set includes image coordinates, and the information of each boundary point in the partial boundary point set includes image coordinates and a category;

[0146] The processing module 42 is used for:

[0147] Taking the center point of the image as a vertex, the vehicle body environment image of the current frame is divided into sectors to obtain a plurality of first grids;

[0148] According to each of the first grids, the complete boundary point set and the partial boundary point set are screened and fused to obtain target boundary points, and information of each target boundary point includes image coordinates and categories.

[0149] Furthermore, the processing module 42 is specifically configured to:

[0150] According to each of the first grids, the complete boundary point set is screened to obtain a first boundary point;

[0151] According to each of the first grids, the partial boundary point set is screened to obtain second boundary points;

[0152] A fusion process is performed according to each of the first boundary points, each of the second boundary points, and each of the first grids to obtain the target boundary point.

[0153] Furthermore, the processing module 42 is further configured to:

[0154] For each of the first grids, the following processing is performed:

[0155] Using the boundary points in the complete boundary point set belonging to the first grid as third boundary points;

[0156] The boundary point with the smallest distance from the center point among all the third boundary points is used as the first boundary point.

[0157] Furthermore, the processing module 42 is further configured to:

[0158] For each of the first grids, the following processing is performed:

[0159] The boundary points of the partial boundary points that belong to the first grid are concentrated as fourth boundary points;

[0160] The boundary point with the smallest distance from the center point among all the fourth boundary points is used as the second boundary point.

[0161] Furthermore, the processing module 42 is further configured to:

[0162] For each of the first grids, the following processing is performed:

[0163] If the first grid only includes the first boundary point, taking the first boundary point as the target boundary point, and determining the category of the target boundary point to be a preset category;

[0164] If the first grid only includes the second boundary point, taking the second boundary point as the target boundary point;

[0165] If the first grid includes the first boundary point and the second boundary point, a target boundary point is generated according to the first boundary point and the second boundary point, and the category of the target boundary point is determined to be the category of the second boundary point.

[0166] Furthermore, the processing module 42 is further configured to:

[0167] If the distance between the first boundary point and the second boundary point is greater than a preset distance threshold, taking the boundary point between the first boundary point and the second boundary point that is closer to the center point as the target boundary point;

[0168] If the distance between the first boundary point and the second boundary point is less than or equal to the preset distance threshold, the midpoint between the first boundary point and the second boundary point is used as the target boundary point.

[0169] Furthermore, the processing module 42 is also used for:

[0170] Using a preset number of frames of historical images before the current frame of the vehicle body environment image and the current frame of the vehicle body environment image as images to be processed;

[0171] For each of the images to be processed, taking the center point of the image as a vertex, the image to be processed is divided into sectors to obtain a plurality of second grids corresponding to the image to be processed, wherein the angle of each of the second grids is greater than the angle of the first grid;

[0172] For each image to be processed, screening an initial part of the boundary point set of the image to be processed according to a plurality of second grids corresponding to the image to be processed, to obtain the boundary points to be mapped of the image to be processed;

[0173] The boundary points to be mapped of each of the images to be processed are mapped to the vehicle body environment image of the current frame to obtain the partial boundary point set.

[0174] The drivable area boundary point processing device provided in this embodiment is used to execute the technical solution in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0175] Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 5 As shown, the electronic device 50 includes:

[0176] Processor 51, memory 52, and communication interface 53;

[0177] The memory 52 is used to store executable instructions of the processor 51;

[0178] The processor 51 is configured to execute the technical solution in any of the aforementioned method embodiments by executing the executable instructions.

[0179] Optionally, the memory 52 may be independent or integrated with the processor 51 .

[0180] Optionally, when the memory 52 is a device independent of the processor 51, the electronic device 50 may further include:

[0181] The bus 54 , the memory 52 and the communication interface 53 are connected to the processor 51 via the bus 54 and communicate with each other. The communication interface 53 is used to communicate with other devices.

[0182] Optionally, the communication interface 53 may be implemented by a transceiver. The communication interface is used to implement communication between the database access device and other devices (such as a client, a read-write library, and a read-only library). The memory may include a random access memory (RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.

[0183] The bus 54 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0184] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0185] The electronic device is used to execute the technical solution in any of the aforementioned method embodiments, and its implementation principle and technical effect are similar and will not be repeated here.

[0186] An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the technical solution provided by any of the aforementioned method embodiments is implemented.

[0187] An embodiment of the present application also provides a computer program product, including a computer program, which is used to implement the technical solution provided by any of the aforementioned method embodiments when executed by a processor.

[0188] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for processing boundary points of a drivable area, characterized in that: include: Acquire a complete boundary point set and a partial boundary point set of the vehicle body environment image of the current frame, wherein the information of each boundary point in the complete boundary point set includes image coordinates, and the information of each boundary point in the partial boundary point set includes image coordinates and categories; Taking the center point of the image as a vertex, the vehicle body environment image of the current frame is divided into sectors to obtain a plurality of first grids; According to each of the first grids, the complete boundary point set and the partial boundary point set are screened and fused to obtain target boundary points, and information of each target boundary point includes image coordinates and categories.

2. The method according to claim 1, characterized in that The step of screening and fusing the complete boundary point set and the partial boundary point set according to each of the first grids to obtain target boundary points includes: According to each of the first grids, the complete boundary point set is screened to obtain a first boundary point; According to each of the first grids, the partial boundary point set is screened to obtain second boundary points; A fusion process is performed according to each of the first boundary points, each of the second boundary points, and each of the first grids to obtain the target boundary point.

3. The method according to claim 2, characterized in that The step of screening the complete boundary point set according to each of the first grids to obtain the first boundary points includes: For each of the first grids, the following processing is performed: Using the boundary points in the complete boundary point set belonging to the first grid as third boundary points; The boundary point with the smallest distance from the center point among all the third boundary points is used as the first boundary point.

4. The method according to claim 2, characterized in that: The step of screening the partial boundary point set according to each of the first grids to obtain the second boundary point comprises: For each of the first grids, the following processing is performed: The boundary points of the partial boundary points that belong to the first grid are concentrated as fourth boundary points; The boundary point with the smallest distance from the center point among all the fourth boundary points is used as the second boundary point.

5. The method according to claim 2, characterized in that: The step of performing fusion processing according to each of the first boundary points, each of the second boundary points, and each of the first grids to obtain the target boundary point includes: For each of the first grids, the following processing is performed: If the first grid only includes the first boundary point, taking the first boundary point as the target boundary point, and determining the category of the target boundary point to be a preset category; If the first grid only includes the second boundary point, taking the second boundary point as the target boundary point; If the first grid includes the first boundary point and the second boundary point, a target boundary point is generated according to the first boundary point and the second boundary point, and the category of the target boundary point is determined to be the category of the second boundary point.

6. The method according to claim 5, characterized in that The generating a target boundary point according to the first boundary point and the second boundary point includes: If the distance between the first boundary point and the second boundary point is greater than a preset distance threshold, taking the boundary point between the first boundary point and the second boundary point that is closer to the center point as the target boundary point; If the distance between the first boundary point and the second boundary point is less than or equal to the preset distance threshold, the midpoint between the first boundary point and the second boundary point is used as the target boundary point.

7. The method according to any one of claims 1 to 6, characterized in that: Before obtaining the complete boundary point set and the partial boundary point set of the vehicle body environment image of the current frame, the method further includes: Using a preset number of frames of historical images before the current frame of the vehicle body environment image and the current frame of the vehicle body environment image as images to be processed; For each of the images to be processed, taking the center point of the image as a vertex, the image to be processed is divided into sectors to obtain a plurality of second grids corresponding to the image to be processed, wherein the angle of each of the second grids is greater than the angle of the first grid; For each image to be processed, screening an initial part of the boundary point set of the image to be processed according to a plurality of second grids corresponding to the image to be processed, to obtain the boundary points to be mapped of the image to be processed; The boundary points to be mapped of each of the images to be processed are mapped to the vehicle body environment image of the current frame to obtain the partial boundary point set.

8. A device for processing boundary points of a drivable area, characterized in that: include: An acquisition module, used to acquire a complete boundary point set and a partial boundary point set of the vehicle body environment image of the current frame, wherein the information of each boundary point in the complete boundary point set includes image coordinates, and the information of each boundary point in the partial boundary point set includes image coordinates and categories; Processing modules for: Taking the center point of the image as a vertex, the vehicle body environment image of the current frame is divided into sectors to obtain a plurality of first grids; According to each of the first grids, the complete boundary point set and the partial boundary point set are screened and fused to obtain target boundary points, and information of each target boundary point includes image coordinates and categories.

9. An electronic device, characterized in that: include: Processor, memory, communication interface; The memory is used to store executable instructions of the processor; Wherein, the processor is configured to execute the drivable area boundary point processing method according to any one of claims 1 to 7 by executing the executable instructions.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for processing boundary points of a drivable area according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that It includes a computer program, which is used to implement the drivable area boundary point processing method according to any one of claims 1 to 7 when executed by a processor.

Images