Method, apparatus, storage medium, and computer device for generating road edge lines

Abstract

The present application discloses a method, apparatus, storage medium, and computer device for generating a road edge line. The method includes: obtaining a first captured image; obtaining the number of multiple edge regions; determining multiple target edge regions within the multiple edge regions according to the number and a number threshold; obtaining the driving trajectory information of the vehicle; determining a first interval and a second interval within the first captured image according to the driving trajectory information; determining a first best edge region within the first interval and a second best edge region within the second interval according to the multiple target edge regions and a preset ratio threshold; calculating multiple first edge lines of the first best edge region and multiple second edge lines of the second best edge region; and generating a target edge line of the road according to the multiple first edge lines, the multiple second edge lines, and a first preset condition. The accuracy of extracting the road edge line of an irregular road is effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of road boundary detection, and particularly to a method, device, storage medium, and computer device for generating a road edge line. Background Art

[0002] In related technologies, algorithms for regular road edge detection mainly rely on obvious structures and linear characteristics in the road, such as lane line information, for processing. However, the road surface in a mining area is a dirt road or a gravel and soil block road surface. Such a road surface has no unified standard, and there are irregular retaining walls and cliff faults formed after mining at the road edge. Therefore, using regular road edge detection algorithms to detect the complex and irregular road environment in a mining area results in a low accuracy of the detected road edge results. Summary of the Invention

[0003] In view of this, the present application provides a method, device, storage medium, and computer device for generating a road edge line.

[0004] According to one aspect of the present application, a method for generating a road edge line is provided, including:

[0005] Responding to a generation request for a road edge line, and obtaining a first captured image;

[0006] Determining at least one edge region of the road according to a preset instance and the first captured image;

[0007] When there are multiple edge regions, obtaining the number of the multiple edge regions;

[0008] Determining multiple target edge regions within the multiple edge regions according to the number and a number threshold;

[0009] Obtaining the driving trajectory information of the vehicle;

[0010] Determining a first interval and a second interval within the first captured image according to the driving trajectory information;

[0011] Determining a first best edge region within the first interval and a second best edge region within the second interval according to the multiple target edge regions and a preset proportion threshold;

[0012] Calculating multiple first edge lines of the first best edge region and multiple second edge lines of the second best edge region;

[0013] Generating a target edge line of the road according to the multiple first edge lines, the multiple second edge lines, and a first preset condition.

[0014] Optionally, the step of determining at least one edge region of the road according to a preset instance and the first captured image specifically includes:

[0015] Obtain the second captured image;

[0016] Generate a training data set according to a preset instance and the second captured image;

[0017] Train the instance segmentation network according to the training data set to obtain pre-trained weights;

[0018] Infer the first captured image according to the pre-trained weights to generate at least one edge region;

[0019] Among them, the preset instance includes at least one of the following: retaining wall, polygonal mound, and cliff.

[0020] Optionally, the step of determining multiple target edge regions within multiple edge regions according to the quantity and the quantity threshold specifically includes:

[0021] If the quantity is less than or equal to the quantity threshold, determine that the multiple edge regions are multiple target edge regions;

[0022] If the quantity is greater than the quantity threshold, obtain multiple width values of the multiple edge regions, where the width value is the numerical value of the coincidence position between the multiple edge regions and the boundary of the first captured image;

[0023] Sort the multiple edge regions in descending order according to the multiple width values;

[0024] Determine multiple target edge regions within the sorted multiple edge regions according to the quantity, the quantity threshold, and the multiple width values.

[0025] Optionally, the step of determining the first interval and the second interval within the first captured image according to the driving trajectory information specifically includes:

[0026] Perform matrix conversion on the driving trajectory information to generate the first pixel point information of the vehicle on the road;

[0027] Determine the road center line of the first captured image according to the first pixel point information;

[0028] Divide the first captured image into a first interval and a second interval according to the road center line.

[0029] Optionally, the step of determining the first best edge region within the first interval and the second best edge region within the second interval according to the multiple target edge regions and the preset ratio threshold specifically includes:

[0030] Obtain the second pixel point information of each target edge region;

[0031] Calculate the first occupancy ratio of each target edge region within the first interval and the second occupancy ratio of each target edge region within the second interval according to the second pixel point information;

[0032] If the first occupancy ratio is greater than the preset occupancy ratio threshold, determine that the target edge region corresponding to the first occupancy ratio is within the first interval;

[0033] If the second occupancy ratio is greater than the preset occupancy ratio threshold, determine that the target edge region corresponding to the second occupancy ratio is within the second interval;

[0034] Obtain the number of target edge regions within the first interval;

[0035] When there are multiple target edge regions within the first interval, obtain the pixel positions of each target edge region on the first preset boundary line of the first acquisition image;

[0036] Sort the multiple target edge regions within the first interval in ascending order of pixel position;

[0037] Determine the first best edge region within the first interval according to the second preset condition;

[0038] Obtain the number of target edge regions within the second interval;

[0039] When there are multiple target edge regions within the second interval, obtain the pixel positions of each target edge region on the second preset boundary line of the first acquisition image;

[0040] Sort the multiple target edge regions within the second interval in ascending order of pixel position;

[0041] Determine the second best edge region within the second interval according to the second preset condition.

[0042] Optionally, the steps of calculating multiple first edge lines of the first best edge region and multiple second edge lines of the second best edge region specifically include:

[0043] Using the canny edge detection algorithm, extract the third pixel point information of the first best edge region and the fourth pixel point information of the second best edge region respectively to generate multiple first edge pixel point sets and multiple second edge pixel point sets;

[0044] Using the least squares method, perform second-order polynomial fitting on each first edge pixel point set and each second edge pixel point set respectively to generate the edge equation y = ax + b, where x is the abscissa of each pixel point, y is the ordinate of each pixel point, a is the slope, and b is the intercept;

[0045] Determine multiple first coordinate position points of the first optimal edge region and multiple second coordinate position points of the second optimal edge region according to the edge equation and the third preset condition;

[0046] Determine multiple corresponding first edge lines according to the multiple first coordinate position points;

[0047] Determine multiple corresponding second edge lines according to the multiple second coordinate position points.

[0048] Optionally, the method further includes: determining invalid pixel points according to the edge equation and a preset distance, and deleting the invalid pixel points within the optimal edge region.

[0049] According to another aspect of the present application, there is provided a device for generating a road edge line, including:

[0050] A first acquisition module, configured to acquire a first acquisition image in response to a request for generating a road edge line;

[0051] A first determination module, configured to determine at least one edge region of the road according to a preset instance and the first acquisition image;

[0052] A second acquisition module, configured to acquire the number of multiple edge regions when there are multiple at least one edge region;

[0053] A second determination module, configured to determine multiple target edge regions within the multiple edge regions according to the quantity and a quantity threshold;

[0054] A third acquisition module, configured to acquire the driving trajectory information of the vehicle;

[0055] A fourth determination module, configured to determine a first interval and a second interval within the first acquisition image according to the driving trajectory information;

[0056] A fifth determination module, configured to determine a first optimal edge region within the first interval and a second optimal edge region within the second interval according to the multiple target edge regions and a preset proportion threshold;

[0057] A calculation module, configured to calculate multiple first edge lines of the first optimal edge region and multiple second edge lines of the second optimal edge region;

[0058] A generation module, configured to generate a target edge line of the road according to the multiple first edge lines, the multiple second edge lines and a first preset condition.

[0059] Optionally, the first determination module is specifically configured to:

[0060] Acquire a second acquisition image;

[0061] Generate a training data set according to a preset instance and the second acquisition image;

[0062] Train the instance segmentation network according to the training data set to obtain pre-trained weights;

[0063] Infer the first captured image according to the pre-trained weights to generate at least one edge region;

[0064] Among them, the preset instances include at least one of the following: retaining wall, polygonal mound and cliff.

[0065] Optionally, the second determination module is specifically used for:

[0066] If the quantity is less than or equal to the quantity threshold, determine that the multiple edge regions are multiple target edge regions;

[0067] If the quantity is greater than the quantity threshold, obtain multiple width values of the multiple edge regions, where the width value is the numerical value of the coincidence position of the multiple edge regions and the boundary of the first captured image;

[0068] Sort the multiple edge regions in descending order according to the multiple width values;

[0069] Determine multiple target edge regions within the sorted multiple edge regions according to the quantity, quantity threshold and multiple width values.

[0070] Optionally, the third determination module is specifically used for:

[0071] Perform matrix conversion on the driving trajectory information to generate the first pixel point information of the vehicle on the road;

[0072] Determine the road center line of the first captured image according to the first pixel point information;

[0073] Divide the first captured image into a first interval and a second interval according to the road center line.

[0074] Optionally, the fourth determination module is specifically used for:

[0075] Obtain the second pixel point information of each target edge region;

[0076] Calculate the first occupancy ratio of each target edge region in the first interval and the second occupancy ratio of each target edge region in the second interval according to the second pixel point information;

[0077] If the first occupancy ratio is greater than the preset occupancy ratio threshold, determine that the target edge region corresponding to the first occupancy ratio is located in the first interval;

[0078] If the second occupancy ratio is greater than the preset occupancy ratio threshold, determine that the target edge region corresponding to the second occupancy ratio is located in the second interval;

[0079] Obtain the number of target edge regions within the first interval;

[0080] When there are multiple target edge regions within the first interval, obtain the pixel positions of each target edge region on the first preset boundary line of the first acquisition image;

[0081] Sort the multiple target edge regions within the first interval in ascending order of pixel positions;

[0082] Determine the first best edge region within the first interval according to the second preset condition;

[0083] Obtain the number of target edge regions within the second interval;

[0084] When there are multiple target edge regions within the second interval, obtain the pixel positions of each target edge region on the second preset boundary line of the first acquisition image;

[0085] Sort the multiple target edge regions within the second interval in ascending order of pixel positions;

[0086] Determine the second best edge region within the second interval according to the second preset condition.

[0087] Optionally, the calculation module is specifically configured to:

[0088] Use the canny edge detection algorithm to extract the third pixel point information of the first best edge region and the fourth pixel point information of the second best edge region respectively, and generate multiple first edge pixel point sets and multiple second edge pixel point sets;

[0089] Use the least squares method to perform second-order polynomial fitting on each first edge pixel point set and each second edge pixel point set respectively, and generate an edge equation y = ax + b, where x is the abscissa of each pixel point, y is the ordinate of each pixel point, a is the slope, and b is the intercept;

[0090] Determine multiple first coordinate position points of the first best edge region and multiple second coordinate position points of the second best edge region according to the edge equation and the third preset condition;

[0091] Determine the corresponding multiple first edge lines according to the multiple first coordinate position points;

[0092] Determine the corresponding multiple second edge lines according to the multiple second coordinate position points.

[0093] Optionally, the device for generating the road edge line further includes:

[0094] A sixth determination module, configured to determine invalid pixel points according to an edge equation and a preset distance, and delete the invalid pixel points within the optimal edge region.

[0095] With the above technical solutions, a method for generating a road edge line provided by the present application detects irregular retaining walls and cliff instances in an image as irregular road edge instances by using an instance segmentation algorithm, classifies and processes the irregular road instances, and then performs irregular road edge detection, so that the extracted road edge line is more applicable to the irregular road conditions in a mining area, effectively improving the accuracy and precision of the irregular road edge line.

[0096] The above description is only an overview of the technical solutions of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specifically illustrates the specific embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0098] Figure 1 A schematic flowchart of a method for generating a road edge line provided by an embodiment of the present application is shown;

[0099] Figure 2 Schematic diagrams of multiple target edge regions of a first captured image provided by an embodiment of the present application are shown;

[0100] Figure 3 Schematic diagrams of pixel points on a target edge line provided by an embodiment of the present application are shown;

[0101] Figure 4 Schematic diagrams of multiple edge lines of an optimal edge region of a first captured image provided by an embodiment of the present application are shown;

[0102] Figure 5 A schematic structural diagram of a device for generating a road edge line provided by an embodiment of the present application is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0103] The present application will be described in detail below with reference to the drawings and in conjunction with embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0104] In this embodiment, a method for generating a road edge line is provided. As Figure 1 shown, the method includes:

[0105] Step 102: In response to a generation request for a road edge line, obtain a first captured image;

[0106] Step 104: Determine at least one edge region of the road according to a preset instance and the first captured image;

[0107] Step 106: Determine whether the number of at least one edge region is multiple. If so, proceed to step 108; if not, proceed to step 118;

[0108] Step 108: Obtain the number of multiple edge regions;

[0109] Step 110: Determine multiple target edge regions within the multiple edge regions according to the number and a number threshold;

[0110] Step 112: Obtain the driving trajectory information of the vehicle;

[0111] Step 114: Determine a first interval and a second interval within the first captured image according to the driving trajectory information;

[0112] Step 116: Determine a first best edge region within the first interval and a second best edge region within the second interval according to the multiple target edge regions and a preset proportion threshold;

[0113] Step 118: Calculate multiple first edge lines of the first best edge region and multiple second edge lines of the second best edge region;

[0114] Step 120: Determine the target edge line of the road according to the multiple first edge lines, the multiple second edge lines and a first preset condition.

[0115] An embodiment of the present application proposes a method for generating a road edge line. Specifically, during the process of a vehicle driving autonomously in a mining area, in response to a generation request for a road edge line, a first captured image is obtained, where the first captured image refers to a real-time road image collected by a vehicle sensor system located at the front end of the vehicle. Subsequently, according to a preset instance and the collected real-time road image, at least one edge region within the first captured image is determined, where the preset instance refers to instances of road edge retaining walls, irregular soil piles, and cliffs generated due to mining in the mining area that need to be recognized as road edge regions. Further, according to these three instances of road edge retaining walls, irregular soil piles, and cliffs, the instance segmentation algorithm is used to infer the road image collected during the real-time driving process of the vehicle, and at least one edge region of the irregular road within the first captured image is recognized.

[0116] Further, since there are usually many irregular road edge areas during the vehicle driving process in the mining area, when it is determined that the edge areas of the irregular road are multiple edge areas, it is necessary to preprocess the multiple edge areas to determine the edge area closest to the vehicle driving trajectory on one side within the multiple edge areas. Specifically, obtain the number of the multiple edge areas, and screen out multiple target edge areas that meet the range of the quantity threshold within the multiple edge areas according to the number of the multiple edge areas and the quantity threshold. Then, receive the driving trajectory information of the vehicle sent by the vehicle controller, determine the road center line of the first captured image according to the driving trajectory information of the vehicle, and then divide the first captured image into a first interval and a second interval according to the road center line. It should be noted that the first interval refers to the interval on the left side of the road center line in the captured image, and the second interval refers to the interval on the right side of the road center line in the captured image. After determining the left and right intervals of the first captured image, calculate the proportions of the multiple target edge areas in the first interval and the second interval respectively, and determine the first best edge area in the first interval and the second best edge area in the second interval according to the proportion values of the multiple target edge areas in the first interval and the second interval and the preset proportion threshold. Among them, the first best edge area refers to the edge area closest to the vehicle driving trajectory in the left interval of the first captured image, and the second best edge area refers to the edge area closest to the vehicle driving trajectory in the right interval of the first captured image.

[0117] It can be understood that when it is determined that the edge area of the irregular road is single, this edge area is the best edge area in the first captured image.

[0118] Further, calculate multiple first edge lines of the first best edge area and multiple second edge lines of the second best edge area respectively. Then, screen out the edge lines that meet the first preset condition within the multiple first edge lines, which are the target edge lines in the first interval, and screen out the edge lines that meet the first preset condition within the multiple second edge lines, which are the target edge lines in the second interval. Among them, the first preset condition is the edge line with the pixel points closest to the lower edge of the image.

[0119] By applying the technical solution of this embodiment, based on the specific mining area environment, retaining walls, irregular soil piles and cliffs are regarded as irregular road edge instances to be recognized, and then the edge areas of the irregular road are determined by processing the real-time road image. Then, the irregular road edge lines are extracted from the determined edge areas of the irregular road, effectively improving the accuracy and precision of the extraction of the irregular road edge lines.

[0120] Optionally, the quantity threshold can be set to 4. By presetting the quantity threshold, the processing time for extracting the irregular road edge lines is effectively reduced, and the occupancy rate of computing resources is reduced.

[0121] In an embodiment of the present application, optionally, step 104 may specifically include: obtaining a second captured image; generating a training data set according to a preset instance and the second captured image; training an instance segmentation network according to the training data set to obtain pre-trained weights; performing inference on the first captured image according to the pre-trained weights to generate at least one edge region; wherein, the preset instance includes at least one of the following: retaining wall, polygonal mound, and cliff.

[0122] In this embodiment, a second captured image is obtained, where the second captured image refers to an image of the relevant road conditions in the mining area, specifically, it may be an image of the historical road conditions in the mining area. After obtaining the image of the relevant road conditions in the mining area, the second captured image is marked with irregular polygons according to the three preset instances of the retaining wall at the road edge, the irregular mound, and the cliff to generate a training data set. Further, the instance segmentation network is trained using the training data set to obtain the pre-trained weights for instance segmentation of irregular regions in the corresponding scenario. Subsequently, the first captured image is inferred using the pre-trained weights to obtain at least one edge region, that is, the edge region of the irregular road.

[0123] By the above method, using the retaining wall at the road edge, the irregular mound, and the cliff as preset instances, and adopting an instance segmentation network to detect the edge region of the irregular road in the mining area. Compared with the method for detecting regular roads that uses regular lane lines for edge detection in the prior art, the present application uses an instance segmentation algorithm to detect irregular retaining wall and cliff instances in the image as irregular road edge instances, and classifies the irregular road instances, improving the accuracy of identifying the edge region of the irregular road, making the determined road edge region more suitable for the irregular road in the mining area, and ensuring the accuracy of subsequent extraction of the irregular road edge line.

[0124] In an embodiment of the present application, optionally, step 110 may specifically include: if the quantity is less than or equal to the quantity threshold, determining multiple edge regions as multiple target edge regions; if the quantity is greater than the quantity threshold, obtaining multiple width values of the multiple edge regions, where the width value is the value of the coincidence position of the multiple edge regions and the boundary of the first captured image; sorting the multiple edge regions in descending order according to the multiple width values; determining multiple target edge regions within the sorted multiple edge regions according to the quantity, the quantity threshold, and the multiple width values.

[0125] In this embodiment, after identifying that multiple edge regions are included in the first captured image, the number of the multiple edge regions is compared with a number threshold. When the number of the multiple edge regions is less than the number threshold, it indicates that the number of edge regions in the first captured image is within a preset range. At this time, all the multiple edge regions are used as target edge regions. Further, when the number of the multiple edge regions is greater than the number threshold, it indicates that the number of edge regions in the first captured image exceeds the preset range. At this time, the width value of each edge region is collected, that is, the value of the coincidence position of each edge region with the side boundary of the first captured image, and the multiple edge regions are sorted in descending order according to the width value. Among the sorted multiple edge regions, according to the number threshold, the edge regions with the front width values are selected as target edge regions.

[0126] Through the above method, according to the width of the edge region and the set number range, the retained target edge regions can effectively reduce the subsequent processing time of extracting the edge line and reduce the occupancy rate of computing resources.

[0127] In a specific embodiment, the number threshold is set to 4. When the number of the multiple edge regions is less than or equal to 4, all the multiple edge regions are used as target edge regions; when the number of the multiple edge regions is greater than 4, the widths of the edge regions are sorted, and the first 4 edge regions with the widest widths are retained as target edge regions to ensure that the maximum number of edge regions retained in the captured image is 4.

[0128] In the embodiment of the present application, optionally, step 114 may specifically include: performing matrix conversion on the driving trajectory information to generate first pixel point information of the vehicle on the road; determining the road center line of the first captured image according to the first pixel point information; and dividing the first captured image into a first interval and a second interval according to the road center line.

[0129] In this embodiment, after the driving trajectory information of the vehicle is captured, by performing a transformation matrix on the driving trajectory information, the driving trajectory information of the vehicle is transformed into image pixel point information of the vehicle on the current road, that is, the first pixel point information. During the automatic driving process of the vehicle, the driving trajectory of the vehicle on the road can be regarded as the center line of the current road, that is, the first pixel point information is regarded as the road center line on the first captured image. Further, after determining the road center line, the first captured image is divided into a first interval and a second interval according to the left and right sides of the road center line. Among them, the first interval is the interval on the left side of the road center line in the first captured image, and the second interval is the interval on the right side of the road center line in the first captured image.

[0130] In the above manner, by using the driving trajectory of the vehicle, the driving route of the vehicle on the real-time acquired image can be accurately located. At the same time, the road center line determined by the driving trajectory information is used to divide the acquired image into two left and right intervals. By combining the vehicle driving trajectory information and the acquired image, the accuracy of detecting the edges of irregular roads in the mining area is improved.

[0131] It can be understood that the driving trajectory information of the vehicle is the current driving path information of the vehicle. During the automatic driving process of the vehicle, a driving point is collected by the GPS (Global Positioning System) on the vehicle at a preset distance. For example, a point is collected when the vehicle travels 60 meters, and then the driving trajectory information of the vehicle is obtained and sent through the controller end. After receiving the driving trajectory information sent by the vehicle, the three-dimensional point information is converted into two-dimensional point information through a transformation matrix to obtain image pixel point information.

[0132] In the embodiment of the present application, optionally, step 116 may specifically include: obtaining the second pixel point information of each target edge region; calculating the first occupancy ratio of each target edge region in the first interval and the second occupancy ratio of each target edge region in the second interval according to the second pixel point information; if the first occupancy ratio is greater than a preset occupancy ratio threshold, determining that at least one target edge region corresponding to the first occupancy ratio is located in the first interval; if the second occupancy ratio is greater than the preset occupancy ratio threshold, determining that at least one target edge region corresponding to the second occupancy ratio is located in the second interval; in the case where there are multiple at least one target edge regions, obtaining the pixel positions of each target edge region at the boundary of the first acquired image; sorting the multiple target edge regions in the first interval and the second interval respectively in ascending order of pixel position; determining the first best edge region in the first interval and the second best edge region in the second interval according to the preset position.

[0133] In this embodiment, the second pixel point information of each target edge region in the first acquired image is obtained, and the number of pixel points of each edge target region is determined. Then, according to the number of pixel points of each target edge region, the first occupancy ratio of each target edge region in the first interval is calculated, that is, the occupancy ratio of the target edge region in the left interval of the image is calculated. At the same time, according to the number of pixel points of each target edge region, the second occupancy ratio of each target edge region in the second interval is calculated, that is, the occupancy ratio of the target edge region in the right interval of the image is calculated.

[0134] Further, after calculating the proportion of each target edge region in the left and right intervals respectively, according to a preset proportion threshold, it is determined whether each target edge region belongs to the first interval or the second interval. Specifically, when the proportion of the target edge region in the first interval is greater than the preset proportion threshold, it indicates that the target edge region is the edge region on the left side of the image, and the target edge region is divided into the first interval. Further, when the proportion of the target edge region in the second interval is greater than the preset proportion threshold, it is determined that the target edge region is the edge region on the right side of the image, and the target edge region is divided into the second interval.

[0135] Further, the number of target edge regions located in the first interval and the second interval is obtained respectively. When there are multiple target edge regions in the first interval or the second interval, the pixel positions where each target edge region in the first interval or the second interval overlaps with the preset boundary line on the first acquisition image are determined. According to the pixel positions of the multiple target edge regions, the best edge regions in the first interval and the second interval are determined respectively.

[0136] Specifically, when there are multiple target edge regions in the first interval, the pixel positions of each target edge region on the first preset boundary line in the first acquisition image are obtained. It should be noted that the first preset boundary line is the left boundary line of the image. Thereafter, the lowest pixel position among the pixel positions of each target edge region is determined, and the lowest pixel positions of the multiple target edge regions are sorted in ascending order on the left boundary line of the image, and the target edge region that meets the second preset condition is selected as the best edge region, where the second preset condition is the pixel position closest to the bottom of the image. That is, the target edge region corresponding to the pixel position closest to the bottom of the image is retained as the first best edge region in the first interval.

[0137] Further, when there are multiple target edge regions in the second interval, the pixel positions of each target edge region on the second preset boundary line in the first acquisition image are obtained, where the second preset boundary line is the right boundary line of the first acquisition image, that is, the pixel positions of each target edge region in the second interval on the right boundary line of the image are obtained. Thereafter, the lowest pixel position among the pixel positions of each target edge region is determined, and the lowest pixel positions of the multiple target edge regions are sorted in ascending order on the right boundary line of the image, and the target edge region corresponding to the pixel position closest to the bottom of the image is retained as the second best edge region in the second interval.

[0138] By the above method, an edge region closest to the bottom of the image is determined in the left and right intervals of the acquisition image respectively, and then the road edge line is determined by using the determined best edge region, effectively reducing the computational complexity of subsequent edge line extraction and reducing the operation difficulty of the system.

[0139] In a specific embodiment, as Figure 2 shown, it is a schematic diagram of multiple target edge regions of the first captured image. In the figure, the target edge region A is located within the first interval. Since there is a single target edge region within the first interval, the target edge region A is taken as the first best edge region of the first interval. Further, in the figure, the target edge regions B and C are located within the second interval, and it is necessary to determine the best edge region of the second interval according to the pixel positions of the target edge regions B and C. Specifically, the pixel positions b1 and b2 of the target edge region B located on the right boundary line of the captured image are obtained. At the same time, the pixel positions c1 and c2 of the target edge region C located on the right boundary line of the captured image are obtained. The pixel position b2 at the bottommost of the target edge region B and the pixel position c2 at the bottommost of the target edge region C are sorted in ascending order according to their positions on the right boundary line of the image, and it is determined that c2 is the pixel position closest to the bottom of the image. Therefore, it is determined that the target edge region C is the target edge region closest to the bottom of the image, and the target edge region C is taken as the second best edge region of the second interval.

[0140] In an embodiment of the present application, optionally, step 118 may specifically include: using the canny edge detection algorithm to extract the pixel point information of the first best edge region and the pixel point information of the second best edge region respectively, generating a plurality of first edge pixel point sets and a plurality of second edge pixel point sets; using the least squares method to perform second-order polynomial fitting on each first edge pixel point set and each second edge pixel point set respectively, generating an edge equation y = ax + b, where x and y are the horizontal and vertical coordinates of each pixel point respectively, a is the slope, and b is the intercept; determining a plurality of first coordinate position points of the first best edge region and a plurality of second coordinate position points of the second best edge region according to the edge equation and the third preset condition; determining a plurality of first edge lines corresponding thereto according to the plurality of first coordinate position points; and determining a plurality of second edge lines corresponding thereto according to the plurality of second coordinate position points.

[0141] In this embodiment, through the Canny edge detection algorithm, the edge pixel point information of multiple edge lines in the left and right edge regions is extracted respectively, obtaining a plurality of first edge pixel point sets and a plurality of second edge pixel point sets. Subsequently, through the least squares method, each edge pixel point set extracted by the Canny edge detection algorithm is fitted to an edge equation y = ax + b regarding the horizontal and vertical coordinate information of the pixel points, and the position information of each pixel point can be represented by the position information of the horizontal and vertical coordinate axes. Further, according to the edge equation, a plurality of first coordinate position points and a plurality of second coordinate position points that meet the third preset condition in the first interval and the second interval are calculated respectively, where the third preset condition is the vertical coordinate position point of the pixel located at the horizontal pixel of 0 in the edge equation. That is, calculate a plurality of vertical coordinate position points of the pixel at the horizontal pixel point of 0 in the left edge equation, and calculate a plurality of vertical coordinate position points of the pixel at the horizontal pixel of 0 in the right edge equation. Subsequently, according to the calculated plurality of first coordinate position points, the corresponding plurality of edge lines are determined, and according to the calculated plurality of second coordinate position points, the corresponding plurality of edge lines are determined.

[0142] It should be noted that the judgment slope of the edge equation requires that the slope in the first interval is greater than 0, and the slope in the second interval is less than 0.

[0143] Through the above method, by combining the Canny edge detection algorithm with the least squares method, a plurality of edge lines in the left and right intervals of the acquired image are calculated, ensuring the accuracy of the extraction of the edge lines in the edge region of the image.

[0144] In a specific embodiment, as Figure 4 shown, it is a schematic diagram of a plurality of edge lines in the best edge region of the first acquired image. Specifically, after respectively determining the best edge regions in the left and right intervals of the acquired image, by combining the Canny edge detection algorithm with the least squares method, the vertical coordinate position points p1 and p2 at the horizontal pixel of 0 in the first best edge region of the first interval, and the vertical coordinate position points p3 and p4 at the horizontal pixel of 0 in the second best edge region of the second interval are deduced. Subsequently, according to the determined first coordinate position point p1, the corresponding edge line 1 is determined, according to the determined first coordinate position point p2, the corresponding edge line 2 is determined, according to the determined second coordinate position point p3, the corresponding edge line 3 is determined, and according to the determined second coordinate position point p4, the corresponding edge line 4 is determined. Further, after determining the edge line 1, edge line 2, edge line 3, and edge line 4, the edge lines retained in the left and right intervals of the acquired image are sorted in ascending order according to the edge points located on the image boundary line, and the edge line closest to the lower edge of the image among the edge points is retained as the target edge line on the left and right sides of the acquired image, that is, edge line 2 and edge line 4 in the figure.

[0145] In an embodiment of the present application, optionally, the method further includes: determining invalid pixel points according to an edge equation and a preset distance, and deleting the invalid pixel points within the optimal edge region.

[0146] In this embodiment, for a single road in the mining area environment, when the length is sufficient, the direction of the road should be consistent. Therefore, after fitting the edge equations on both the left and right sides, calculate the distance from different points to the line according to the edge equation. If the calculated distance is greater than the preset distance, it indicates that there is a road with a sudden turn, then determine this point as an invalid pixel point, and delete the determined invalid pixel points within the road edge line to ensure the accuracy of the finally generated road edge line.

[0147] It can be understood that the preset distance can be the average value of the distances from different points to the line.

[0148] Optionally, as Figure 3 shown, it is a schematic diagram of pixel points on the target edge line. After determining the target edge line, determine the discrete points and missing points within the road edge line according to the degree of dispersion from the road, and then delete the discrete points and / or supplement the missing points to ensure the continuity of the finally extracted road edge and improve the accuracy of the irregular road edge line.

[0149] An algorithm based on instance segmentation applicable to the extraction of irregular edge lines of mining area roads is proposed, obtaining irregular road edges that can guide the driving of autonomous vehicles, increasing the safety and stability of autonomous driving.

[0150] In an embodiment of the present application, for a specific mining area environment, a method for generating a road edge line is proposed. By using an instance segmentation algorithm to detect irregular retaining walls and cliff instances in the relevant image as irregular road edge instances. Subsequently, through steps such as irregular road instance classification processing, irregular road edge detection, and irregular road edge optimization, an irregular road edge line is extracted and applied to the autonomous driving of mining area roads to achieve the edge detection of irregular roads.

[0151] Specifically, since the road edge retaining walls and cliffs in the mining area are irregular in shape and the detection of the region of interest cannot be achieved through the modeling of regular shapes, an instance segmentation model with a polygon output is selected for training to obtain an irregular polygon region of interest. That is to say, in the mining area environment, the road edge retaining walls of the current lane where the vehicle is traveling, the opposite-shaped soil piles, and the cliffs generated due to mining in the mining area are regarded as irregular road edge region instances to be recognized, that is, preset instances.

[0152] Further, relevant road condition pictures, i.e., the second acquisition images, are collected in the mining area. According to three preset instances, namely the road edge retaining wall, the irregular soil heap, and the cliff, irregular polygon annotation is performed to serve as the training data set for instance segmentation. Subsequently, the instance segmentation network is trained using the training data set to obtain the pre-trained weights for instance segmentation of irregular regions in the corresponding scenario. The pre-trained weights are used to infer the first acquisition images obtained during the actual vehicle driving to obtain the irregular road edge regions.

[0153] Further, after obtaining the road edge region in a single first acquisition image, by calculating the number of edge regions, the irregular road edge regions are divided into single edge regions and multiple edge regions for processing.

[0154] Further, when it is determined that the road edge region is a multiple edge region, since the multiple edge regions are processed during driving, the irregular road edge regions are usually long, and misidentified instances need to be deleted. Then, the position region where the edge region exists in the acquisition image is judged to distinguish its left and right sides with respect to the vehicle. Furthermore, the position states are respectively judged according to the left and right edges, and finally, at most one instance is retained on each side.

[0155] Specifically, the number of multiple edge regions is obtained. When the number of regions extracted by the algorithm is greater than the quantity threshold, for example, when there are 4 regions, by sorting the widths and lengths of the region edges, the first 4 regions with the widest widths are retained. Through this step, it is ensured that the number of multiple edge regions is at most 4. Subsequently, the 4 regions are divided into two parts, namely the left and right retaining walls, i.e., the first interval and the second interval. By obtaining the vehicle driving trajectory information and converting it into the image pixel point information of the current lane, the proportion of each edge region on the left and right sides of the lane pixels is calculated. When the proportion of the left pixels exceeds the preset proportion threshold, for example, 75%, then the edge region is classified as the left region; when the proportion of the right pixels exceeds the preset proportion threshold, for example, 75%, then the edge region is classified as the right region. Further, the left and right edge regions are sorted according to the position of the bottommost edge pixel, and the edge region closest to the bottom of the image is respectively retained on each side. Through this step, the best edge region closest to the vehicle driving area on each side can be obtained.

[0156] Further, when it is determined that the edge region of the irregular road is a single edge region, the edge region can be directly used as the best edge region closest to the vehicle driving area on each side.

[0157] Further, after determining the best edge region closest to the vehicle driving area on each side, according to the best edge regions on the left and right sides, the edge lines of the edge regions are extracted using the canny edge detection algorithm. Then, the two lines closest to the vehicle direction are selected, and after discrete sampling, the irregular road edge lines during vehicle driving are obtained.

[0158] Specifically, using the Canny edge detection algorithm, the edge pixel point sets of the left and right edge regions are extracted respectively. By the least squares method, each edge pixel point set extracted by the Canny edge algorithm is fitted into a straight line equation about the horizontal and vertical coordinate information of the pixel points: y = ax + b, where it is judged that the slope of the left side should be greater than 0 and the slope of the right side should be less than 0. Further, calculate the vertical coordinate position points of the pixels at the horizontal pixel of 0 in the left edge equation, and the vertical coordinate position points of the pixels at the horizontal pixel of 0 in the right edge equation, and retain the edge line with the edge point closest to the lower edge of the image as the target edge line.

[0159] Furthermore, since there are discrete points and discontinuous regions in the irregular road edge line calculated by the above method, it is necessary to increase the post-processing of discrete points and optimize the edge continuity to improve the extraction accuracy of the irregular road edge line and output the final road edge line. Specifically, find the discrete points of the road edge, and select to delete discrete points and supplement missing points according to the degree of discreteness from the road.

[0160] Furthermore, for a single road, when the length is sufficient, the direction of the road should be consistent. Therefore, it is necessary to optimize the road with sudden mutations and turns.

[0161] The road boundary line generation method provided by the embodiments of the present application realizes the edge detection of irregular roads in a specific mining area environment, and has environmental adaptability and stability.

[0162] Furthermore, as Figure 1 a specific implementation of the method, the embodiments of the present application provide a device for generating a road edge line, as Figure 5 shown. The device includes:

[0163] A first acquisition module, configured to acquire a first acquisition image in response to a request for generating a road edge line;

[0164] A first determination module, configured to determine at least one edge region of the road according to a preset instance and the first acquisition image;

[0165] A second acquisition module, configured to acquire the number of multiple edge regions when there are multiple edge regions;

[0166] A second determination module, configured to determine multiple target edge regions within the multiple edge regions according to the number and a number threshold;

[0167] A third acquisition module, configured to acquire the driving trajectory information of the vehicle;

[0168] A fourth determination module, configured to determine a first interval and a second interval in the first acquisition image according to the driving trajectory information;

[0169] A fifth determination module, configured to determine a first optimal edge region in a first interval and a second optimal edge region in a second interval according to a plurality of target edge regions and a preset ratio threshold;

[0170] A calculation module, configured to calculate a plurality of first edge lines of the first optimal edge region and a plurality of second edge lines of the second optimal edge region;

[0171] A generation module, configured to generate a target edge line of a road according to the plurality of first edge lines, the plurality of second edge lines, and a first preset condition.

[0172] Optionally, the first determination module is specifically configured to:

[0173] Obtain a second acquisition image;

[0174] Generate a training data set according to a preset instance and the second acquisition image;

[0175] Train an instance segmentation network according to the training data set to obtain pre-trained weights;

[0176] Infer the first acquisition image according to the pre-trained weights to generate at least one edge region;

[0177] Wherein, the preset instance includes at least one of the following: retaining wall, polygonal mound, and cliff.

[0178] Optionally, the second determination module is specifically configured to:

[0179] If the quantity is less than or equal to the quantity threshold, determine the plurality of edge regions as the plurality of target edge regions;

[0180] If the quantity is greater than the quantity threshold, obtain a plurality of width values of the plurality of edge regions, where the width value is the numerical value of the coincidence position of the plurality of edge regions and the boundary of the first acquisition image;

[0181] Sort the plurality of edge regions in descending order according to the plurality of width values;

[0182] Determine the plurality of target edge regions within the sorted plurality of edge regions according to the quantity, the quantity threshold, and the plurality of width values.

[0183] Optionally, the third determination module is specifically configured to:

[0184] Perform matrix conversion on the driving trajectory information to generate first pixel point information of the vehicle on the road;

[0185] Determine the road center line of the first acquisition image according to the first pixel point information;

[0186] Divide the first acquisition image into a first interval and a second interval according to the road center line.

[0187] Optionally, the fourth determination module is specifically configured to:

[0188] Obtain the second pixel point information of each target edge region;

[0189] According to the second pixel point information, calculate the first occupancy ratio of each target edge region in the first interval and the second occupancy ratio of each target edge region in the second interval;

[0190] If the first occupancy ratio is greater than the preset occupancy ratio threshold, determine that the target edge region corresponding to the first occupancy ratio is located in the first interval;

[0191] If the second occupancy ratio is greater than the preset occupancy ratio threshold, determine that the target edge region corresponding to the second occupancy ratio is located in the second interval;

[0192] Obtain the number of target edge regions in the first interval;

[0193] When there are multiple target edge regions in the first interval, obtain the pixel positions of each target edge region on the first preset boundary line of the first acquisition image;

[0194] Sort the multiple target edge regions in the first interval in ascending order of pixel position;

[0195] Determine the first best edge region in the first interval according to the second preset condition;

[0196] Obtain the number of target edge regions in the second interval;

[0197] When there are multiple target edge regions in the second interval, obtain the pixel positions of each target edge region on the second preset boundary line of the first acquisition image;

[0198] Sort the multiple target edge regions in the second interval in ascending order of pixel position;

[0199] Determine the second best edge region in the second interval according to the second preset condition.

[0200] Optionally, the calculation module is specifically configured to:

[0201] Use the canny edge detection algorithm to extract the third pixel point information of the first best edge region and the fourth pixel point information of the second best edge region respectively, and generate multiple first edge pixel point sets and multiple second edge pixel point sets;

[0202] Using the least squares method, perform second-order polynomial fitting on each set of first edge pixel points and each set of second edge pixel points respectively to generate an edge equation y = ax + b, where x is the abscissa of each pixel point, y is the ordinate of each pixel point, a is the slope, and b is the intercept;

[0203] According to the edge equation and the third preset condition, determine multiple first coordinate position points of the first optimal edge region and multiple second coordinate position points of the second optimal edge region;

[0204] According to the multiple first coordinate position points, determine the corresponding multiple first edge lines;

[0205] According to the multiple second coordinate position points, determine the corresponding multiple second edge lines.

[0206] Optionally, the device for generating a road edge line further includes:

[0207] A sixth determination module, configured to determine invalid pixel points according to the edge equation and a preset distance, and delete the invalid pixel points within the optimal edge region.

[0208] It should be noted that for other corresponding descriptions of each functional unit involved in the device for generating a road edge line provided in the embodiments of the present application, reference can be made to Figure 1 the corresponding description in the method, which will not be elaborated here.

[0209] Based on the above method as Figure 1 shown, correspondingly, the embodiments of the present application further provide a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for generating a road edge line as Figure 1 shown above is implemented.

[0210] Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various implementation scenarios of the present application.

[0211] Based on the above method as Figure 1 shown, and Figure 5 the virtual device embodiment shown, in order to achieve the above object, the embodiments of the present application further provide a computer device, which can specifically be a personal computer, a server, a network device, etc., and the computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the method for generating a road edge line as Figure 1 shown above.

[0212] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, sensors, an audio circuit, a WI-FI module, and so on. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc. Optionally, the user interface may further include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0213] Those skilled in the art can understand that the structure of a computer device provided in this embodiment does not constitute a limitation on the computer device, and it may include more or fewer components, or combine certain components, or have different component arrangements.

[0214] The storage medium may further include an operating system and a network communication module. The operating system is a program for managing and storing the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to implement communication between the components inside the storage medium, as well as communication between other hardware and software in the entity device.

[0215] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or can be implemented by hardware. By using an instance segmentation algorithm to detect irregular retaining walls and cliff instances in an image as irregular road edge instances, classifying and processing the irregular road instances, and then performing irregular road edge detection, the extracted road edge line is more suitable for the irregular road conditions in the mining area, effectively improving the accuracy and precision of the irregular road edge line.

[0216] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred embodiment scenario, and the modules or processes in the drawings are not necessarily essential for implementing the present application. Those skilled in the art can understand that the modules in the device in the embodiment scenario can be distributed in the device in the embodiment scenario according to the description of the embodiment scenario, or can be correspondingly changed and located in one or more devices different from the present embodiment scenario. The modules in the above embodiment scenario can be combined into one module, or further split into multiple sub-modules.

[0217] The above serial numbers of the present application are only for description and do not represent the advantages or disadvantages of the embodiment scenario. The above disclosure is only several specific embodiment scenarios of the present application. However, the present application is not limited thereto, and any changes that can be thought of by those skilled in the art should fall within the protection scope of the present application.

Claims

1. A method for generating a road edge line, characterized in that, Including: In response to a generation request for a road edge line, obtain a first captured image; Determine at least one edge region of the road according to a preset instance and the first captured image; When there are multiple of the at least one edge region, obtain the number of the multiple edge regions; Determine multiple target edge regions within the multiple edge regions according to the number and a number threshold; Obtain the driving trajectory information of the vehicle; Determine a first interval and a second interval within the first captured image according to the driving trajectory information; Determine a first best edge region within the first interval and a second best edge region within the second interval according to the multiple target edge regions and a preset proportion threshold; Calculate multiple first edge lines of the first best edge region and multiple second edge lines of the second best edge region; Generate a target edge line of the road according to the multiple first edge lines, the multiple second edge lines and a first preset condition; The step of determining multiple target edge regions within the multiple edge regions according to the number and the number threshold specifically includes: If the number is less than or equal to the number threshold, determine the multiple edge regions as the multiple target edge regions; If the number is greater than the number threshold, obtain multiple width values of the multiple edge regions, where the width value is the value of the coincidence position of the multiple edge regions and the boundary of the first captured image; Sort the multiple edge regions in descending order according to the multiple width values; Determine the multiple target edge regions within the sorted multiple edge regions according to the number, the number threshold and the multiple width values.

2. The method according to claim 1, characterized in that, The step of determining at least one edge region of the road according to a preset instance and the first captured image specifically includes: Obtain a second captured image; Generate a training data set according to the preset instance and the second captured image; Train an instance segmentation network according to the training data set to obtain pre-trained weights; Perform inference on the first captured image according to the pre-trained weights to generate the at least one edge region; Wherein, the preset instance includes at least one of the following: retaining wall, polygonal soil mound and cliff.

3. The method according to claim 1, characterized in that, The step of determining a first interval and a second interval within the first captured image according to the driving trajectory information specifically includes: Perform matrix conversion on the driving trajectory information to generate first pixel point information of the vehicle on the road; Determine the road center line of the first captured image according to the first pixel point information; Divide the first captured image into the first interval and the second interval according to the road center line.

4. The method according to claim 1, characterized in that, The step of determining a first best edge region within the first interval and a second best edge region within the second interval according to multiple target edge regions and a preset proportion threshold specifically includes: Obtain second pixel point information of each target edge region; According to the second pixel point information, calculate a first proportion value of each target edge region within the first interval and a second proportion value of each target edge region within the second interval; If the first occupancy ratio is greater than the preset occupancy ratio threshold, determine that the target edge region corresponding to the first occupancy ratio is located in the first interval; If the second occupancy ratio is greater than the preset occupancy ratio threshold, determine that the target edge region corresponding to the second occupancy ratio is located in the second interval; Obtain the number of target edge regions within the first interval; When there are multiple target edge regions within the first interval, obtain the pixel positions of each target edge region on the first preset boundary line of the first acquisition image; Sort the multiple target edge regions within the first interval in ascending order according to the pixel positions; Determine the first optimal edge region within the first interval according to the second preset condition; Obtain the number of target edge regions within the second interval; When there are multiple target edge regions within the second interval, obtain the pixel positions of each target edge region on the second preset boundary line of the first acquisition image; Sort the multiple target edge regions within the second interval in ascending order according to the pixel positions; Determine the second optimal edge region within the second interval according to the second preset condition.

5. The method according to claim 1, wherein The steps of calculating the multiple first edge lines of the first optimal edge region and the multiple second edge lines of the second optimal edge region specifically include: Using the canny edge detection algorithm, extract the third pixel point information of the first optimal edge region and the fourth pixel point information of the second optimal edge region respectively, and generate multiple first edge pixel point sets and multiple second edge pixel point sets; Using the least squares method, perform second-order polynomial fitting on each first edge pixel point set and each second edge pixel point set respectively to generate an edge equation y = ax + b, where x is the abscissa of each pixel point, y is the ordinate of each pixel point, a is the slope, and b is the intercept; According to the edge equation and the third preset condition, determine multiple first coordinate position points of the first optimal edge region and multiple second coordinate position points of the second optimal edge region; Determine the corresponding multiple first edge lines according to the multiple first coordinate position points; Determine the corresponding multiple second edge lines according to the multiple second coordinate position points.

6. The method according to any one of claims 1 to 5, wherein The method further includes: Determine invalid pixel points according to the edge equation and the preset distance, and delete the invalid pixel points within the optimal edge region.

7. An apparatus for generating a road edge line, wherein Includes: A first acquisition module, configured to acquire a first acquisition image in response to a generation request for a road edge line; A first determination module, configured to determine at least one edge region of the road according to a preset instance and the first acquisition image; A second acquisition module, configured to obtain the number of multiple edge regions when there are multiple of the at least one edge region; A second determination module, configured to determine multiple target edge regions within the multiple edge regions according to the number and the number threshold; A third acquisition module, configured to acquire the driving trajectory information of the vehicle; A fourth determination module, configured to determine a first interval and a second interval within the first acquisition image according to the driving trajectory information; A fifth determination module, configured to determine a first optimal edge region within the first interval and a second optimal edge region within the second interval according to the multiple target edge regions and a preset proportion threshold; A calculation module, configured to calculate multiple first edge lines of the first optimal edge region and multiple second edge lines of the second optimal edge region; A generation module, configured to generate a target edge line of the road according to the multiple first edge lines, the multiple second edge lines, and a first preset condition; A second determination module, specifically configured to: If the quantity is less than or equal to a quantity threshold, determine multiple edge regions as multiple target edge regions; If the quantity is greater than the quantity threshold, obtain multiple width values of the multiple edge regions, where the width value is a numerical value of the coincidence position of the multiple edge regions and the boundary of the first captured image; Sort the multiple edge regions in descending order according to the multiple width values; Determine multiple target edge regions within the sorted multiple edge regions according to the quantity, the quantity threshold, and the multiple width values.

8. A storage medium having a computer program stored thereon, wherein When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 6.

9. A computer device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein When the processor executes the computer program, it implements the method according to any one of claims 1 to 6.

Images