Parking frame recognition method, device, electronic device and storage medium

Abstract

The present invention relates to the field of autonomous driving technology, and more particularly to a parking frame recognition method, device, electronic device, and storage medium. The method is applied to a vehicle equipped with at least two cameras. The method first acquires original images captured by the at least two cameras, then dedistorts each original image and splices it together to obtain an undistorted image. The method then classifies and identifies the parking frame edges and corners in the undistorted image to obtain a parking frame edge and corner classification result in the undistorted image. The parking frame corners are then matched on the parking frame edge and corner classification results to obtain parking frame parameters in the undistorted image. Finally, a target parking frame image is calculated based on the parking frame parameters. Therefore, the parking frame recognition method obtains a more accurate target parking frame image by identifying the parking frame edges and corners, and the method is more robust and has a lower missed detection rate.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a parking frame recognition method, device, electronic device and storage medium. Background Art

[0002] In recent years, with the development of society, the popularity of cars has continued to increase, and the surround-view parking assistance system is an important component of the assisted driving system. In the surround-view parking assistance system, the intelligent detection of the parking frame is an important step.

[0003] Current surround-view parking frame detection technology typically uses cameras positioned around the vehicle to capture images. A bird's-eye view is then stitched together using perspective conversion. Traditional algorithms, such as thresholding and edge detection, or deep learning algorithms, are then applied to the images to determine the pixel positions of the parking frame. Finally, post-processing algorithms, such as the Hough transform and template matching, are used to determine the parking frame's graphical parameters (length, width, orientation angle, and center position). Traditional algorithms, such as thresholding and edge detection, that determine the pixel positions of the parking frame are often susceptible to variations in illumination, noise, and parking frame type, resulting in poor robustness and accuracy, and a high rate of missed detections. Summary of the Invention

[0004] The embodiments of the present invention aim to provide a parking frame recognition method, device, electronic device and storage medium to solve the problems of poor robustness and accuracy of parking frame images obtained by the prior art.

[0005] To solve the above technical problems, a technical solution adopted in an embodiment of the present invention is to provide a parking frame recognition method applied to a vehicle, wherein the vehicle is provided with at least two cameras, and the method comprises:

[0006] Acquire original images respectively captured by the at least two cameras;

[0007] Dedistorting each of the original images and then splicing them together to obtain a non-distorted image;

[0008] Classifying and identifying parking frame edges and parking frame corners in the non-distorted image to obtain a parking frame edge line classification result and a parking frame corner point classification result in the non-distorted image;

[0009] performing parking frame corner point matching on the parking frame edge line classification result and the parking frame corner point classification result to obtain parameters of the parking frame in the undistorted image;

[0010] According to the parameters of the parking frame, a target parking frame image is calculated and obtained.

[0011] In some embodiments, classifying and identifying the parking frame edge lines and parking frame corner points in the undistorted image to obtain a parking frame edge line classification result and a parking frame corner point classification result in the undistorted image includes:

[0012] Performing convolution processing on the undistorted image to obtain image features at different levels;

[0013] fusing the image features at different levels by short residual connections to obtain a high-dimensional feature map;

[0014] Upsampling the high-dimensional feature map so that the image size of the high-dimensional feature map is the same as the image size of the non-distorted image, so as to obtain a prediction result of the parking frame edge line and a prediction result of the parking frame corner point in the high-dimensional feature map after the size change;

[0015] Based on the prediction results of the parking frame edge lines and the prediction results of the parking frame corner points, the parking frame edge line classification results and the parking frame corner point classification results are obtained.

[0016] In some embodiments, the step of classifying and identifying the parking frame edge lines and parking frame corner points in the undistorted image to obtain a parking frame edge line classification result and a parking frame corner point classification result in the undistorted image is performed by a preset algorithm model;

[0017] The method further includes obtaining a preset algorithm model, wherein obtaining the preset algorithm model includes:

[0018] Acquire a training sample, wherein the training sample includes a batch of non-distorted image data;

[0019] annotating the batch of undistorted image data according to preset labels to obtain annotated training samples, wherein the preset labels include parking frame edge positions, parking frame corner point categories, and parking frame corner point positions;

[0020] Inputting the labeled training samples into the preset algorithm model to output a parking box edge classification result and a parking box corner classification result of the training samples, wherein the parking box edge classification result includes a parking box edge position, and the parking box corner classification result includes a parking box corner category and a parking box corner position;

[0021] Constraining the relationship between the classification result and the preset label according to a classification loss function so that the classification result output by the preset algorithm model is close to the preset label;

[0022] The algorithm model whose output classification result is closest to the classification of the preset label is used as the preset algorithm model.

[0023] In some embodiments, performing parking frame corner point matching on the parking frame edge classification result and the parking frame corner point classification result to obtain parameters of the parking frame in the undistorted image includes:

[0024] Processing the parking frame edge lines and parking frame corners in the non-distorted image to obtain a parking frame corner image;

[0025] Traversing the parking frame corner points of the parking frame corner point image to obtain attributes of the parking frame corner points;

[0026] Matching the corner points of the parking frame according to the attributes of the corner points of the parking frame to obtain a matching relationship between the corner points of the parking frame;

[0027] According to the matching relationship between the corner points of the parking frame, the parameters of the parking frame in the undistorted image are obtained.

[0028] In some embodiments, processing the parking frame edge lines and parking frame corners in the undistorted image to obtain a parking frame corner image includes:

[0029] Masking and thinning the parking frame edge lines and the parking frame corners in the undistorted image to obtain a parking frame mask image;

[0030] Segmenting the parking frame mask image according to the parking frame corner point classification result to obtain sub-mask images;

[0031] Performing a connected domain analysis on the sub-mask image to determine the position coordinates of the corner points of the parking frame in the sub-mask image;

[0032] The sub-mask image is cropped according to the position coordinates of the parking frame corner points and a preset cropping size to obtain the parking frame corner point image.

[0033] In some embodiments, traversing the parking frame corner points of the parking frame corner point image to obtain attributes of the parking frame corner points includes:

[0034] acquiring a traversal strategy according to the parking frame corner point image and the categories of the parking frame corner points in the parking frame corner point image;

[0035] According to the traversal strategy, the parking frame corner points of the parking frame corner point image are traversed to obtain the attributes of the parking frame corner points.

[0036] In some embodiments, matching the corner points of the parking frame according to the attributes of the corner points of the parking frame to obtain a matching relationship of the corner points of the parking frame includes:

[0037] Obtaining a corner point matching strategy according to the category of the parking frame corner point;

[0038] Matching the corner points of the parking frame according to the corner point matching strategy and the attributes of the corner points of the parking frame to obtain a first matching result;

[0039] According to the first matching result, an angle judgment is performed on the two corner points of the parking frame in the first matching result. When the two corner points of the parking frame meet a preset condition, it is determined that the two corner points of the parking frame in the first matching result are matched, that is, the two corner points of the parking frame are in a matching relationship;

[0040] Repeat the above steps to obtain the matching relationship of all parking frame corner points.

[0041] To solve the above technical problems, another technical solution adopted in the embodiment of the present invention is to provide a parking frame recognition device applied to a vehicle, wherein the vehicle is provided with at least two cameras, and the device comprises:

[0042] An acquisition module, configured to acquire original images captured by the at least two cameras respectively;

[0043] a dedistortion module, configured to perform dedistortion processing on each of the original images and then splice the original images to obtain a non-distorted image;

[0044] a classification and recognition module, configured to classify and recognize parking frame edges and parking frame corners in the non-distorted image to obtain a parking frame edge line classification result and a parking frame corner point classification result in the non-distorted image;

[0045] a corner point matching module, configured to perform parking frame corner point matching on the parking frame edge line classification result and the parking frame corner point classification result to obtain parameters of the parking frame in the undistorted image;

[0046] The calculation module calculates and obtains the target parking frame image according to the parameters of the parking frame.

[0047] To solve the above technical problems, another technical solution adopted in the embodiment of the present invention is to provide an electronic device, the electronic device comprising:

[0048] at least one processor; and,

[0049] a memory communicatively connected to the at least one processor; wherein,

[0050] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0051] In order to solve the above technical problems, another technical solution adopted in the embodiment of the present invention is: providing a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by an electronic device, the electronic device executes the method as described above.

[0052] Differentiating from existing technologies, the parking frame recognition method, device, electronic device, and storage medium provided by the present invention obtain original images captured by at least two cameras, dedistort each original image, and then stitch them together to produce an undistorted image. The method then classifies and identifies the parking frame edges and corners in the undistorted image to obtain a parking frame edge and corner classification result. The method then performs parking frame corner matching on the parking frame edge and corner classification results to obtain parking frame parameters in the undistorted image. Based on the parking frame parameters, the method, device, electronic device, and storage medium calculate a target parking frame image. By identifying parking frame edges and corners, the method, device, electronic device, and storage medium achieve more accurate target parking frame images, resulting in stronger robustness and a lower missed detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] One or more embodiments are exemplarily illustrated by corresponding drawings, which do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0054] Figure 1 1 is a flow chart of a parking frame recognition method provided by an embodiment of the present invention;

[0055] Figure 2 1 is a flow chart of a method for obtaining parking frame edge line classification results and parking frame corner point classification results in a non-distorted image provided by an embodiment of the present invention;

[0056] Figure 3 is a flow chart of a parking frame recognition method provided by another embodiment of the present invention;

[0057] Figure 4 Schematic diagram of a method for obtaining a preset algorithm model provided by an embodiment of the present invention;

[0058] Figure 5 1 is a flow chart of a method for obtaining parameters of a parking frame in an undistorted image provided by an embodiment of the present invention;

[0059] Figure 6 1 is a flow chart of a method for obtaining a parking frame corner point image provided by an embodiment of the present invention;

[0060] Figure 7 1 is a flow chart of a method for obtaining attributes of corner points of a parking frame provided by an embodiment of the present invention;

[0061] Figure 8 1 is a flow chart of a method for obtaining a matching relationship between corner points of a parking frame provided by an embodiment of the present invention;

[0062] Figure 9 Schematic diagram of the positions of parking frame corner point i and parking frame corner point k provided in an embodiment of the present invention;

[0063] Figure 10 is a vector diagram of i-point pairs and k-point pairs provided by an embodiment of the present invention;

[0064] Figure 11 is a parking frame mask image provided by an embodiment of the present invention;

[0065] Figure 12 is a target parking frame image provided by an embodiment of the present invention;

[0066] Figure 13 This is a schematic structural diagram of a parking frame recognition device provided by an embodiment of the present invention;

[0067] Figure 14 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0069] It should be noted that, unless there is a conflict, the various features of the embodiments of the present invention may be combined with each other and are all within the scope of protection of the present invention. In addition, although the functional modules are divided in the device schematics and the logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than the module division in the device schematics or the order in the flowcharts.

[0070] Unless otherwise defined, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which this invention pertains. The terms used in this specification are intended solely for the purpose of describing specific embodiments and are not intended to limit the invention. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0071] The parking frame recognition method and device provided by the embodiments of the present invention primarily utilize semantic segmentation to identify parking frame edges and corners at the pixel level, resulting in a more accurate target parking frame image. This method also offers enhanced robustness and a lower missed detection rate. Even in situations where a vehicle's positioning is lost, inaccurate, or lacks map information, parking frame recognition can be implemented, thereby improving parking success rates and applicability.

[0072] See also Figure 1 , Figure 1 This is a flow chart of a facial pigment spot recognition method provided by an embodiment of the present invention. The parking frame recognition method is applied to a vehicle, wherein the vehicle is equipped with at least two cameras, and the method includes:

[0073] S1. Obtain original images captured by at least two cameras.

[0074] In this embodiment, the camera may be a fisheye camera. One fisheye camera may be installed at the front and rear of the vehicle, or four fisheye cameras may be installed at the front, rear, and left and right rearview mirrors. The original image is captured by the fisheye camera and includes a parking frame area. The parking frame area refers to the image area covered by the parking frame, which may be a parking frame in a parking lot such as an underground parking lot in a shopping mall or a roadside parking lot.

[0075] S3. Dedistort each original image and then stitch them together to obtain a non-distorted image.

[0076] The undistorted image is a complete bird's-eye view image centered on the vehicle.

[0077] Specifically, each original image is first dedistorted, and the dedistorted image is converted into a bird's-eye view image through perspective transformation. Then, each bird's-eye view image is spliced with the vehicle as the center to obtain a non-distorted image.

[0078] In some embodiments, at least two checkerboards are arranged around the vehicle, and the at least two checkerboards correspond to at least two cameras respectively. The conversion of the dedistorted image into a bird's-eye view image through perspective transformation includes: measuring the relative position of the checkerboards and the cameras corresponding to the checkerboards, determining the ratio of pixels in the bird's-eye view image to actual distances based on the relative positions, calculating the pixel coordinates of the checkerboard vertices in the bird's-eye view image, obtaining the vertex coordinates of the checkerboards in the dedistorted image, obtaining a projection matrix based on the perspective transformation principle, and obtaining the bird's-eye view image based on the coordinate data in the dedistorted image and the projection matrix.

[0079] After dedistorting each original image and converting it into a bird's-eye view image through perspective transformation, all the bird's-eye view images are spliced into a complete bird's-eye view image centered on the vehicle according to the relative position of the camera installed on the vehicle body. The complete bird's-eye view image is the non-distorted image.

[0080] S5. Classify and identify the parking frame edge lines and parking frame corner points in the non-distorted image to obtain a parking frame edge line classification result and a parking frame corner point classification result in the non-distorted image.

[0081] In this embodiment, pixel-level classification of images can be achieved, that is, each pixel is classified.

[0082] See also Figure 2 , Figure 2 The flowchart of the method for obtaining parking frame edge line classification results and parking frame corner point classification results in an undistorted image provided by an embodiment of the present invention is shown. Specifically, the method classifies and identifies parking frame edge lines and parking frame corner points in the undistorted image to obtain parking frame edge line classification results and parking frame corner point classification results in the undistorted image, including:

[0083] S51. Perform convolution processing on the non-distorted image to obtain image features at different levels.

[0084] Before performing convolution on the undistorted image, the convolution kernel and stride are determined. Each unit of the convolution kernel has a weight. The convolution kernel is moved across the undistorted image according to the stride. For example, a stride of 1 means the kernel moves one pixel to the right at a time. As the convolution kernel moves, the pixels in the image are multiplied by the corresponding weights of the convolution kernel. Finally, all the products are added together to produce an output. The undistorted image is represented by a matrix, with each pixel in the undistorted image forming the matrix. The convolution kernels of different layers can be the same or different. The operations on the convolution kernel and the image matrix described above can extract partial image features, such as color features. To obtain higher-level features, convolution can be performed further based on the current output image features. During the convolution process, the output of the previous layer becomes the input to the next layer. After multiple operations, deeper image features (such as semantic features) are obtained. These deeper image features incorporate the previous lower-level image features.

[0085] S52. Image features at different levels are fused through short residual connections to obtain a high-dimensional feature map.

[0086] After multiple convolution operations, the image resolution decreases. To restore the low-resolution image to its original size and perform classification predictions for each pixel in the original image, a deconvolution operation is required. To improve the accuracy of the deconvolution results, before performing the deconvolution operation, image features at different levels are fused using short residual connections to obtain a high-dimensional feature map. For example, the high-level feature map outputted by S51 is convolved with any of the previous convolution layers. Typically, the high-level convolution layer is convolved with the low-level convolution layer. Specifically, the matrix corresponding to the high-level convolution layer is multiplied with the matrix corresponding to the low-level convolution layer to obtain a high-dimensional feature map. The high-dimensional feature map has richer features, which is beneficial for upsampling to restore the richness of features. Although the image features obtained by S51 also fuse multiple features, the noise is removed step by step during the convolution process. That is, after each layer, the noise is reduced, but the image details are also greatly reduced. Therefore, short residual connections are used to restore the image details. It should be noted that when performing short residual connections, the image features obtained at different levels can perform short residual connections with any of the aforementioned convolutional layers.

[0087] S53. Up-sampling the high-dimensional feature map so that the image size of the high-dimensional feature map is the same as the image size of the non-distorted image, so as to obtain the prediction results of the parking frame edge line and the parking frame corner points in the high-dimensional feature map after the size change.

[0088] The purpose of upsampling is to restore the low-resolution feature map to high resolution, restoring the high-dimensional feature map to the same size as the input undistorted image while preserving the spatial information in the original input undistorted image. The result of upsampling is that each pixel in the image receives several predictions. The number of predictions for each pixel is equal to the number of channels in the image, which is the total number of categories. The prediction for each pixel specifically includes category information and the probability that the pixel belongs to that category.

[0089] Among them, there are two teams to identify the parking frame edges and parking frame corners: one team identifies the parking frame edges, and the other team identifies the parking frame corners.

[0090] When predicting the parking frame boundary, the category information includes background and parking frame boundary. The probability of a pixel belonging to a category, such as the probability of the pixel belonging to the background and the probability of the pixel belonging to the parking frame boundary, is also included.

[0091] When predicting parking frame corners, the category information includes background, T-shaped corners, L-shaped corners, and I-shaped corners. The probability of a pixel belonging to a category includes, for example, the probability of the pixel belonging to the background, the probability of the pixel belonging to a T-shaped corner, the probability of the pixel belonging to an L-shaped corner, and the probability of the pixel belonging to an I-shaped corner. T-shaped corners represent T-shaped parking frame corners, typically connecting two parking frames. L-shaped corners represent the corners of a single parking frame. I-shaped corners represent the endpoints of isolated parking frame line segments that appear in the image due to factors such as lighting.

[0092] The prediction result for each pixel includes the probability of the pixel belonging to that category. There are as many probability values as there are categories. For example, if there are three categories, the prediction result for each pixel includes the probability of the pixel belonging to the first category, the probability of the pixel belonging to the second category, and the probability of the pixel belonging to the third category. A larger probability value indicates that the pixel is more likely to belong to that category; a smaller probability value indicates that the pixel is less likely to belong to that category.

[0093] S54. Based on the prediction results of the parking frame edge lines and the prediction results of the parking frame corner points, obtain the parking frame edge line classification results and the parking frame corner point classification results.

[0094] The algorithm outputs the category of each pixel. The parking frame edges and corners in the undistorted image are composed of pixels. Therefore, pixels belonging to the same category are aggregated to achieve classification of the parking frame edges and corners.

[0095] In this embodiment, the network decoding component is designed as two branches with identical structures. One branch sets the final pixel output to two categories: background and parking frame border. The parking frame border is annotated as a polygon, with the annotated categories being background and parking frame border. The grayscale of the background image is set to 0, and the grayscale of the parking frame image is set to 1.

[0096] The other branch sets the number of pixel categories in the final output to four categories, that is, the corner points of the parking frame are marked in the form of circles, and the marked categories are background, T-shaped corner points, L-shaped corner points and I-shaped corner points. The image grayscale of the background is set to 0, the image grayscale of the T-shaped corner points is set to 1, the image grayscale of the L-shaped corner points is set to 1, and the image grayscale of the I-shaped corner points is set to 1.

[0097] Existing techniques only use a single branch for classification. Simply outputting the parking frame edge as the binary classification result of the semantic segmentation network is insufficient. This segmentation result also fails to correlate with the parking frame's attributes, such as length, width, and position. This solution proposes an additional network branch for parking frame corner segmentation, eliminating the need for an additional network. This solution completes the segmentation task while maintaining high performance and accurately identifies both parking frame corners and edges.

[0098] S7. Perform parking frame corner point matching on the parking frame edge line classification result and the parking frame corner point classification result to obtain parameters of the parking frame in the undistorted image.

[0099] After identifying the parking frame corner points and parking frame edges, the different parking frame corner points in the non-distorted image are matched to determine the matching relationship between the parking frame corner points, and the parking frame parameters are obtained through the matching relationship.

[0100] S9. Calculate and obtain a target parking frame image based on the parameters of the parking frame.

[0101] The parameters of the parking frame include: the coordinates of the center point of the parking frame, the length of the parking frame, the width of the parking frame, and the orientation angle of the parking frame.

[0102] The parking frame recognition method and device provided by the embodiments of the present invention primarily utilize semantic segmentation to identify parking frame edges and corners at the pixel level, resulting in a more accurate target parking frame image. This method also offers enhanced robustness and a lower missed detection rate. Even in situations where a vehicle's positioning is lost, inaccurate, or lacks map information, parking frame recognition can be implemented, thereby improving parking success rates and applicability.

[0103] See also Figure 3 , Figure 3 is a flow chart of a parking frame recognition method provided by another embodiment of the present invention. In some embodiments, the above step S5 is performed by a preset algorithm model, specifically, as Figure 3 As shown, the method includes:

[0104] S2. Obtain original images captured by at least two cameras.

[0105] S4. Dedistort each original image and then stitch them together to obtain a non-distorted image.

[0106] S6. Obtain a preset algorithm model.

[0107] In some embodiments, a lightweight semantic segmentation network based on the LiteSeg network is designed. The network is divided into encoding and decoding parts. The encoding part first uses the MobileNet network structure as the backbone network to extract features, then uses DASPP (Dense Atrous Spatial Pyramid Pooling) to obtain features of different spatial scales. The input and output of DASPP are combined through short residual connections as the encoder output. In the decoding part, the encoder output is then fused with the first layer output of the backbone network through a long residual connection, and the final segmentation result is obtained through upsampling.

[0108] S8. Classify and identify the parking frame edge lines and parking frame corner points in the non-distorted image using a preset algorithm model to obtain a parking frame edge line classification result and a parking frame corner point classification result in the non-distorted image.

[0109] S10: performing parking frame corner point matching on the parking frame edge classification results and the parking frame corner point classification results to obtain parameters of the parking frame in the non-distorted image.

[0110] S12. Calculate and obtain a target parking frame image based on the parameters of the parking frame.

[0111] For details of the above steps S2, S4, S8, S10 and S12, reference may be made to the above method embodiment.

[0112] See also Figure 4 , Figure 4 : is a flow chart of a method for obtaining a preset algorithm model provided by an embodiment of the present invention. In this embodiment, the preset algorithm model is LiteSeg, such as Figure 4 As shown, S6, obtaining a preset algorithm model, including:

[0113] S61. Obtain training samples, where the training samples include batches of non-distorted image data.

[0114] S62. Labeling the batch of non-distorted image data according to preset labels to obtain labeled training samples. The preset labels include the parking frame edge position, parking frame corner point category, and parking frame corner point position.

[0115] S63. Input the labeled training samples into a preset algorithm model to output the parking box edge classification results and parking box corner classification results of the training samples. The parking box edge classification results include the parking box edge position, and the parking box corner classification results include the parking box corner category and parking box corner position.

[0116] S64. Constrain the relationship between the classification result and the preset label according to the classification loss function, so that the classification result output by the preset algorithm model is close to the preset label.

[0117] S65: The algorithm model whose output classification result is closest to the classification of the preset label is used as the preset algorithm model.

[0118] Steps S61 to S65 above are the specific process of LiteSeg training. Training samples can be obtained from public datasets. The main purpose of LiteSeg training is to determine a set of optimal parameters that can make the LiteSeg classification results closest to the real results.

[0119] The classification loss function is a non-negative real-valued function used to measure the degree of inconsistency between the model's predicted value and the true value. The smaller the classification loss function, the better the robustness of the model. Commonly used classification loss functions include hinge loss (such as hinge loss), cross entropy loss (such as cross entropy loss, softmax loss), etc. In this embodiment, the softmax loss function can be used to constrain the relationship between the classification result and the preset label, so that the classification result output by the preset algorithm model is close to the preset label.

[0120] See also Figure 5 , Figure 5 The flowchart of a method for obtaining parking frame parameters in an undistorted image provided by an embodiment of the present invention is shown. In some embodiments, the parking frame corner point matching is performed on the parking frame edge classification results and the parking frame corner point classification results to obtain the parking frame parameters in the undistorted image, including:

[0121] S71. Process the parking frame edge lines and parking frame corners in the non-distorted image to obtain a parking frame corner image.

[0122] See also Figure 6 , Figure 6 The following is a flow chart of a method for obtaining a parking frame corner image provided by an embodiment of the present invention. In some embodiments, the parking frame edge lines and parking frame corners in the undistorted image are processed to obtain the parking frame corner image, including:

[0123] S711. Mask and refine the parking frame edge lines and parking frame corners in the non-distorted image to obtain a parking frame mask image.

[0124] Specifically, the parking frame edge output by the network is masked and binarized, where the edge part takes a value of 255, and then the Zhang-Suen thinning algorithm is used to convert the regional parking frame edge into line segments.

[0125] S712. Segment the parking frame mask image according to the parking frame corner point classification result to obtain sub-mask images.

[0126] According to the difference in image grayscale corresponding to the different parking frame corner point categories mentioned above, they are separated into three types of corner point masks: T-type corner point, L-type corner point and I-type corner point, and a sub-mask image of each corner point is obtained.

[0127] S713: Perform connected domain analysis on the sub-mask image to determine the position coordinates of the corner points of the parking frame in the sub-mask image.

[0128] Connected domain analysis is performed on each corner point's sub-mask image using a fast connected domain detection and analysis algorithm. Noise is filtered out using an area threshold. In this step, since each corner point is labeled as a circle during network training and inherently conforms to the connected domain property, the centroid of the segmented connected domain is approximated as the center coordinate of the parking frame corner point. The center coordinate of the parking frame corner point represents its location.

[0129] S714: Crop the sub-mask image according to the position coordinates of the parking frame corner points and the preset cropping size to obtain a parking frame corner point image.

[0130] After obtaining the center coordinates of the parking frame corner points, a mask sub-image of the corresponding size is cropped from the sub-mask image with the center of the parking frame corner points as the center point of a 50×50 pixel square area. The mask sub-image includes a parking frame corner point, and the mask sub-image is the parking frame corner point image.

[0131] S72. Traverse the parking frame corner points of the parking frame corner point image to obtain attributes of the parking frame corner points.

[0132] See also Figure 7 , Figure 7 1 is a flow chart of a method for obtaining attributes of parking frame corner points provided by an embodiment of the present invention. In some embodiments, traversing the parking frame corner points of the parking frame corner point image to obtain the attributes of the parking frame corner points includes:

[0133] S721. Obtain a traversal strategy according to the parking frame corner point image and the categories of the parking frame corner points in the parking frame corner point image.

[0134] S722. According to the traversal strategy, traverse the parking frame corner points of the parking frame corner point image to obtain attributes of the parking frame corner points.

[0135] The attributes of the parking frame corner points include valid attributes and invalid attributes, which are used to remove invalid parking frame corner points.

[0136] Specifically, for the parking frame corner point image obtained in the previous step, traverse the parking frame corner point categories. First, traverse the grayscale values of the boundary pixels in the parking frame corner point image. For pixels with non-zero grayscale values, record them as end points. The end point attributes include their coordinates and type. The coordinates represent the location of the end point, and the type represents the type of the end point, including upper boundary end points, right boundary end points, lower boundary end points, and left boundary end points. For example, position 0 can be defined as the upper boundary, position 1 as the right boundary, position 2 as the lower boundary, and position 3 as the left boundary.

[0137] The traversal strategies include: T-shaped corner point traversal strategy, L-shaped corner point traversal strategy and I-shaped corner point traversal strategy.

[0138] The traversal strategy for T-type corner points is as follows: Determine whether the number of end points in the parking frame corner image is 3. If so, the parking frame corner point classification is correct. If not, the parking frame corner point classification is incorrect. This corner point is set as an invalid point and filtered out in subsequent algorithms. Similarly, the traversal strategy for L-type corner points is as follows: Determine whether the number of end points in the parking frame corner image is 2. If so, the parking frame corner point classification is correct. If not, this corner point classification is incorrect. This corner point is set as an invalid point and filtered out in subsequent algorithms. The traversal strategy for I-type corner points is as follows: Determine whether the number of end points in the parking frame corner image is 1. If so, the parking frame corner point classification is correct. When the number of end points in the parking frame corner point image is not 1, it means that there is an error in the parking frame corner point category. The corner point is set as an invalid point and filtered out in the subsequent algorithm.

[0139] The above method can filter out parking frame corner points with invalid attributes, that is, invalid points.

[0140] S73. Match the corner points of the parking frame according to the attributes of the corner points of the parking frame to obtain a matching relationship between the corner points of the parking frame.

[0141] See also Figure 8 , Figure 8 The flowchart of the method for obtaining the matching relationship of parking frame corner points provided by an embodiment of the present invention is shown in FIG. In some embodiments, the parking frame corner points are matched according to the attributes of the parking frame corner points to obtain the matching relationship of the parking frame corner points, including:

[0142] S731. Obtain a corner point matching strategy based on the category of the parking frame corner point.

[0143] The corner point matching strategy is to place all parking frame corner points into a parking frame sequence and then perform a cyclic pairwise matching test. First, pairwise matching is tested to see if the categories of the two parking frame corner points are potentially matched. There are ninety-six possible matching possibilities for corner points in each of the twelve subcategories of the three categories T, L, and I. If a possible match is found between the two parking frame corner points, the spatial relationship between the two parking frame corner points is further tested. The angle between the line connecting the two parking frame corner points i and k, the orientation angle of the end point of parking frame corner point i, and the orientation angle of the end point of parking frame corner point k are calculated. If the difference between these three values is within a set threshold, the two parking frame corner points are considered to meet the conditions, i.e., the two parking frame corner points match. Otherwise, the two parking frame corner points are considered to be mismatched.

[0144] S732. Match the corner points of the parking frame according to the corner point matching strategy and the attributes of the corner points of the parking frame to obtain a first matching result.

[0145] The first matching result is among the ninety-six possible pairings.

[0146] S733. According to the first matching result, perform angle judgment on the two parking frame corner points in the first matching result. When the two parking frame corner points meet the preset conditions, determine that the two parking frame corner points in the first matching result are matched, that is, the matching relationship between the two parking frame corner points.

[0147] See also Figure 9 , Figure 9 2 is a schematic diagram of the positions of the parking frame corner point i and the parking frame corner point k provided in an embodiment of the present invention.

[0148] In the undistorted image, when the categories of parking frame corner point i and parking frame corner point k meet the ninety-six possible pairings, first use the center coordinates (x i ,y i ) is the coordinate origin, and the center coordinate of the parking frame corner point k (x k ,y k ) as the endpoints, calculate the angle θ of the line connecting the two parking frame corner points i and k, and the calculation formula is

[0149] In the parking frame corner point image, the parking frame corner point i is taken as the origin, and the end point (x epi ,y epi ) is the endpoint, and the angle α of the end point of the parking frame corner point i is calculated using the following formula: The end point (x epk ,y epk ) is the origin, the parking frame corner point k is the endpoint, and the parking frame corner point k is calculated to calculate the end point orientation angle β. The calculation formula is:

[0150] After calculating the angle θ between the line connecting corner points i and k, the orientation angle α of the end point of corner point i, and the orientation angle β of the end point of corner point k, the first absolute value of the difference between the angle θ connecting the line connecting corner points i and k and the orientation angle α of the end point of corner point i is calculated, as well as the second absolute value of the difference between the angle θ connecting the line connecting corner points i and k and the orientation angle β of the end point of corner point k is calculated. If both the first and second absolute values are less than a preset threshold, the center distance between the two corner points i and k is calculated and compared with the shortest distance between the end points of corner point i. If the current center distance is greater than the shortest distance, the data is not updated. Otherwise, the current center distance is set as the shortest distance and the corner point matching information is updated, indicating that the two corner points i and k are matched.

[0151] S734. Repeat the above steps to obtain the matching relationship of all parking frame corner points.

[0152] When parking frame corner point i is matched, the point pair that matches parking frame corner point i is stored in a point pair sequence. This point pair sequence is the matching relationship of parking frame corner point i. Then, the matching process begins for the next parking frame corner point until all parking frame corner points are matched, obtaining the matching relationship of all parking frame corner points.

[0153] S74. Obtain parameters of the parking frame in the undistorted image based on the matching relationship between the corner points of the parking frame.

[0154] Specifically, all the above point pair sequences are tested. First, two sets of point pairs are checked for overlapping parking frame corners, thereby determining whether the two sets of point pairs may belong to the same parking frame. Then, the orientation vectors of the two sets of point pairs are calculated (the direction vector of the line connecting the point pairs is superimposed with the direction vector of the point pairs pointing to the center of the parking frame). If the angle between the orientation vectors of the two sets of point pairs meets the conditions and is less than a preset threshold, the parking space's center point, length, width, and orientation angle are determined based on the classification of the point pairs.

[0155] See also Figure 10 , Figure 10 3 is a vector diagram of i-point pairs and k-point pairs provided in an embodiment of the present invention.

[0156] Read the point pair sequence and traverse the point pair sequence. For the point pair with sequence number i, i.e., point pair i, match it with the point pair with sequence number i+1, i.e., point pair k. First, check whether the i-point pair and the k-point pair have overlapping parking frame corner points. If there are overlapping nodes between the two sets of point pairs, first calculate the direction vector of the i-point pair. Assume that the coordinates of the corner points of the parking frame that coincide with each other in the i-point pair are (x iA ,y iA ), the coordinates of the non-overlapping parking frame corners are (x iB ,y iB ), then the direction vector coordinates are The coordinates of the non-overlapping corner points of the parking frame in the k-point pair are (x kB ,y kB ), the k-point pair is calculated similarly

[0157] According to the above 96 possible pairings, we can determine whether the i-point pair and the k-point pair can uniquely determine the direction of the parking frame. When the i-point pair can determine the direction of the parking frame, we define the i-point pair frame feature vector Let the coordinates of the end point of point i aligned with the parking frame be (x epi ,y epi ), the coordinates of the frame eigenvector of point pair i are The same calculation is done for k-point pairs

[0158] If both the i-point pair and the k-point pair have frame eigenvectors, then the direction vector of the i-point pair and the frame eigenvector are superimposed to form the pointing vector The k-point pair is obtained in the same way Calculate two vectors and When the angle is less than the preset threshold, it means that the two points form a parking box.

[0159] After determining the point-pair matching relationship of the parking frame, the parameters of the parking frame are calculated. The parameters of the parking frame include: the coordinates of the center point of the parking frame, the length of the parking frame, the width of the parking frame, and the orientation angle of the parking frame.

[0160] The coordinates of the center point of the parking frame are The length of the parking frame is the larger length of the i-point pair and the k-point pair (x iB -x iA ) 2 +(y iB -y iA ) 2 The width of the parking frame is the smaller length of the i-point pair and the k-point pair (x kB -x iA ) 2 +(y kB -y iA ) 2 The orientation angle of the parking frame is the direction vector of the corresponding point pair in the length direction of the parking frame, that is,

[0161] After obtaining the parameters of the parking frame, a complete parking frame pattern can be drawn according to the parameters.

[0162] See also Figure 11 and Figure 12 , Figure 11 is a parking frame mask image provided by an embodiment of the present invention, Figure 12 is the target parking frame image provided by the embodiment of the present invention. Figure 11 and Figure 12 As shown. In an embodiment of the present invention, a method for matching parking frame corners based on the parking frame edge and corner classification results to obtain parking frame parameters in an undistorted image combines the classification results of parking frame corners with those of parking frame edges. Point pairs are generated through the matching relationships between parking frame corners, and then the parking frame is generated based on the matching relationships between point pairs. This method has a clear logical hierarchy and can generate stable and accurate parking frame recognition results even when camera sampling is poor. Compared to previous methods of directly generating detection results using parking frame edge detection and machine learning, the method proposed in this embodiment of the present invention offers convenient parameter adjustment and greater anti-interference capabilities.

[0163] The parking frame recognition method provided by an embodiment of the present invention is applied to a vehicle equipped with at least two cameras. The method first obtains original images captured by the at least two cameras. Next, each original image is dedistorted and then stitched together to obtain an undistorted image. The method then classifies and identifies the parking frame edges and corners in the undistorted image to obtain a parking frame edge and corner classification result in the undistorted image. The parking frame corners are then matched against the classified results to obtain parking frame parameters in the undistorted image. Finally, a target parking frame image is calculated based on the parking frame parameters. Therefore, the parking frame recognition method obtains a more accurate target parking frame image by identifying the parking frame edges and corners. Furthermore, the method is more robust and has a lower missed detection rate.

[0164] See also Figure 13 , Figure 13 FIG2 is a schematic diagram of the structure of a parking frame recognition device provided by an embodiment of the present invention. The parking frame recognition device 10 is applied to a vehicle, wherein the vehicle is equipped with at least two cameras. The device includes: an acquisition module 101, a dedistortion module 102, a classification and recognition module 103, a corner point matching module 104, and a calculation module 105.

[0165] The acquisition module 101 is used to acquire original images captured by at least two cameras. The dedistortion module 102 is used to dedistort each of the original images and then stitch them together to obtain an undistorted image. The classification and recognition module 103 is used to classify and recognize the parking frame edges and corners in the undistorted image to obtain a classification result for the parking frame edges and corners in the undistorted image. The corner matching module 104 is used to perform parking frame corner matching on the parking frame edges and corners classification results to obtain the parameters of the parking frame in the undistorted image. The calculation module 105 is used to calculate and obtain a target parking frame image based on the parking frame parameters.

[0166] It should be noted that the aforementioned parking frame recognition device can execute the parking frame recognition method provided in the embodiments of the present invention and has the corresponding functional modules and beneficial effects of executing the method. For technical details not fully described in the embodiments of the parking frame recognition device, please refer to the parking frame recognition method provided in the embodiments of the present invention.

[0167] See also Figure 14 , Figure 14 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, which can be used to execute the parking frame recognition method described above. Figure 14 As shown, the electronic device 20 includes:

[0168] One or more processors 21 and memory 22, Figure 14 A processor 21 is taken as an example.

[0169] The processor 21 and the memory 22 may be connected via a bus or other means. Figure 14 The bus connection is taken as an example.

[0170] The memory 22 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the program instructions / modules corresponding to the parking frame recognition method in the embodiment of the present invention (for example, the attached Figure 13 The processor 21 executes the various functional applications and data processing of the parking frame recognition device by running the non-volatile software programs, instructions, and modules stored in the memory 22, thereby implementing the parking frame recognition method of the above-mentioned method embodiment.

[0171] The memory 22 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function; the data storage area may store data generated based on the use of the parking frame recognition device. Furthermore, the memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 22 may optionally include a remote memory device relative to the processor 21. Such remote memory device may be connected to the parking frame recognition device via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0172] One or more modules are stored in the memory 22, and when executed by one or more processors 21, perform the parking box recognition method in any of the above method embodiments, for example, perform the above described Figure 1Steps S1 to S9 of the method, Figure 2 Steps S51 to S54 of the method, Figure 3 Steps S2 to S12 of the method, Figure 4 Steps S61 to S65 of the method, Figure 5 Steps S71 to S74 of the method, Figure 6 Steps S711 to S714 of the method, Figure 7 Steps S721 to S722 of the method, Figure 8 Steps S731 to S734 of the method are implemented Figure 13 The functions of modules 101-105 in FIG.

[0173] The above-mentioned product can execute the parking frame recognition method provided by the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the parking frame recognition method provided by the embodiment of the present invention.

[0174] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are used by an electronic device to execute the parking frame recognition method in any of the above method embodiments, for example, to execute the above-described Figure 1 Steps S1 to S9 of the method, Figure 2 Steps S51 to S54 of the method, Figure 3 Steps S2 to S12 of the method, Figure 4 Steps S61 to S65 of the method, Figure 5 Steps S71 to S74 of the method, Figure 6 Steps S711 to S714 of the method, Figure 7 Steps S721 to S722 of the method, Figure 8 Steps S731 to S734 of the method are implemented Figure 13 The functions of modules 101-105 in FIG.

[0175] An embodiment of the present invention provides a computer program product, including a computer program stored on a non-volatile computer-readable storage medium, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the parking frame recognition method in any of the above method embodiments, for example, the above described Figure 1 Steps S1 to S9 of the method, Figure 2 Steps S51 to S54 of the method, Figure 3 Steps S2 to S12 of the method, Figure 4 Steps S61 to S65 of the method, Figure 5Steps S71 to S74 of the method, Figure 6 Steps S711 to S714 of the method, Figure 7 Steps S721 to S722 of the method, Figure 8 Steps S731 to S734 of the method are implemented Figure 13 The functions of modules 101-105 in FIG.

[0176] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0177] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that each embodiment can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. It can be understood by those skilled in the art that all or part of the processes in the above embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Under the idea of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above. For the sake of simplicity, they are not provided in detail. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of this application.

Claims

1. A parking frame recognition method, applied to a vehicle, wherein: The vehicle is provided with at least two cameras, and the method comprises: Acquire original images respectively captured by the at least two cameras; Dedistorting each of the original images and then splicing them together to obtain a non-distorted image; Classifying and identifying parking frame edges and parking frame corners in the non-distorted image to obtain a parking frame edge line classification result and a parking frame corner point classification result in the non-distorted image; performing parking frame corner point matching on the parking frame edge line classification result and the parking frame corner point classification result to obtain parameters of the parking frame in the undistorted image; Calculating and obtaining a target parking frame image according to the parameters of the parking frame; Among them, the parking frame corner point matching is performed on the parking frame edge line classification result and the parking frame corner point classification result to obtain the parameters of the parking frame in the non-distorted image, including: processing the parking frame edge line and parking frame corner points in the non-distorted image to obtain a parking frame corner point image; traversing the parking frame corner points of the parking frame corner point image to obtain the attributes of the parking frame corner points; matching the parking frame corner points according to the attributes of the parking frame corner points to obtain the matching relationship of the parking frame corner points; obtaining the parameters of the parking frame in the non-distorted image according to the matching relationship of the parking frame corner points.

2. The method according to claim 1, characterized in that The classifying and identifying the parking frame edge lines and parking frame corner points in the non-distorted image to obtain a parking frame edge line classification result and a parking frame corner point classification result in the non-distorted image includes: Performing convolution processing on the undistorted image to obtain image features at different levels; fusing the image features at different levels by short residual connections to obtain a high-dimensional feature map; Upsampling the high-dimensional feature map so that the image size of the high-dimensional feature map is the same as the image size of the non-distorted image, so as to obtain a prediction result of the parking frame edge line and a prediction result of the parking frame corner point in the high-dimensional feature map after the size change; Based on the prediction results of the parking frame edge lines and the prediction results of the parking frame corner points, the parking frame edge line classification results and the parking frame corner point classification results are obtained.

3. The method according to claim 1, characterized in that The step of classifying and identifying the parking frame edge lines and parking frame corner points in the non-distorted image to obtain a parking frame edge line classification result and a parking frame corner point classification result in the non-distorted image is performed by a preset algorithm model; The method further includes obtaining a preset algorithm model, wherein obtaining the preset algorithm model includes: Acquire a training sample, wherein the training sample includes a batch of non-distorted image data; annotating the batch of undistorted image data according to preset labels to obtain annotated training samples, wherein the preset labels include parking frame edge positions, parking frame corner point categories, and parking frame corner point positions; Inputting the labeled training samples into the preset algorithm model to output a parking box edge classification result and a parking box corner classification result of the training samples, wherein the parking box edge classification result includes a parking box edge position, and the parking box corner classification result includes a parking box corner category and a parking box corner position; Constraining the relationship between the classification result and the preset label according to a classification loss function so that the classification result output by the preset algorithm model is close to the preset label; The algorithm model whose output classification result is closest to the classification of the preset label is used as the preset algorithm model.

4. The method according to any one of claims 1 to 3, characterized in that The processing of the parking frame edge lines and parking frame corners in the non-distorted image to obtain a parking frame corner image includes: Masking and thinning the parking frame edge lines and the parking frame corners in the undistorted image to obtain a parking frame mask image; Segmenting the parking frame mask image according to the parking frame corner point classification result to obtain sub-mask images; Performing a connected domain analysis on the sub-mask image to determine the position coordinates of the corner points of the parking frame in the sub-mask image; The sub-mask image is cropped according to the position coordinates of the parking frame corner points and a preset cropping size to obtain the parking frame corner point image.

5. The method according to any one of claims 1 to 3, characterized in that The traversing the parking frame corner points of the parking frame corner point image and obtaining attributes of the parking frame corner points includes: acquiring a traversal strategy according to the parking frame corner point image and the categories of the parking frame corner points in the parking frame corner point image; According to the traversal strategy, the parking frame corner points of the parking frame corner point image are traversed to obtain the attributes of the parking frame corner points.

6. The method according to any one of claims 1 to 3, characterized in that The matching of the corner points of the parking frame according to the attributes of the corner points of the parking frame to obtain a matching relationship of the corner points of the parking frame includes: Obtaining a corner point matching strategy according to the category of the parking frame corner point; Matching the corner points of the parking frame according to the corner point matching strategy and the attributes of the corner points of the parking frame to obtain a first matching result; According to the first matching result, an angle judgment is performed on the two corner points of the parking frame in the first matching result. When the two corner points of the parking frame meet a preset condition, it is determined that the two corner points of the parking frame in the first matching result are matched, that is, the two corner points of the parking frame are in a matching relationship; Repeat the above steps to obtain the matching relationship of all parking frame corner points.

7. A parking frame recognition device, applied to a vehicle, wherein: The vehicle is provided with at least two cameras, characterized in that the device comprises: An acquisition module, configured to acquire original images captured by the at least two cameras respectively; a dedistortion module, configured to perform dedistortion processing on each of the original images and then splice the original images to obtain a non-distorted image; a classification and recognition module, configured to classify and recognize parking frame edges and parking frame corners in the non-distorted image to obtain a parking frame edge line classification result and a parking frame corner point classification result in the non-distorted image; a corner point matching module, configured to perform parking frame corner point matching on the parking frame edge line classification result and the parking frame corner point classification result to obtain parameters of the parking frame in the undistorted image; a calculation module, for calculating and obtaining a target parking frame image according to the parameters of the parking frame; Among them, the corner point matching module is also used to: process the parking frame edge lines and parking frame corner points in the non-distorted image to obtain a parking frame corner point image; traverse the parking frame corner points of the parking frame corner point image to obtain the attributes of the parking frame corner points; match the parking frame corner points according to the attributes of the parking frame corner points to obtain the matching relationship of the parking frame corner points; obtain the parameters of the parking frame in the non-distorted image according to the matching relationship of the parking frame corner points.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-volatile computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 6.

Images