Parking space identification method and device, medium and electronic equipment
By introducing multi-scale modules and adaptive computing into the parking space recognition model, the problem of parking space recognition algorithm occupying a lot of computing power is solved, the parking space recognition speed and efficiency are improved, and the recognition accuracy and user experience are improved.
Patent Information
- Application Number
- CN202310612800.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-05-26
AI Technical Summary
In the prior art, parking space recognition algorithms occupy a large amount of computer computing power, resulting in slower recognition of parking spaces around the vehicle, making it difficult to meet the needs of rapid identification.
Using multi-scale modules, including separation submodules, multi-scale operation submodules and merged submodules, the input is divided into multiple slicing layers for calculation, reducing intermediate parameters, saving computer computing power, and optimizing feature extraction through adaptive operations .
Without reducing the accuracy of parking space recognition, the parking space recognition speed is significantly improved, the computer computing power is occupied, and the user experience is improved.
Smart Images

Figure CN119027904A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the fields of computers and vehicles, and in particular, to a parking space recognition method, device, medium and electronic equipment. Background Art
[0002] How to identify parking spaces around vehicles is a hot topic for technical personnel to study. In related technologies, identifying parking spaces around vehicles requires huge computing power. How to reduce the computer computing power occupied by parking space recognition algorithms so that computers can identify parking spaces around vehicles more quickly is an issue that needs to be solved urgently. Summary of the invention
[0003] The purpose of the present disclosure is to provide a parking space recognition method, device, medium and electronic equipment, which can reduce the occupation of computer computing power.
[0004] In order to achieve the above object, the present disclosure provides a parking space recognition method, the method comprising:
[0005] Acquire an environmental image around the vehicle;
[0006] Inputting the acquired environment image into a pre-trained parking space recognition model to obtain an output of the parking space recognition model;
[0007] Determine the parking space information around the vehicle according to the output of the parking space recognition model;
[0008] The parking space recognition model includes a multi-scale module, the input and output of the multi-scale module pass through a separation submodule, a multi-scale operation submodule and a merging submodule, the separation submodule is configured to evenly divide the input of the separation submodule into N slice layers according to the number of channels, and the output of the separation submodule is the N slice layers;
[0009] The multi-scale operation submodule is configured to output N layers of multi-scale operation results according to the N slice layers, wherein the first slice layer is convolved with the convolution kernel to obtain the first layer of multi-scale operation results; starting from the second slice layer, each slice layer is convolved with the convolution kernel to obtain the corresponding multi-scale operation result after the addition result of the multi-scale operation result of the previous layer is added to each slice layer;
[0010] The merging submodule is configured to merge N layers of multi-scale operation results and output the merged result.
[0011] Optionally, the multi-scale operator module further includes:
[0012] The adaptive submodule is configured to output the multi-scale operation results of the N layers after adaptive operation according to the multi-scale operation results of the N layers, wherein the input multi-scale operation results of each layer are convolved with the convolution kernel respectively to obtain the adaptive coefficients corresponding to the multi-scale operation results of each layer; the product of the multi-scale operation results of each layer and the corresponding adaptive coefficient is the multi-scale operation result of each layer after adaptive operation;
[0013] The merging submodule is configured to merge the multi-scale operation results of the N layers after the adaptive operation, and output the merged result.
[0014] Optionally, the environmental image is divided into a predetermined number of grids; the output of the parking space recognition model includes a first confidence that each grid contains a first corner point, a predicted grid of a second corner point corresponding to each grid, a second confidence that each grid contains a second corner point, parking space angle information corresponding to each grid, first corner point coordinate information corresponding to each grid, and second corner point coordinate information corresponding to each grid.
[0015] Optionally, determining the parking space information around the vehicle according to the output of the parking space recognition model includes:
[0016] For each grid, determining whether the grid includes a first corner point according to a first confidence level of the grid;
[0017] For each grid containing a first corner point, if the second confidence of the second corner point prediction grid corresponding to the grid is greater than a predetermined confidence threshold, the parking space information around the vehicle is determined based on the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, and the second corner point coordinate information corresponding to the second corner point prediction grid corresponding to the grid.
[0018] Optionally, the determining the parking space information around the vehicle according to the output of the parking space recognition model further includes:
[0019] For each grid including the first corner point, if the second confidence of the second corner point prediction grid corresponding to the grid is less than the confidence threshold, searching for the second confidence of the grid within a preset distance from the second corner point prediction grid corresponding to the grid;
[0020] If the found grid meets the re-matching condition, the parking space information around the vehicle is determined according to the second corner point coordinate information corresponding to the found grid, the parking space angle information, and the first corner point coordinate information to be paired with the found grid, wherein the re-matching condition includes that the second confidence of the found grid is greater than the confidence threshold, and the found grid is not the second corner point predicted grid corresponding to the remaining grids containing the first corner point.
[0021] Optionally, the output of the parking space recognition model also includes parking space occupancy information corresponding to each grid; the determining of the parking space information around the vehicle according to the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, and the second corner point coordinate information corresponding to the grid predicting the second corner point coordinate information corresponding to the grid comprises:
[0022] The parking space information around the vehicle is determined according to the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, the second corner point coordinate information corresponding to the grid, and the parking space occupancy information corresponding to the grid.
[0023] Optionally, the parking space recognition model outputs multiple features, and the parking space recognition model is trained according to the loss value corresponding to each feature.
[0024] Optionally, the loss value corresponding to each feature is determined according to the absolute error between the output value of the parking space recognition model and the label value and the KL divergence between the output value of the parking space recognition model and the label value.
[0025] Optionally, the loss value corresponding to each feature is determined according to the following formula:
[0026]
[0027] Among them, loss is the loss value, output is the output value of the parking space recognition model, α is the stretching coefficient, label is the label value, is the balance coefficient, k1 and k2 are preset minimum values. The present disclosure also provides a parking space recognition device, including:
[0028] The acquisition module is configured to acquire an environmental image around the vehicle; the acquisition module is configured to input the acquired environmental image into a pre-trained parking space recognition model to obtain an output of the parking space recognition model;
[0029] a determination module, configured to determine parking space information around the vehicle according to an output of the parking space recognition model;
[0030] The parking space recognition model includes a multi-scale module, the input and output of the multi-scale module pass through a separation submodule, a multi-scale operation submodule and a merging submodule, the separation submodule is configured to evenly divide the input of the separation submodule into N slice layers according to the number of channels, and the output of the separation submodule is the N slice layers;
[0031] The multi-scale operation submodule is configured to output N layers of multi-scale operation results according to the N slice layers, wherein the first slice layer is convolved with the convolution kernel to obtain the first layer of multi-scale operation results; starting from the second slice layer, each slice layer is convolved with the convolution kernel to obtain the corresponding multi-scale operation result after the addition result of the multi-scale operation result of the previous layer is added to each slice layer;
[0032] The merging submodule is configured to merge N layers of multi-scale operation results and output the merged result.
[0033] The present disclosure also provides a non-volatile computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above-mentioned parking space identification method are implemented.
[0034] The present disclosure also provides an electronic device, comprising:
[0035] a memory having a computer program stored thereon;
[0036] A processor is used to execute the computer program in the memory to implement the steps of the above-mentioned parking space recognition method.
[0037] Through the above technical solution, the parking space recognition model includes a multi-scale module, which includes a separation sub-module, a multi-scale operation sub-module and a merging sub-module. The separation sub-module is used to divide its input into N slice layers and then input them into the multi-scale operation sub-module for calculation. In this way, the intermediate parameters generated by the multi-scale module are greatly reduced, the computer computing power is saved, and the speed of parking space recognition is improved without reducing the recognition accuracy of the parking space recognition model, which can bring a better user experience to users.
[0038] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:
[0040] Figure 1 The figure is a flow chart of a parking space recognition method according to an exemplary embodiment of the present disclosure.
[0041] Figure 2 It is a block diagram of a parking space recognition model according to an exemplary embodiment of the present disclosure.
[0042] Figure 3 It is a block diagram of a multi-scale module shown in an exemplary embodiment of the present disclosure.
[0043] Figure 4 is a block diagram of a multi-scale operator module shown in an exemplary embodiment of the present disclosure.
[0044] Figure 5 is an image used for training a parking space recognition model shown in an exemplary embodiment of the present disclosure.
[0045] Figure 6 It is a flowchart of a parking space recognition method shown in yet another exemplary embodiment of the present disclosure.
[0046] Figure 7 The figure is a block diagram of a parking space recognition device according to an exemplary embodiment of the present disclosure.
[0047] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0048] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.
[0049] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.
[0050] Figure 1 FIG. 1 is a flow chart of a parking space recognition method according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the parking space recognition method includes steps S101 to S103.
[0051] In step S101 , an environment image around the vehicle is acquired.
[0052] In step S102, the acquired environment image is input into a pre-trained parking space recognition model to obtain the output of the parking space recognition model.
[0053] In step S103, the parking space information around the vehicle is determined according to the output of the parking space recognition model.
[0054] In one embodiment, the acquired environmental image around the vehicle may be a 360-degree panoramic bird's-eye view around the vehicle. The parking space recognition model may be a pre-trained neural network model for identifying parking spaces around the vehicle. For example, the parking space recognition model may recognize and output information such as the coordinates of the first corner point, the coordinates of the second corner point, the parking space line angle, etc. included in the environmental image for identifying parking spaces around the vehicle. After the acquired environmental image is input into the parking space recognition model, the output of the parking space recognition model may be obtained. Since the parking space recognition model can output information for identifying parking spaces around the vehicle, the parking space information around the vehicle may be determined based on the output of the parking space recognition model. The parking space information may be information indicating the parking space situation around the vehicle, such as the size of the parking space, the placement of the parking space (such as parallel parking spaces, vertical parking spaces, and diagonal parking spaces).
[0055] Among them, the parking space recognition model includes a multi-scale module, and the input and output of the multi-scale module pass through a separation sub-module, a multi-scale operation sub-module and a merging sub-module.
[0056] The separation submodule is configured to evenly divide the input of the separation submodule into N slice layers according to the number of channels, and the output of the separation submodule is the N slice layers.
[0057] The multi-scale operation submodule is configured to output N layers of multi-scale operation results according to the N slice layers, wherein the first slice layer is convolved with the convolution kernel to obtain the first layer of multi-scale operation results; starting from the second slice layer, each slice layer is convolved with the convolution kernel to obtain the corresponding multi-scale operation result after the addition result of the multi-scale operation result of the previous layer is added to the convolution kernel.
[0058] The merging submodule is configured to merge the N-layer multi-scale operation results and output the merged result.
[0059] Figure 2 FIG. 1 is a block diagram of a parking space recognition model according to an exemplary embodiment of the present disclosure. Figure 2 As shown, the input of the parking space recognition model passes through 5 convolutional layers, 5 multi-scale modules and a detection head to obtain the output of the parking space recognition model. In one embodiment, the step size of the convolutional layer used in the parking space recognition model can be set to 2, so that the length and width of the feature map obtained after the input of the convolutional layer passes through the convolutional layer can be reduced to half of the length and width before passing through the convolutional layer. In order to ensure the integrity of the feature information, when setting the number of channels of the convolutional layer, the number of channels of the next convolutional layer can be set to twice the number of channels of the previous convolutional layer. For example, Figure 2 The number of channels of the first convolutional layer is set to 64, the number of channels of the second convolutional layer is set to 128, the number of channels of the third convolutional layer is set to 256, the number of channels of the fourth convolutional layer is set to 512, and the number of channels of the fifth convolutional layer is set to 1024.
[0060] For the detection head of the parking space recognition model, a convolution kernel with a size of 3*3 may be used. In one implementation, when the parking space recognition model outputs, a sigmoid activation function may be used to make the value output by the parking space recognition model between 0 and 1.
[0061] Figure 3 FIG. 1 is a block diagram of a multi-scale module shown in an exemplary embodiment of the present disclosure. Figure 3 As shown in FIG, the multi-scale module includes a separation submodule, a multi-scale operation submodule and a merging submodule. Figure 3 As shown, the multi-scale module includes multiple convolution kernels. The multi-scale module input is convolved with the first convolution kernel to obtain the input of the separation submodule. The separation submodule is configured to divide the input of the separation submodule into N slice layers on average according to the number of channels. N is a preset value, for example, N can be 2, 3, or 4. In one embodiment, the number of channels of the input of a separation submodule is 64, and the separation submodule can separate the 64 channels and divide them into 4 slice layers on average (when N is 4) to obtain the output of the separation submodule. For example, the 1st to 16th layers of channels in the input of the separation submodule are used as the first slice layer, the 17th to 32nd layers of channels in the input of the separation submodule are used as the second slice layer, the 33rd to 48th layers of channels in the separation submodule are used as the third slice layer, and the 49th to 64th layers of channels in the separation submodule are used as the fourth slice layer.
[0062] like Figure 3 As shown in , the separation submodule divides its input into four slice layers and then inputs them into the multi-scale operation submodule. The multi-scale operation result is the output of the multi-scale operation submodule. Figure 3 , after the first slice layer is convolved with the second convolution kernel, the first layer multi-scale operation result is obtained. The sum of the first layer multi-scale operation result and the second slice layer is convolved with the third convolution kernel to obtain the second layer multi-scale operation result. The sum of the second layer multi-scale operation result and the third slice layer is convolved with the fourth convolution kernel to obtain the third layer multi-scale operation result. The sum of the third layer multi-scale operation result and the fourth slice layer is convolved with the fifth convolution kernel to obtain the fourth layer multi-scale operation result. Thus, 4 layers of multi-scale operation results can be obtained. The multi-scale operation submodule can input the 4 layers of multi-scale operation results into the merging submodule. The merging submodule can merge the obtained 4 layers of multi-scale operation results so that the output channel number of the merging submodule is 64 (that is, the separation submodule divides the input 64 channels into 4 slice layers, and the merging submodule merges the multi-scale operation results corresponding to the 4 slice layers). The merged result is the output of the merging submodule.
[0063] The output of the merging submodule is convolved with the sixth convolution kernel to obtain a result which is added to the multi-scale module input, and the obtained addition result can be used as the multi-scale module output.
[0064] Compared with the residual module in the related art, the multi-scale module can extract channel multi-scale information, and in the calculation process, the number of parameters generated is less than the number of parameters generated in the residual module calculation process in the related art, saving computer computing power. Since starting from the second slice layer, the sum of each slice layer and the multi-scale operation result of the previous layer is convolved with the convolution kernel to obtain the corresponding multi-scale operation result, so that the scale information of each layer of multi-scale operation result is different.
[0065] Through the above technical solution, the parking space recognition model includes a multi-scale module, which includes a separation sub-module, a multi-scale operation sub-module and a merging sub-module. The separation sub-module is used to divide its input into N slice layers and then input them into the multi-scale operation sub-module for calculation. In this way, the intermediate parameters generated by the multi-scale module are greatly reduced, the computer computing power is saved, and the speed of parking space recognition is improved without reducing the recognition accuracy of the parking space recognition model, which can bring a better user experience to users.
[0066] In yet another embodiment, the multi-scale operation submodule further includes an adaptive submodule.
[0067] The adaptive submodule is configured to output the multiscale operation results of the N layers after adaptive operation according to the multiscale operation results of the N layers, wherein the input multiscale operation results of each layer are convolved with the convolution kernel respectively to obtain the adaptive coefficients corresponding to the multiscale operation results of each layer; the product of the multiscale operation results of each layer and the corresponding adaptive coefficient is the multiscale operation result of each layer after adaptive operation. The merging submodule is configured to merge the multiscale operation results of the N layers after adaptive operation and output the merged result.
[0068] Figure 4 FIG. 1 is a block diagram of a multi-scale operator module shown in an exemplary embodiment of the present disclosure. Figure 4 As shown, the multi-scale operation submodule also includes an adaptive submodule. Figure 4 As shown, the multi-scale operation submodule inputs the 4 slice layers output by the separation submodule and obtains 4 layers of multi-scale operation results. After the 4 layers of multi-scale operation results are input into the adaptive submodule, the multi-scale operation results after adaptive operation are obtained.
[0069] The adaptive operation process includes:
[0070] (1) After the multi-scale operation result is convolved with the convolution kernel, an adaptive coefficient corresponding to the multi-scale operation result is obtained, wherein it should be noted that the adaptive coefficient is the result obtained after the multi-scale operation result is convolved with the convolution kernel in the adaptive submodule;
[0071] (2) The multi-scale operation result is multiplied by its corresponding adaptive coefficient to obtain the multi-scale operation result after adaptive operation.
[0072] like Figure 4 As shown, the first layer multi-scale operation result is convolved with the seventh convolution kernel to obtain an adaptive coefficient corresponding to the first layer multi-scale operation result, the second layer multi-scale operation result is convolved with the seventh convolution kernel to obtain an adaptive coefficient corresponding to the second layer multi-scale operation result, the third layer multi-scale operation result is convolved with the seventh convolution kernel to obtain an adaptive coefficient corresponding to the third layer multi-scale operation result, and the fourth layer multi-scale operation result is convolved with the seventh convolution kernel to obtain an adaptive coefficient corresponding to the fourth layer multi-scale operation result. In one embodiment, the seventh convolution kernel can be a convolution kernel with a layer size information of 1*1.
[0073] like Figure 4 As shown in , after obtaining the adaptive coefficients corresponding to the multi-scale operation results of each layer, the multi-scale operation results of each layer are multiplied by their corresponding adaptive coefficients to obtain the multi-scale operation results after adaptive operation. Figure 4 As shown, the multi-scale operation result of the first layer is multiplied by the adaptive coefficient corresponding to the multi-scale operation result of the first layer to obtain the multi-scale operation result of the first layer after adaptive operation; the multi-scale operation result of the second layer is multiplied by the adaptive coefficient corresponding to the multi-scale operation result of the second layer to obtain the multi-scale operation result of the second layer after adaptive operation; the multi-scale operation result of the third layer is multiplied by the adaptive coefficient corresponding to the multi-scale operation result of the third layer to obtain the multi-scale operation result of the third layer after adaptive operation; the multi-scale operation result of the fourth layer is multiplied by the adaptive coefficient corresponding to the multi-scale operation result of the fourth layer to obtain the multi-scale operation result of the fourth layer after adaptive operation.
[0074] The merging submodule inputs the multi-scale operation results of each layer after the adaptive operation, merges the multi-scale operation results of each layer after the adaptive operation, and outputs the merged result.
[0075] In this embodiment, the multi-scale operation submodule also includes an adaptive submodule, which can determine the adaptive coefficient corresponding to each layer of multi-scale operation results, and multiply each layer of multi-scale operation results by the corresponding adaptive results to obtain the multi-scale operation results after adaptive operation. In this way, the internal parameters in the parking space recognition model are optimized, so that the recognition accuracy of the parking space recognition model is higher.
[0076] In another embodiment, the environmental image is divided into a predetermined number of grids; the output of the parking space recognition model includes a first confidence that each grid contains a first corner point, a predicted grid of a second corner point corresponding to each grid, a second confidence that each grid contains a second corner point, parking space angle information corresponding to each grid, first corner point coordinate information corresponding to each grid, and second corner point coordinate information corresponding to each grid.
[0077] After the environment image is recognized by the parking space recognition model, it can be divided into a predetermined number of grids. For example, in one embodiment, the environment image is an image with a resolution of 384*384. After the environment image is recognized by the parking space recognition model, the environment image can be divided into 12*12 grids. In one embodiment, after the environment image with a resolution of 384*384 is input into the parking space recognition model, it passes through 5 convolution layers with a step size of 2, thereby adjusting the output size to 12*12, thereby dividing the environment image into 12*12 grids.
[0078] The first confidence level is a parameter used to determine whether a grid contains a first corner point. For example, the first confidence level can be used to compare with a predetermined threshold value. If the first confidence level is greater than the predetermined threshold value, it indicates that the grid contains the first corner point. In one embodiment, in the feature map output by the parking space recognition model, each grid in the 12*12 grids can have a first confidence level corresponding to the grid, which is used to determine whether each grid in the 12*12 grids contains the first corner point.
[0079] It should be noted that the first corner point and the second corner point refer to the two outer corner points of the four corner points of the parking space (i.e., the two corner points on a parking space line for vehicles to enter and exit among the four corner points of the parking space). In the top view of the parking space, the four parking space lines of the parking space form a closed loop. The direction from the first corner point to the second corner point along a parking space line for vehicles to enter and exit is counterclockwise. For example, Figure 5 As shown, Figure 5 This is a top view including parking spaces. Figure 5 Divided into 12*12 grids. Figure 5 In the example, the grid in the 4th row and 8th column and the grid in the 8th row and 8th column each contain an outer corner point of the parking space. Draw a line connecting the outer corner point in the grid in the 4th row and 8th column and the outer corner point in the grid in the 8th row and 8th column (i.e., the parking space line), and the direction from the outer corner point in the grid in the 4th row and 8th column to the outer corner point in the grid in the 8th row and 8th column along the line is counterclockwise in the closed loop formed by the four corner points of the parking space. Therefore, the outer corner point in the grid in the 4th row and 8th column is the first corner point, and the outer corner point in the grid in the 8th row and 8th column is the second corner point.
[0080] For example, in the feature map output by the parking space recognition model that records the first confidence that each grid contains the first corner point, if the first confidence corresponding to the grid with grid coordinates (8,3) (the grid in the 8th column and 3rd row of the 12*12 grids) is greater than a predetermined threshold, then it can be considered that the grid with grid coordinates (8,3) contains the first corner point.
[0081] For a certain grid, the second corner point prediction grid corresponding to the grid refers to the grid where the second corner point corresponding to the first corner point in the grid is predicted to be located. The second corner point corresponding to the first corner point refers to the second corner point that belongs to the same parking space as the first corner point. For example, in the feature map output by the parking space recognition model that records the second corner point prediction grids corresponding to each grid, the grid with grid coordinates (8,3) records the grid coordinates (8,5), then the second corner point prediction grid corresponding to the grid with grid coordinates (8,3) is (8,5). In other words, corresponding to the first corner point in the grid with grid coordinates (8,3), the second corner point corresponding to the first corner point is predicted to be in the grid with grid coordinates (8,5).
[0082] The second confidence is a parameter used to determine whether a grid contains a second corner point. For example, if the second confidence that a grid contains a second corner point is greater than a predetermined confidence threshold, it can be determined that the grid contains the second corner point. The confidence threshold can be preset, and technicians can preset the confidence threshold based on experience. For example, in the feature map output by the parking space recognition model that records the second confidence that each grid contains a second corner point, the second confidence corresponding to the grid with grid coordinates (8,5) is greater than the confidence threshold, then it can be considered that the grid with grid coordinates (8,5) contains the second corner point.
[0083] The parking space angle information corresponding to the grid refers to the angle of the parking space line passing through the corner points contained in the grid. The parking space angle corresponding to the grid can be used to assist in identifying the layout of the parking space (parallel parking space, vertical parking space, diagonal parking space).
[0084] The first corner point coordinate information corresponding to the grid refers to the specific coordinates of the first corner point contained in the grid (the specific coordinates are different from the grid coordinates and can be the specific coordinates of the corner point in the environment image). The second corner point coordinate information corresponding to the grid refers to the specific coordinates of the second corner point contained in the grid.
[0085] In this embodiment, the output of the parking space recognition model includes a first confidence that each grid contains a first corner point, a predicted grid of a second corner point corresponding to each grid, a second confidence that each grid contains a second corner point, parking space angle information corresponding to each grid, first corner point coordinate information corresponding to each grid, and second corner point coordinate information corresponding to each grid. In this way, the parking spaces around the vehicle can be directly determined based on the information output by the parking space recognition model, and "end-to-end" prediction of parking spaces can be achieved without the need for other image processing steps, and the detection performance and efficiency are good.
[0086] In yet another embodiment, determining parking space information around the vehicle according to the output of the parking space recognition model includes:
[0087] For each grid, determining whether the grid contains a first corner point according to a first confidence level of the grid;
[0088] For each grid containing a first corner point, if the second confidence of the second corner point prediction grid corresponding to the grid is greater than a predetermined confidence threshold, the parking space information around the vehicle is determined based on the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, and the second corner point coordinate information corresponding to the grid prediction grid.
[0089] For example, the first confidence level of each grid in the 12*12 grids output by the parking space recognition model may be traversed to determine the grid containing the first corner point in the 12*12 grids.
[0090] Afterwards, the second corner point prediction grid corresponding to each grid containing the first corner point is searched, and it is determined whether the second confidence of the second corner point prediction grid corresponding to the grid is greater than the confidence threshold.
[0091] If the second confidence of the second corner point prediction grid corresponding to a certain grid is greater than the confidence threshold, it can be considered that the second corner point prediction grid corresponding to the grid includes the second corner point. In this way, a group of corresponding first corner points and second corner points can be obtained, and the parking space information corresponding to the group of first corner points and second corner points can be determined according to the first corner point coordinate information corresponding to the grid, the second corner point coordinate information corresponding to the second corner point prediction grid corresponding to the grid, and the parking space angle information corresponding to the grid. For each grid containing the first corner point, the parking space information corresponding to the first corner point in the grid can be determined.
[0092] In the related art, parking space recognition is performed through traditional image processing methods (such as edge detection, Hough transform, etc.). Since these image processing methods are difficult to process complex scenes, they can only detect parking spaces in a single scene, and are difficult to process complex scenes and difficult to suppress noise. In the related art, the algorithm based on semantic segmentation can process complex scenes and suppress noise when performing parking space recognition, but the output of this type of algorithm is the outline of the parking space line, and more cumbersome post-processing is required to correctly analyze the specific coordinates of the parking space.
[0093] In this embodiment, whether a grid contains a first corner point is determined based on the first confidence level of each grid, and for the grid containing the first corner point, whether the second confidence level of the corresponding second corner point prediction grid is greater than the confidence level threshold is determined to complete the matching of the first corner point and the second corner point. Then, the first corner point coordinate information, the second corner point coordinate information corresponding to the second corner point prediction grid, and the parking space angle information are combined to determine the parking space information around the vehicle without image processing and tedious post-processing. The method is simple and has good robustness.
[0094] In yet another embodiment, determining parking space information around the vehicle according to the output of the parking space recognition model further includes:
[0095] For each grid including the first corner point, if the second confidence of the second corner point prediction grid corresponding to the grid is less than the confidence threshold, searching for the second confidence of the grid within a preset distance from the second corner point prediction grid corresponding to the grid;
[0096] If the found grid meets the re-matching condition, the parking space information around the vehicle is determined according to the second corner point coordinate information corresponding to the found grid, the parking space angle information, and the first corner point coordinate information to be paired with the found grid, wherein the re-matching condition includes that the second confidence of the found grid is greater than the confidence threshold, and the found grid is not the second corner point predicted grid corresponding to the remaining grids.
[0097] For a grid containing a first corner point, if the second confidence of the second corner point prediction grid corresponding to the grid is less than the confidence threshold, it can be considered that the second corner point prediction grid corresponding to the grid does not contain the second corner point, that is, the first corner point contained in the grid is not matched with the second corner point. In such a case, it is possible that the grid coordinates of the second corner point prediction grid predicted by the parking space recognition model have errors. The second confidence of the grid within a preset distance from the second corner point prediction grid corresponding to the grid containing the first corner point can be searched. The preset distance can be preset based on experience. For example, the preset distance can be preset to 1 grid, that is, the second confidence of the grid within 1 grid from the second corner point prediction grid corresponding to the grid containing the first corner point is searched.
[0098] For example, the grid with grid coordinates (8,3) contains the first corner point, and the grid coordinates of the second corner point prediction grid corresponding to the grid are (8,5). The second confidence of the grid with grid coordinates (8,5) is less than the confidence threshold, and the second confidence of the grid one grid away from the grid with grid coordinates (8,5) can be found, that is, the second confidence of the eight grids with grid coordinates (7,4), (7,5), (7,6), (8,4), (8,6), (9,4), (9,5), (9,6) can be found. The first corner point to be paired with the found grid is the first corner point in the grid that contains the first corner point and the corresponding second corner point prediction grid whose second confidence is less than the confidence threshold. As in the example above, the first corner point to be paired with the eight grids with grid coordinates (7,4), (7,5), (7,6), (8,4), (8,6), (9,4), (9,5), and (9,6) is the first corner point contained in the grid with grid coordinates (8,3).
[0099] For the eight grids with grid coordinates of (7,4), (7,5), (7,6), (8,4), (8,6), (9,4), (9,5), and (9,6), if there is a grid among these eight grids that meets the re-pairing condition, it can be paired with the first corner point contained in the grid with grid coordinates of (8,3) to identify the corresponding parking space information.
[0100] The re-matching condition includes that the second confidence of the found grid is greater than the confidence threshold, and the found grid is not the second corner point prediction grid corresponding to the remaining grids. It is worth noting that if the second confidence of the found grid is greater than the confidence threshold, it can be considered that there is a second corner point in the found grid; and if the found grid is not the second corner point prediction grid corresponding to the remaining grids containing the first corner point, it can be considered that the second corner point in the found grid has no first corner point matching it. Therefore, it can be considered that the second corner point in the found grid is the second corner point that matches the first corner point (the first corner point to be matched) that has not yet matched the second corner point.
[0101] Continuing with the above example, if the grid with grid coordinates (7,4) meets the re-pairing condition, the parking space information around the vehicle can be determined based on the second corner point coordinate information corresponding to the grid with grid coordinates (7,4), the first corner point coordinate information corresponding to the grid with grid coordinates (8,3), and the parking space angle information corresponding to the grid with grid coordinates (8,3). That is, the parking space information corresponding to the second corner point in the grid with grid coordinates (7,4) and the first corner point in the grid with grid coordinates (8,3) is determined.
[0102] In this embodiment, for a first corner point that fails to successfully match a second corner point, an unpaired second corner point is continued to be searched for near a second corner point prediction grid corresponding to the grid where the first corner point is located to match the first corner point. In this way, the impact of the output error of the parking space recognition model can be reduced, the parking spaces in the vehicle's surroundings can be more accurately identified, and a better user experience can be brought to users.
[0103] In yet another embodiment, the output of the parking space recognition model also includes parking space occupancy information corresponding to each grid;
[0104] Determining parking space information around the vehicle according to parking space angle information corresponding to the grid, first corner point coordinate information corresponding to the grid, and second corner point coordinate information corresponding to the grid predicts the second corner point coordinate information corresponding to the grid, including:
[0105] The parking space information around the vehicle is determined according to the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, the second corner point coordinate information corresponding to the grid, and the parking space occupancy information corresponding to the grid.
[0106] The parking space occupancy information is information indicating the parking space occupancy situation. In this embodiment, the parking space recognition model also includes the parking space occupancy information, so that the determined parking space information around the vehicle also records the parking space occupancy situation, which facilitates the use of the user and brings a better user experience.
[0107] In one embodiment, the output of the parking space recognition model may include 11 output channels, that is, outputting 12*12*11 matrix information. The 11 output channels of the parking space recognition model may output the following information:
[0108] (1) The first output channel is used to output the confidence that each grid contains the first corner point;
[0109] (2) The second output channel is used to output the horizontal coordinate in the first corner point coordinate information corresponding to each grid, and the third output channel is used to output the vertical coordinate in the first corner point coordinate information corresponding to each grid;
[0110] (3) The fourth output channel is used to output the cosine value of the parking space line angle, and the fifth output channel is used to output the sine value of the parking space line angle, wherein the cosine value of the parking space line angle and the sine value of the parking space line angle are recorded in the grid where the first corner point of the parking space is located;
[0111] (4) The sixth output channel is used to output parking space occupancy information, wherein the parking space occupancy information is recorded in the grid where the first corner point of the parking space is located;
[0112] (5) The seventh output channel is used to output the horizontal coordinate of the second corner point prediction grid, and the eighth output channel is used to output the vertical coordinate of the second corner point prediction grid, wherein the horizontal coordinate of the second corner point prediction grid and the vertical coordinate of the second corner point prediction grid are recorded in the grid where the first corner point of the parking space is located;
[0113] (6) The ninth output channel is used to output the confidence that each grid contains the second corner point, wherein the confidence that each grid contains the second corner point is recorded in the grid where the second corner point of the parking space is located;
[0114] (7) The 10th output channel is used to output the horizontal coordinate in the second corner point coordinate information corresponding to each grid, and the 11th output channel is used to output the vertical coordinate in the second corner point coordinate information corresponding to each grid, wherein the horizontal coordinate and the vertical coordinate in the second corner point coordinate information are recorded in the grid where the second corner point of the parking space is located.
[0115] Figure 5 is an image used to train a parking space recognition model as shown in an exemplary embodiment of the present disclosure. Figure 5 As shown, in the image, the grid with grid coordinates (8,4) contains the first corner point, and the grid with grid coordinates (8,8) contains the second corner point.
[0116] This can be done by Figure 5 The labels of the training data are annotated as shown:
[0117] (1) In the first channel, fill the grid (8,4) with the number 1;
[0118] (2) In the second channel, the horizontal coordinate in the normalized first corner point coordinate information is filled at the grid (8, 4); in the third channel, the vertical coordinate in the normalized first corner point coordinate information is filled at the grid (8, 4);
[0119] (3) In the fourth channel, the cosine value of the parking space line angle is filled in the grid (8, 4), and in the fifth channel, the sine value of the parking space line angle is filled in the grid (8, 4); in one embodiment, since the output of the parking space recognition model as a whole uses the sigmoid function, the cosine value and the sine value can be added by 1 and then divided by 2, and the obtained quotient can be filled in the grid (8, 4);
[0120] (4) In the sixth channel, fill the grid (8,4) with the number 0, which indicates that the parking space is idle (if the parking space is occupied, the number 1, which indicates that the parking space is occupied, can be filled in when the parking space is occupied);
[0121] (5) In the seventh channel, the horizontal coordinate of the grid where the second corner point corresponding to the first corner point in the grid is located is filled in at the grid (8,4). The horizontal coordinate of the grid can be normalized and 0.667 (the quotient of 8 divided by 12) is filled in at the grid (8,4); in the eighth channel, the vertical coordinate of the grid where the second corner point corresponding to the first corner point in the grid is located is filled in at the grid (8,4). The vertical coordinate of the grid can be normalized and 0.667 (the quotient of 8 divided by 12) is filled in at the grid (8,4);
[0122] (6) Since the grid (8,8) contains the second corner point, fill the grid (8,8) with the number 1 in the 9th channel;
[0123] (7) In the 10th channel, the horizontal coordinate in the normalized second corner point coordinate information is filled at the grid (8,8); in the 11th channel, the vertical coordinate in the normalized second corner point coordinate information is filled at the grid (8,8).
[0124] For other grids in the 11 channels of the training data that are not mentioned above, the values of the remaining grids can be filled with 0.
[0125] In one embodiment, if the first corner point or the second corner point in the image serving as training data is occluded, the corner point can still be considered to be included in the grid, and the training data can be labeled as in the above example. In this way, the trained parking space recognition model can still recognize the parking space even when the corner point is occluded.
[0126] In another embodiment, if the first corner point or the second corner point is an isolated external corner point within the image, and another external corner point matching the external corner point is outside the image (due to the limited size and length of the image), the isolated external corner point may not be marked in the training data to reduce the impact of the isolated external corner point on the recognition result.
[0127] Figure 6 FIG. 1 is a flowchart of a parking space recognition method shown in another exemplary embodiment of the present disclosure. Figure 6 As shown, the parking space recognition method includes the following steps:
[0128] (1) Acquire the environment image around the vehicle;
[0129] (2) inputting the acquired environment image into a pre-trained parking space recognition model to obtain the output of the parking space recognition model;
[0130] (3) determining whether a first confidence level of each grid in the first output channel containing a first corner point is greater than a predetermined threshold, and determining the first corner point contained in the environment image;
[0131] (4) for each grid including the first corner point, determining whether a second confidence level of a second corner point prediction grid corresponding to the grid is greater than a predetermined confidence level threshold;
[0132] If yes, pairing the first corner point in the grid containing the first corner point with the second corner point in the second corner point prediction grid;
[0133] If not, determine whether the grid within a preset distance from the second corner point prediction grid (the second corner point prediction grid corresponding to the grid containing the first corner point) meets the re-matching condition, and if the found grid meets the re-matching condition, match the second corner point in the found grid with the first corner point in the grid containing the first corner point;
[0134] (5) for each paired group of first corner points and second corner points, determine the parking space information corresponding to the group according to the coordinate information of the first corner points, the coordinate information of the second corner points, the parking space occupancy information, and the parking space angle information of the group;
[0135] (6) Return all parking space information.
[0136] In yet another embodiment, the parking space recognition model outputs a plurality of features, and the parking space recognition model is trained according to the loss value corresponding to each feature.
[0137] The multiple features output by the parking space recognition model are parameters output by the parking space recognition model for representing the first confidence that each grid contains the first corner point, the second corner point prediction grid corresponding to each grid, the second confidence that each grid contains the second corner point, the parking space angle information corresponding to each grid, the first corner point coordinate information corresponding to each grid, the second corner point coordinate information corresponding to each grid, the parking space occupancy information corresponding to each grid, and other information. For example, the environment image is divided into 12*12 grids, and the parking space recognition model outputs information through 11 channels. Then, the loss value can be calculated for each of the 12*12*11 parameters output by the parking space recognition model to train the parking space recognition model.
[0138] In this embodiment, the parking space recognition model is trained according to the loss value corresponding to each feature, so that the trained parking space recognition model can more accurately recognize the parking spaces around the vehicle.
[0139] In another embodiment, the loss value corresponding to each feature is determined based on the absolute error between the output value of the parking space recognition model and the label value and the KL divergence between the output value of the parking space recognition model and the label value. The absolute error between the output value of the parking space recognition model and the label value refers to the difference between the output value of the parking space recognition model and the label value. In this embodiment, the loss value of each feature output by the parking space recognition model is determined based on the absolute error between the output value of the parking space recognition model and the label value and the KL divergence between the output value of the parking space recognition model and the label value. In this way, the parking space recognition model can be better trained according to the loss value of each feature, thereby improving the accuracy of the parking space recognition model in identifying parking spaces.
[0140] In yet another embodiment, the loss value corresponding to each feature is determined according to the following formula:
[0141]
[0142] Among them, loss is the loss value, output is the output value of the parking space recognition model, α is the stretching coefficient, and label is the label value. It should be noted that abs() represents the absolute value, and the technicians can preset the stretching coefficient and the balance coefficient based on experience, and k1 and k2 are preset minimum values.
[0143] As shown in formula (1), when determining the loss value corresponding to each feature, the loss value corresponding to each feature can be determined based on L1 loss and KL divergence. Since the output range of the model is between 0 and 1, the output value and label value of the parking space recognition model can be stretched in the L1 loss part so that the determined loss value can better train the parking space recognition model.
[0144] When determining the loss, the point-to-point loss can be determined for each feature output by the parking recognition model and the corresponding label value. In other words, the point-to-point loss can be determined based on the 12*12*11 features output by the parking recognition model and the corresponding 12*12*11 label values. When calculating the loss, the averaging method can be used instead. The dimension of the loss of the determined parking recognition model is 12*12*11. In the process of optimizing the parking recognition model, the value of the loss matrix is optimized separately, and each feature will not affect each other.
[0145] The minimum value may be a positive number with a small absolute value. The technician may preset the minimum value based on experience. For example, in one embodiment, the values of k1 and k2 may be preset to 0.0001. Since the output value of the parking space recognition model and the training data include parameters with a value of 0, in order to avoid the true number and the denominator being 0, the minimum values k1 and k2 may be introduced into formula (1).
[0146] In this embodiment, the loss value corresponding to each feature of the parking space recognition model is determined according to formula (1), so that the trained parking space recognition model can more accurately recognize the parking spaces around the vehicle.
[0147] Figure 7 FIG. 1 is a block diagram of a parking space recognition device according to an exemplary embodiment of the present disclosure. Figure 7 As shown, the parking space recognition device 600 includes an acquisition module 601 , an obtaining module 602 and a determination module 603 .
[0148] The acquisition module 601 is configured to acquire an environment image around the vehicle.
[0149] The acquisition module 602 is configured to input the acquired environment image into a pre-trained parking space recognition model to obtain the output of the parking space recognition model.
[0150] The determination module 603 is configured to determine the parking space information around the vehicle according to the output of the parking space recognition model.
[0151] Among them, the parking space recognition model includes a multi-scale module. The input and output of the multi-scale module pass through a separation sub-module, a multi-scale operation sub-module and a merging sub-module. The separation sub-module is configured to divide the input of the separation sub-module into N slice layers evenly according to the number of channels, and the output of the separation sub-module is N slice layers.
[0152] The multi-scale operation submodule is configured to output N layers of multi-scale operation results according to the N slice layers, wherein the first slice layer is convolved with the convolution kernel to obtain the first layer of multi-scale operation results; starting from the second slice layer, each slice layer is convolved with the convolution kernel to obtain the corresponding multi-scale operation result after the addition result of the multi-scale operation result of the previous layer is added to the convolution kernel.
[0153] The merging submodule is configured to merge the N-layer multi-scale operation results and output the merged result.
[0154] In yet another embodiment, the multi-scale operation submodule further includes an adaptive submodule.
[0155] The adaptive submodule is configured to output the multi-scale operation results of N layers after adaptive operation according to the multi-scale operation results of N layers, wherein the multi-scale operation results of each layer input are convolved with the convolution kernel respectively to obtain the adaptive coefficients corresponding to the multi-scale operation results of each layer; the product of the multi-scale operation results of each layer and the corresponding adaptive coefficient is the multi-scale operation result of each layer after adaptive operation.
[0156] The merging submodule is further configured to merge the multi-scale operation results of the N layers after the adaptive operation, and output the merged result.
[0157] In another embodiment, the environmental image is divided into a predetermined number of grids; the output of the parking space recognition model includes a first confidence that each grid contains a first corner point, a predicted grid of a second corner point corresponding to each grid, a second confidence that each grid contains a second corner point, parking space angle information corresponding to each grid, first corner point coordinate information corresponding to each grid, and second corner point coordinate information corresponding to each grid.
[0158] In yet another embodiment, the determination module 603 includes a first determination submodule and a second determination submodule.
[0159] The first determination submodule is configured to determine, for each grid, whether the grid includes a first corner point according to a first confidence level of the grid.
[0160] The second determination submodule is configured to determine the parking space information around the vehicle according to the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, and the second corner point coordinate information corresponding to the grid if the second confidence of the second corner point prediction grid corresponding to the grid is greater than a predetermined confidence threshold for each grid containing the first corner point.
[0161] In yet another embodiment, the determination module 603 further includes a searching submodule and a third determining submodule.
[0162] The search submodule is configured to search for the second confidence of a grid within a preset distance from the second corner point prediction grid corresponding to the grid for each grid containing the first corner point if the second confidence of the second corner point prediction grid corresponding to the grid is less than the confidence threshold.
[0163] The third determination submodule is configured to determine the parking space information around the vehicle based on the second corner point coordinate information corresponding to the found grid, the parking space angle information, and the first corner point coordinate information to be paired with the found grid if the found grid satisfies the re-pairing condition, wherein the re-pairing condition includes that the second confidence of the found grid is greater than the confidence threshold, and the found grid is not the second corner point predicted grid corresponding to the remaining grids containing the first corner point.
[0164] In another embodiment, the output of the parking space recognition model also includes parking space occupancy information corresponding to each grid; the second determination submodule is further configured to determine the parking space information around the vehicle based on the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, the second corner point coordinate information corresponding to the grid, and the parking space occupancy information corresponding to the grid.
[0165] In yet another embodiment, the parking space recognition model outputs a plurality of features, and the parking space recognition model is trained according to the loss value corresponding to each feature.
[0166] In another embodiment, the loss value corresponding to each feature is determined based on the absolute error between the output value of the parking space recognition model and the label value and the KL divergence between the output value of the parking space recognition model and the label value.
[0167] In yet another embodiment, the loss value corresponding to each feature is determined according to the following formula:
[0168]
[0169] Among them, loss is the loss value, output is the output value of the parking space recognition model, α is the stretching coefficient, label is the label value, is the balance coefficient, k1 and k2 are preset minimum values.
[0170] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0171] Through the above technical solution, the parking space recognition model includes a multi-scale module, which includes a separation sub-module, a multi-scale operation sub-module and a merging sub-module. The separation sub-module is used to divide its input into N slice layers and then input them into the multi-scale operation sub-module for calculation. In this way, the intermediate parameters generated by the multi-scale module are greatly reduced, the computer computing power is saved, and the speed of parking space recognition is improved without reducing the recognition accuracy of the parking space recognition model, which can bring a better user experience to users.
[0172] The present disclosure also provides a non-volatile computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above-mentioned parking space identification method are implemented.
[0173] The present disclosure also provides an electronic device, comprising:
[0174] a memory having a computer program stored thereon;
[0175] A processor is used to execute the computer program in the memory to implement the steps of the above-mentioned parking space recognition method.
[0176] Figure 8 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 8 As shown, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.
[0177] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned parking space identification method. The memory 702 is used to store various types of data to support the operation of the electronic device 700, and these data may include, for example, instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module, etc.
[0178] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned parking space recognition method.
[0179] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned parking space recognition method are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions may be executed by the processor 701 of the electronic device 700 to complete the above-mentioned parking space recognition method.
[0180] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.
[0181] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0182] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.
Claims
1. A parking space recognition method, characterized in that: The method comprises: Acquire an environmental image around the vehicle; Inputting the acquired environment image into a pre-trained parking space recognition model to obtain an output of the parking space recognition model; Determine the parking space information around the vehicle according to the output of the parking space recognition model; The parking space recognition model includes a multi-scale module, the input and output of the multi-scale module pass through a separation submodule, a multi-scale operation submodule and a merging submodule, the separation submodule is configured to evenly divide the input of the separation submodule into N slice layers according to the number of channels, and the output of the separation submodule is the N slice layers; The multi-scale operation submodule is configured to output N layers of multi-scale operation results according to the N slice layers, wherein the first slice layer is convolved with the convolution kernel to obtain the first layer of multi-scale operation results; starting from the second slice layer, each slice layer is convolved with the convolution kernel to obtain the corresponding multi-scale operation result after the addition result of the multi-scale operation result of the previous layer is added to each slice layer; The merging submodule is configured to merge N layers of multi-scale operation results and output the merged result.
2. The method according to claim 1, characterized in that The multi-scale operator module also includes: The adaptive submodule is configured to output the multi-scale operation results of the N layers after adaptive operation according to the multi-scale operation results of the N layers, wherein the input multi-scale operation results of each layer are convolved with the convolution kernel respectively to obtain the adaptive coefficients corresponding to the multi-scale operation results of each layer; the product of the multi-scale operation results of each layer and the corresponding adaptive coefficient is the multi-scale operation result of each layer after adaptive operation; The merging submodule is configured to merge the multi-scale operation results of the N layers after the adaptive operation, and output the merged result.
3. The method according to claim 1, characterized in that The environmental image is divided into a predetermined number of grids; the output of the parking space recognition model includes a first confidence that each grid contains a first corner point, a second corner point prediction grid corresponding to each grid, a second confidence that each grid contains a second corner point, parking space angle information corresponding to each grid, first corner point coordinate information corresponding to each grid, and second corner point coordinate information corresponding to each grid.
4. The method according to claim 3, characterized in that The determining the parking space information around the vehicle according to the output of the parking space recognition model includes: For each grid, determining whether the grid includes a first corner point according to a first confidence level of the grid; For each grid containing a first corner point, if the second confidence of the second corner point prediction grid corresponding to the grid is greater than a predetermined confidence threshold, the parking space information around the vehicle is determined based on the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, and the second corner point coordinate information corresponding to the second corner point prediction grid corresponding to the grid.
5. The method according to claim 4, characterized in that The step of determining the parking space information around the vehicle according to the output of the parking space recognition model further includes: For each grid including the first corner point, if the second confidence of the second corner point prediction grid corresponding to the grid is less than the confidence threshold, searching for the second confidence of the grid within a preset distance from the second corner point prediction grid corresponding to the grid; If the found grid meets the re-matching condition, the parking space information around the vehicle is determined according to the second corner point coordinate information corresponding to the found grid, the parking space angle information, and the first corner point coordinate information to be paired with the found grid, wherein the re-matching condition includes that the second confidence of the found grid is greater than the confidence threshold, and the found grid is not the second corner point predicted grid corresponding to the remaining grids containing the first corner point.
6. The method according to claim 4, characterized in that The output of the parking space recognition model also includes parking space occupancy information corresponding to each grid; The determining of the parking space information around the vehicle according to the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, and the second corner point coordinate information corresponding to the grid predicts the second corner point coordinate information corresponding to the grid, comprises: The parking space information around the vehicle is determined according to the parking space angle information corresponding to the grid, the first corner point coordinate information corresponding to the grid, the second corner point coordinate information corresponding to the grid, and the parking space occupancy information corresponding to the grid.
7. The method according to claim 1, characterized in that The parking space recognition model outputs multiple features, and the parking space recognition model is trained according to the loss value corresponding to each feature.
8. The method according to claim 7, characterized in that The loss value corresponding to each feature is determined according to the absolute error between the output value of the parking space recognition model and the label value and the KL divergence between the output value of the parking space recognition model and the label value.
9. The method according to claim 8, characterized in that The loss value corresponding to each feature is determined according to the following formula: Among them, loss is the loss value, output is the output value of the parking space recognition model, α is the stretching coefficient, label is the label value, is the balance coefficient, k1 and k2 are preset minimum values.
10. A parking space recognition device, characterized in that: include: An acquisition module is configured to acquire an environmental image around the vehicle; An acquisition module is configured to input the acquired environment image into a pre-trained parking space recognition model to obtain an output of the parking space recognition model; a determination module, configured to determine parking space information around the vehicle according to an output of the parking space recognition model; The parking space recognition model includes a multi-scale module, the input and output of the multi-scale module pass through a separation submodule, a multi-scale operation submodule and a merging submodule, the separation submodule is configured to evenly divide the input of the separation submodule into N slice layers according to the number of channels, and the output of the separation submodule is the N slice layers; The multi-scale operation submodule is configured to output N layers of multi-scale operation results according to the N slice layers, wherein the first slice layer is convolved with the convolution kernel to obtain the first layer of multi-scale operation results; starting from the second slice layer, each slice layer is convolved with the convolution kernel to obtain the corresponding multi-scale operation result after the addition result of the multi-scale operation result of the previous layer is added to each slice layer; The merging submodule is configured to merge N layers of multi-scale operation results and output the merged result.
11. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented.
12. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Parking space detection method and device and computer storage medium
CN114842446A
Vehicle target identification method based on multi-scale yolk algorithm
CN115171074A
Parking space state identification method based on improved YOLO model
CN115953743A
Parking spot detection method and parking spot detection system
US20220245952A1
