Parking space recognition method and device

Abstract

The present disclosure provides a parking space recognition method and device, and relates to the technical field of intelligent transportation. The method comprises the following steps: acquiring a surround view image of a target vehicle, and determining a target region in the surround view image; dividing the target region in a target direction to obtain a plurality of grids, wherein the target direction is perpendicular to the driving direction of the target vehicle; and determining a parking space according to image semantic information corresponding to each of the plurality of grids. The parking space recognition efficiency is high.

Description

[0001] This application is a divisional application of Chinese application No. 202211560566.0, the foregoing contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of intelligent transportation technology, and in particular to a method and apparatus for identifying parking spaces, a computer-readable storage medium, and an electronic device. Background Technology

[0003] Traditional parking space recognition solutions mainly rely on ultrasonic radar sensors for detection, which depends on vehicles in adjacent parking spaces, and the recognition efficiency needs to be improved.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a parking space identification method, a parking space identification device, a computer-readable storage medium, and an electronic device, which can improve the accuracy and efficiency of parking space identification to a certain extent.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, a method for identifying parking spaces is provided. The method includes: acquiring a surround view image of a target vehicle and determining a target region in the surround view image; dividing the target region into multiple grids in a target direction, wherein the target direction is perpendicular to the driving direction of the target vehicle; and determining parking spaces based on image semantic information corresponding to the multiple grids respectively.

[0008] In an exemplary embodiment, based on the above scheme, after obtaining the surround view image of the target vehicle, the method further includes: performing semantic segmentation processing on the surround view image to locate the target object in the surround view image, wherein the type of the target object is at least one.

[0009] In an exemplary embodiment, based on the above scheme, the target region is located on both sides of the target vehicle body; the above-mentioned segmentation of the target region in the target direction to obtain multiple grids includes: segmenting the target region in the target direction to obtain multiple grids and image semantic information corresponding to each grid; wherein, the image semantic information corresponding to each grid includes: the type of target object contained in the image region corresponding to the grid, and the number of pixels of each type of target object; the image semantic information corresponding to the auxiliary grid includes: the type of target object contained in the image region corresponding to the auxiliary grid, and the number of pixels of each type of target object.

[0010] In an exemplary embodiment, based on the above scheme, determining a parking space according to the image semantic information corresponding to the plurality of grids includes: encoding the plurality of grids according to the image semantic information corresponding to the plurality of grids respectively; wherein, if the image region corresponding to the grid is determined to contain at least one type of target object according to the image semantic information, and the number of pixels of at least one type of target object is not less than the corresponding pixel threshold, then the grid is encoded as a first identifier code; otherwise, the grid is encoded as a second identifier code; within the preset distance range, if the number of grids encoded as the second identifier code meets a preset condition, then the image region corresponding to the preset distance range is determined to correspond to a parking space; wherein the preset distance range is related to the actual width of the parking space.

[0011] In an exemplary embodiment, based on the above scheme, after determining that the image area corresponding to the preset distance range corresponds to a parking space, the method further includes: for a preset number of target grids located at the edge position in all grids within the preset distance range, determining whether the target grid corresponds to a simulated parking space line based on the image semantic information corresponding to the target grid and the image semantic information corresponding to the auxiliary grid; wherein, the auxiliary grid is an adjacent grid of the target grid.

[0012] In an exemplary embodiment, based on the above scheme, determining whether the target grid corresponds to a simulated parking space line according to the image semantic information corresponding to the target grid and the image semantic information corresponding to the auxiliary grid includes: determining whether the target grid satisfies being encoded as the first identifier code; determining whether there is a grid in the auxiliary grid that is encoded as the second identifier code; and determining that the target grid corresponds to a simulated parking space line when the target grid is encoded as the first identifier code and there is a grid in the auxiliary grid that is encoded as the second identifier code.

[0013] In an exemplary embodiment, based on the above scheme, after determining that the image region corresponding to the preset distance range corresponds to a parking space, the method further includes: determining whether the image region corresponding to the grid contains pixels of the parking line type, and the number of pixels is not less than the pixel threshold of the parking line type, according to the image semantic information of the grid; if the grid contains pixels of the parking line type, and the number of pixels is not less than the pixel threshold of the parking line type, determining that the grid corresponds to a parking line.

[0014] In an exemplary embodiment, based on the above scheme, before determining the parking space according to the image semantic information corresponding to the multiple grids respectively, the method further includes: if it is determined that there is a distorted image region in the surround view image according to the image semantic information of the surround view image, then at least one auxiliary image is acquired again; a local image region corresponding to the distorted image region is determined from the at least one auxiliary image; and the distortion is determined according to the image semantic information of the local image region.

[0015] In an exemplary embodiment, based on the above scheme, determining the distortion variable according to the image semantic information of the local image region includes: segmenting the local image region in the target direction to obtain N grids, where N is an integer greater than 1; encoding the i-th grid according to the image semantic information of the i-th grid; wherein, if the image region corresponding to the i-th grid contains pixels of the vehicle type and the number of pixels is not less than the pixel threshold corresponding to the vehicle type, the i-th grid is encoded as a first identifier code; otherwise, the i-th grid is encoded as a second identifier code, where i is a positive integer not greater than N; and determining the distortion variable based on the number of grids identified as the first identifier code.

[0016] According to another aspect of this disclosure, a parking space identification device is provided, the device comprising: an acquisition module, a segmentation module, and a first determination module.

[0017] The acquisition module is used to acquire a surround view image of the target vehicle and determine the target area in the surround view image; the segmentation module is used to segment the target area in the target direction to obtain multiple grids, wherein the target direction is perpendicular to the driving direction of the target vehicle; the first determination module is used to determine the parking space based on the image semantic information corresponding to the multiple grids respectively.

[0018] According to another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the parking space identification method as described in the above embodiments.

[0019] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the parking space identification method as described in the above embodiments.

[0020] The parking space identification method, parking space identification device, computer-readable storage medium, and electronic device provided in the embodiments of this disclosure have the following technical effects:

[0021] In the technical solution provided in this application, a target region is determined in the surround view image of the target vehicle, and the target region is divided into multiple grids in a target direction perpendicular to the driving direction of the target vehicle. Furthermore, based on the image semantic information corresponding to each of the multiple grids, a parking space can be determined. The parking space recognition scheme provided in this specification can quickly and accurately determine parking spaces with high recognition efficiency.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0024] Figure 1 This diagram illustrates a flowchart of a parking space identification method in an exemplary embodiment of the present disclosure.

[0025] Figure 2 This diagram illustrates a target vehicle and a target area in an exemplary embodiment of this disclosure.

[0026] Figure 3 This diagram illustrates a scenario in which a target region is divided into a grid.

[0027] Figure 4 This diagram illustrates a flowchart of a parking space identification method in another exemplary embodiment of the present disclosure.

[0028] Figure 5 This diagram illustrates a target object contained in an image region corresponding to a grid in one embodiment of the present disclosure.

[0029] Figure 6a This diagram illustrates the image semantic information corresponding to a grid in one embodiment of the present disclosure.

[0030] Figure 6b Showing the Figure 6a A schematic diagram of encoding using a grid.

[0031] Figure 7 This diagram illustrates a set of grids satisfying a preset distance range according to an embodiment of the present disclosure.

[0032] Figure 8 This diagram illustrates a flowchart of a method for locating parking space lines in an exemplary embodiment of this disclosure.

[0033] Figure 9 This diagram illustrates a target vehicle and a parked vehicle in an exemplary embodiment of this disclosure.

[0034] Figure 10 This diagram illustrates a flowchart of a method for determining abnormal variables in an exemplary embodiment of this disclosure.

[0035] Figure 11 A schematic diagram of the structure of a parking space identification device according to an embodiment of the present disclosure is shown.

[0036] Figure 12 A schematic diagram of the structure of a parking space identification device according to another embodiment of the present disclosure is shown.

[0037] Figure 13 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0039] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0040] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0041] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0042] The following combination Figures 1 to 10 The embodiments of the parking space identification method provided in this disclosure are described in detail below:

[0043] in, Figure 1 This diagram illustrates a flowchart of a parking space identification method according to an exemplary embodiment of this disclosure. (See reference...) Figure 1 The embodiment shown in the figure includes: S110-S130.

[0044] In S110, a surround view image of the target vehicle is acquired, and the target area is determined within the surround view image.

[0045] In an exemplary embodiment, the target vehicle is any vehicle for which a parking space needs to be determined. The surround view image of the target vehicle can be determined by images captured by fisheye cameras installed around the target vehicle. In an exemplary embodiment, the surround view image of the target vehicle can be obtained by performing distortion correction and perspective transformation on the images captured by the cameras, followed by stitching. Specifically, the surround view image of the target vehicle includes a 360-degree panoramic image of the environment in which the target vehicle is located. In the solution provided by the embodiments of this specification, determining the parking space information in the current environment based on the surround view image acquired in real time by the target vehicle is beneficial to the timeliness of parking space identification.

[0046] In an exemplary embodiment, the target region in the global image of the target vehicle is further processed for parking space identification. That is, regions in the global image other than the target region are not included in subsequent calculations, thereby reducing computational load and improving the timeliness of parking space identification.

[0047] In S120, the target area is divided into multiple grids along the target direction, wherein the target direction is perpendicular to the driving direction of the target vehicle.

[0048] It is easy to understand that in the embodiments of this specification, the direction of the vehicle's front is referred to as the front side of the vehicle body, the direction of the vehicle's rear is referred to as the rear side of the vehicle body, and the sides of the vehicle doors are referred to as the left and right sides of the vehicle body.

[0049] In an exemplary embodiment, given that the applicable scenarios for parking space identification schemes are generally located in parking lots, where parking spaces are typically situated on the left and / or right side of the vehicle, therefore, referring to... Figure 2 In this embodiment of the specification, the image regions located on the left and / or right side of the target vehicle 200 (regions 210 and / or 220) are determined as the aforementioned target regions in the surround view image of the target vehicle 200. By identifying high-value local regions in the surround view image, this embodiment of the specification improves recognition efficiency. This embodiment of the specification uses regions 210 and 220 as target regions as an example for explanation.

[0050] In an exemplary embodiment, Figure 3 This diagram illustrates a method for dividing a target region into a grid in an exemplary embodiment of this disclosure, with reference to... Figure 3 In the direction F2 perpendicular to the driving direction F1 of the target vehicle 200, the target area (area 210 and area 220) is divided into multiple grids. For example, the width of the divided grid is adapted to the width of the parking line; the specific grid width setting should be related to factors such as the resolution of the panoramic image. Specifically, the width of each divided grid corresponds to the width of the actual parking line, for example, between 20 and 25 cm. It is understood that this embodiment of the specification does not limit this, and the width of each divided grid can be set according to actual needs.

[0051] Continue to participate Figure 1 In S130, parking spaces are determined based on the image semantic information corresponding to the multiple grids mentioned above.

[0052] In an exemplary embodiment, Figure 4 This diagram illustrates a flowchart of a parking space identification method in another exemplary embodiment of the present disclosure.

[0053] refer to Figure 4 In S410, a surround view image of the target vehicle is acquired. In S420, the target area is determined from the surround view image. Since the specific implementations of S410 and S420 are consistent with the corresponding embodiment of S110, they will not be described again here.

[0054] In S430, semantic segmentation processing is performed on the panoramic image to locate the target object within the panoramic image. The specific implementation order of S430 and S420 is not important; S420 and S430 can be executed simultaneously, S420 can be executed first and then S430, or S430 can be executed first and then S420.

[0055] In an exemplary embodiment, the semantic segmentation process described above can be implemented using a semantic segmentation algorithm. This process associates labels or categories with each pixel in the surround-view image. Furthermore, pixels belonging to the same category or label are grouped into the same class, enabling the localization of target objects of different categories within the surround-view image. For example, the semantic segmentation process can locate pixel sets corresponding to vehicles, pedestrians, traffic signal signs, and crosswalks in the surround-view image.

[0056] In an exemplary embodiment, the target object includes at least one category, such as parking lines and obstacles (including pedestrians and other vehicles). For example, if a surround view image A contains both obstacles and parking lines, semantic segmentation processing of the surround view image can locate the pixel sets corresponding to the parking lines and the pixel sets corresponding to the obstacles in surround view image A. In this embodiment, the image can be divided into multiple channels based on the number of target object types contained in surround view image A. For example, if surround view image A contains parking lines and obstacles, it can be divided into channel 1 for the pixel set corresponding to the parking lines and channel 2 for the pixel set corresponding to the obstacles. Furthermore, to improve image processing speed, each channel can be binarized, making it easier to count the number of pixels of each type of target object falling into each grid.

[0057] In an exemplary embodiment, after determining the target area (e.g., S420) Figure 2 After locating target objects of various categories in the panoramic image (regions 210 and 220) in S430, the target region is divided in the target direction in S440 to obtain multiple grids and the image semantic information corresponding to each grid.

[0058] The semantic information of an image corresponding to a grid includes: the type of the target object contained in the image region corresponding to that grid (e.g., reference). Figure 5The image regions corresponding to grids a-d contain target objects of type pedestrians, and the number of pixels for each type of target object. Specifically, the above-mentioned panoramic image A contains two types of target objects and is split into two channels. After the target region in panoramic image A is segmented, for any grid a, the image semantic information corresponding to grid a is related to the above two channels. On the one hand, whether parking lines are distributed at grid a in channel 1 and the number of pixels distributed therein; on the other hand, whether obstacles are distributed at grid a in channel 2 and the number of pixels distributed therein.

[0059] Continue to refer to Figure 4 In S450, the grid is encoded according to the image semantic information; wherein, if the image region corresponding to the grid is determined to contain at least one type of target object according to the image semantic information, and the number of pixels of at least one type of target object is not less than the corresponding pixel threshold, then the grid is encoded as a first identifier code, otherwise the grid is encoded as a second identifier code.

[0060] In an exemplary embodiment, each grid is encoded based on the image semantic information corresponding to it. Specifically, if the image semantic information corresponding to a grid satisfies a first preset condition, the grid is encoded as a first identifier, such as "1"; if the image semantic information corresponding to a grid does not satisfy the first preset condition, the grid is encoded as a second identifier, such as "0". The first preset condition is that the image region corresponding to a grid contains at least one type of target object, and the number of pixels of at least one type of target object is not less than the corresponding pixel threshold.

[0061] The pixel threshold for each type of target object can be determined based on actual needs; therefore, the pixel threshold may differ for different types of target objects. For example, refer to... Figure 6a The pixel threshold for circle-shaped target objects is 2, for elliptical target objects it is 1, and for square target objects it is 3. Further, based on the above pixel thresholds... Figure 6a Each grid cell is coded based on whether it meets a first preset condition. Specifically:

[0062] For grid e, the corresponding image semantic information is as follows: the corresponding image region contains two types of target objects (circles and ellipses), and the actual number of pixels for circle-type target objects is 1, while the actual number of pixels for elliptical-type target objects is 2. Specifically, the actual number of pixels (1) for circle-type target objects is less than the corresponding pixel threshold (2), and the actual number of pixels (2) for elliptical-type target objects is greater than the corresponding pixel threshold (1). Therefore, since there exists a type of target object with a pixel count not less than its corresponding pixel threshold, grid e satisfies the first preset condition and is encoded as "1".

[0063] For grid f, the corresponding image semantic information is as follows: the corresponding image region contains two types of target objects (circles and ellipses), and the actual number of pixels for circle-type target objects is 3, while the actual number of pixels for elliptical-type target objects is 1. Specifically, the actual number of pixels (3) for circle-type target objects is not less than the corresponding pixel threshold (2), and the actual number of pixels (1) for elliptical-type target objects is not less than the corresponding pixel threshold (1). It can be seen that there are two types of target objects whose pixel count is not less than their corresponding pixel thresholds. Therefore, grid f satisfies the above first preset condition, and thus the grid is encoded as "1".

[0064] For grid g, the corresponding image semantic information is as follows: the corresponding image region contains two types of target objects (square circle type and elliptical type), and the actual number of pixels for square type target objects is 2, while the actual number of pixels for elliptical type target objects is 1. Specifically, the actual number of pixels (2) for square type target objects is less than the corresponding pixel threshold (3), and the actual number of pixels (1) for elliptical type target objects is not less than the corresponding pixel threshold (1). It can be seen that there exists a type of target object whose pixel count is not less than the corresponding pixel threshold. Therefore, grid g satisfies the above first preset condition, and is thus encoded as "1".

[0065] Similarly, it can be determined that grid h also satisfies the first preset condition mentioned above; since the image regions corresponding to grids x, y and z do not contain any type of target object, grids x, y and z do not satisfy the first preset condition mentioned above, so grids x, y and z are all encoded as "0".

[0066] according to Figure 6a Whether each grid in the middle meets the first preset condition will be implemented as follows: Figure 6a The encoding of the grid shown is as follows: Figure 6bAs shown. For example, a grid coded as "1" can be represented by a diagonal shading.

[0067] By determining whether the semantic information of the image corresponding to the grid satisfies the first preset condition, the encoding of all grids can be achieved. Further, S460 is executed: within a preset distance range, if the number of grids encoded as the second identifier code satisfies the preset condition, then the image region corresponding to that preset distance range is determined to correspond to a parking space.

[0068] The preset distance range is related to the actual parking space width. For example, if the actual parking space width is 2.4 meters, the preset distance range can be determined based on the ratio between the captured image and the actual environment; for instance, it could be [28, 32] millimeters. For example, to ensure recognition accuracy, a certain margin can be set, meaning the minimum value of the preset distance range is greater than the mapping value of the actual parking space width in the image.

[0069] In this embodiment, the aforementioned preset condition (which can be referred to as the second preset condition) may be that the percentage of the number of grids encoded as the second identifier within the preset distance range is greater than a preset value (e.g., greater than 95%), or the aforementioned second preset condition may be that the number of grids encoded as the second identifier is greater than a preset value (e.g., 50). For example, Figure 7 This diagram illustrates a set of grids satisfying a preset distance range according to an embodiment of the present disclosure. (Refer to...) Figure 7 If “S” satisfies the preset distance range, and the number of grids encoded as “0” within that range also satisfies the preset condition, then it means that there are a large number of grids encoded as the second identifier “0” within the preset distance range. This also means that there is no target object in a large area of ​​that region, so it can be used as a parking location. Thus, it can be determined that the image area corresponding to the preset distance range corresponds to a parking space.

[0070] In an exemplary embodiment, to further improve the accuracy of parking space positioning, this specification also provides a scheme for locating the "proposed parking space line" and the "parking space line." By locating the proposed parking space line and the parking space line, more accurate parking prompts can be provided to drivers, helping to improve parking safety. Specifically, Figure 8 The flowchart illustrates a method for locating parking space lines in an exemplary embodiment of this disclosure, including steps S810-S830.

[0071] In S810, a predetermined number of target grids located at the edge positions are determined among all grids within the aforementioned preset distance range. For example, this could be 3 to 5. (Refer to...) Figure 7The three target grids located at the edge of the above "S" range are: grids 70-72 on the left and grids 73-75 on the right.

[0072] In this embodiment, the target grid is determined to belong to a parking space line based on the code of the target grid and the codes of the grids adjacent to the target grid (denoted as auxiliary grids). For example, grid 70 is the first target grid (i.e., j is 1), and the auxiliary grids of grid 70 are grid A and grid 71.

[0073] Specifically, it can be determined whether the conditions shown in S820 are met: the j-th target grid satisfies the first preset condition and whether there is a grid encoded with the second identifier in the auxiliary grid of the j-th target grid. If the conditions are met, it can be determined that the j-th target grid corresponds to the simulated parking line (S830), and the judgment condition as shown in S820 is executed for the (j+1)-th target grid. If the conditions are not met, the judgment condition as shown in S820 is executed directly for the (j+1)-th target grid.

[0074] In an exemplary embodiment, reference is made to Figure 7 For grid 70 (j is 1), its corresponding auxiliary grids are grid 71 and grid A. Grid 70 is coded as "0" and satisfies the first preset condition mentioned above. Therefore, grid 70 does not correspond to the simulated parking space line or the parking space line, and the user does not need to avoid the position corresponding to this grid during the parking process.

[0075] In an exemplary embodiment, reference is made to Figure 7 For grid 71 (j is 2), its corresponding auxiliary grids are grid 72 and grid B. Grid 71 is encoded as "1", satisfying the first preset condition mentioned above. The auxiliary grids 72 and B are both encoded as "0", meaning that there is a grid in the auxiliary grids of grid 71 that is encoded with the second identifier. Therefore, grid 71 corresponds to a simulated parking line. Since grid 71 is encoded as "1", it indicates that the current grid may correspond to an obstacle or a parking line, etc. Because it cannot be accurately determined whether it is a parking line, it is recorded as "simulated parking line 2". Therefore, during the user's parking process, the user needs to be prompted to avoid the area corresponding to grid 71.

[0076] In an exemplary embodiment, this specification also provides a scheme for locating "parking lines," specifically: based on the image semantic information of the grid, it is determined whether the image region corresponding to the grid contains pixels of the parking line type, and the number of pixels is not less than the pixel threshold for the parking line type. If this condition is met, it is indicated that the grid is a parking line.

[0077] By using the above-mentioned parking space line positioning scheme and the proposed parking space line positioning scheme, the obstacles that users need to avoid during the parking process can be determined, thereby improving parking safety.

[0078] In some cases, the parked vehicle (such as vehicle B) shown in the above surround view image may be distorted, creating the illusion that vehicle B occupies two parking spaces. For example, Figure 9 This diagram illustrates a target vehicle and a parked vehicle in an exemplary embodiment of this disclosure, with reference to... Figure 9 It can be determined that regions 910, 920, and 930 all satisfy the aforementioned preset distance range. Suppose that vehicle B is parked in the real-world environment corresponding to region 920 in the image. However, in the surround view image obtained for the vehicle to be parked, vehicle B may be distorted, creating the illusion that vehicle B occupies two parking spaces. That is, in addition to the pixels with vehicle type in region 920, there are also pixels with vehicle type in region 910, but in reality, no vehicle is parked in region 910. To address this situation, this specification provides an embodiment for determining the distortion variable. Determining the parking space after determining the distortion variable helps to further improve parking safety.

[0079] For example, Figure 10 This diagram illustrates a flowchart of a method for determining distorted variables in an exemplary embodiment of this disclosure. (Refer to...) Figure 10 The embodiment shown in the figure includes: S1010-S1050.

[0080] In S1010, if it is determined that there is a distorted image region in the panoramic image based on the image semantic information, at least one auxiliary image is acquired again.

[0081] For example, the aforementioned auxiliary image may be a 360-degree surround view image of the vehicle waiting to be parked, or it may be an image with distortion issues.

[0082] In S1020, a local image region corresponding to the distorted image region is determined based on the at least one auxiliary image frame.

[0083] For example, refer to Figure 9 The local image region identified in the auxiliary image is region 910.

[0084] In S1030, the local image region is segmented along the target direction to obtain N grids. N is an integer greater than 1. The segmentation method described above is the same as that in S120, and will not be repeated here. It should be noted that in the segmentation process provided in this embodiment, the width of the segmented grids can be related to the width of the parking line, or other values ​​can be set.

[0085] In S1040, the i-th grid is encoded based on its image semantic information. Here, i is a positive integer not greater than N.

[0086] It is understandable that the i-th semantic information includes: whether the image region corresponding to the i-th grid contains a target object, and if it contains a target object, the number of pixels corresponding to each type of target object.

[0087] The exemplary encoding method is consistent with the encoding method in the above embodiment. That is, if the image region corresponding to the i-th grid contains pixels of the vehicle type based on the image semantic information of the i-th grid, and the number of pixels is not less than the pixel threshold corresponding to the vehicle type, then the grid is encoded as the first identifier code; otherwise, the grid is encoded as the second identifier code.

[0088] In S1050, the distorted variable is determined based on the number of grids identified as the first identifier code.

[0089] If the number of grids encoded as "1" in the aforementioned N grids is less than the grid threshold, then it is determined whether the area between the first target grid and the second target grid corresponds to a parking space. The grid threshold can be determined according to actual needs; for example, when N is even, it can be N / 2, and when N is odd, it can be (N+1) / 2.

[0090] Understandably, reference Figure 9 If vehicle B is parked in the actual environment corresponding to region 920, then the number of grids in region 920 encoded as "1" is definitely greater than the aforementioned grid threshold. However, if no vehicle is actually parked in the actual environment corresponding to region 910, then the number of grids encoded as "1" in region 910 should not exceed the aforementioned grid threshold.

[0091] Furthermore, the number of grids coded as "1" can be counted to identify the distorted variables, thereby assisting in the determination of parking spaces and improving the accuracy of parking prompts and parking space recognition.

[0092] The solution provided in this manual can quickly and accurately detect parking spaces by post-processing the semantic segmentation results of the surround view image, effectively improving the recognition efficiency of parking spaces and helping car owners to park quickly and accurately.

[0093] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0094] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0095] in, Figure 11 A schematic diagram of a parking space identification device according to an embodiment of this disclosure is shown. Please refer to... Figure 11 The parking space recognition device shown in the figure can be implemented as a whole or part of an electronic device through software, hardware, or a combination of both, or it can be integrated as an independent module into an electronic device or server.

[0096] The parking space identification device 1100 in this embodiment includes: an acquisition module 1110, a segmentation module 1120, and a first determination module 1130.

[0097] The acquisition module 1110 is used to acquire a surround view image of the target vehicle and determine the target area in the surround view image; the segmentation module 1120 is used to segment the target area in the target direction to obtain multiple grids, wherein the target direction is perpendicular to the driving direction of the target vehicle; and the first determination module 1130 is used to determine the parking space based on the image semantic information corresponding to the multiple grids respectively.

[0098] In an exemplary embodiment, Figure 12 A schematic diagram of a parking space identification device according to another embodiment of this disclosure is shown. Please refer to... Figure 12 :

[0099] In an exemplary embodiment, based on the foregoing scheme, the acquisition module 1110 is further configured to: after acquiring the surround view image of the target vehicle, perform semantic segmentation processing on the surround view image to locate the target object in the surround view image, wherein the type of the target object is at least one.

[0100] In an exemplary embodiment, based on the foregoing scheme, the target area is located on both sides of the vehicle body of the target vehicle;

[0101] The segmentation module 1120 is specifically used to: segment the target region in the target direction to obtain multiple grids and image semantic information corresponding to each grid; wherein, the image semantic information corresponding to each grid includes: the type of target object contained in the image region corresponding to the target grid, and the number of pixels of each type of target object; the image semantic information corresponding to the auxiliary grid includes: the type of target object contained in the image region corresponding to the auxiliary grid, and the number of pixels of each type of target object.

[0102] In an exemplary embodiment, based on the foregoing scheme, the first determining module 1130 includes: an encoding unit 11301 and a determining unit 11302.

[0103] The encoding unit 11301 is used to: encode the multiple grids according to the image semantic information corresponding to each grid; wherein, if the image region corresponding to the grid contains at least one type of target object, and the number of pixels of at least one type of target object is not less than the corresponding pixel threshold, the grid is encoded as a first identifier code; otherwise, the grid is encoded as a second identifier code; the determining unit 11302 is used to: within the preset distance range, if the number of grids encoded as the second identifier code meets the preset condition, determine that the image region corresponding to the preset distance range corresponds to a parking space; wherein the preset distance range is related to the actual width of the parking space.

[0104] In an exemplary embodiment, based on the foregoing scheme, the device further includes a first positioning module 1140.

[0105] The first positioning module 1140 is used to: determine whether a target grid corresponds to a parking space line based on the image semantic information corresponding to the target grid and the image semantic information corresponding to the auxiliary grid, for a preset number of target grids located at the edge position in all grids within the preset distance range; wherein the auxiliary grid is an adjacent grid of the target grid.

[0106] In an exemplary embodiment, based on the foregoing scheme, the first positioning module 1140 is specifically used to: determine whether the target grid satisfies being encoded as the first identifier code; determine whether there is a grid in the auxiliary grid that is encoded as the second identifier code; and determine that the target grid corresponds to a simulated parking space line when the target grid is encoded as the first identifier code and there is a grid in the auxiliary grid that is encoded as the second identifier code.

[0107] In an exemplary embodiment, based on the foregoing scheme, the device further includes a second positioning module 1150.

[0108] The second positioning module 1150 is used to: determine whether the image region corresponding to the grid contains pixels of the parking line type based on the image semantic information of the grid, and the number of pixels is not less than the pixel threshold of the parking line type; and determine that the grid corresponds to a parking line when the grid contains pixels of the parking line type and the number of pixels is not less than the pixel threshold of the parking line type.

[0109] In an exemplary embodiment, based on the foregoing scheme, the above-mentioned device further includes a second determining module 1160.

[0110] The second determining module 1160 is configured to: acquire at least one auxiliary image again when a distorted image region is determined to exist in the panoramic image based on the image semantic information of the panoramic image; determine a local image region corresponding to the distorted image region in the at least one auxiliary image; and determine the distortion variable based on the image semantic information of the local image region.

[0111] In an exemplary embodiment, based on the aforementioned scheme, the second determining module 1160 is specifically used to: segment the local image region in the target direction to obtain N grids, where N is an integer greater than 1; encode the i-th grid according to the image semantic information of the i-th grid; wherein, if the image region corresponding to the i-th grid contains pixels of the vehicle type and the number of pixels is not less than the pixel threshold corresponding to the vehicle type, the i-th grid is encoded as a first identifier code; otherwise, the i-th grid is encoded as a second identifier code, where i is a positive integer not greater than N; and determine the distortion variable according to the number of grids identified as the first identifier code.

[0112] It should be noted that the parking space identification device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the parking space identification method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the parking space identification device and the parking space identification method embodiments provided in the above embodiments belong to the same concept. Therefore, for details not disclosed in the device embodiments of this disclosure, please refer to the above embodiments of the parking space identification method of this disclosure, which will not be repeated here.

[0113] The sequence numbers of the embodiments disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0114] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described in the foregoing embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0115] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0116] Figure 13 A schematic diagram of the structure of an electronic device according to an embodiment of this disclosure is shown. Please refer to... Figure 13 As shown, the electronic device 1300 includes a processor 1301 and a memory 1302.

[0117] In this embodiment, processor 1301 is the control center of the computer system, and can be a processor of a physical machine or a processor of a virtual machine. Processor 1301 may include one or more processing cores, such as a 4-core processor or an 8-core processor. Processor 1301 can be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1301 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state.

[0118] In this embodiment of the disclosure, the processor 1301 is specifically used for:

[0119] Obtain a surround view image of the target vehicle and determine the target area within the surround view image; divide the target area into multiple grids along the target direction, wherein the target direction is perpendicular to the driving direction of the target vehicle; determine the parking space based on the image semantic information corresponding to the multiple grids.

[0120] Furthermore, the aforementioned processor 1301 is also specifically used for:

[0121] After obtaining the surround view image of the target vehicle, semantic segmentation processing is performed on the surround view image to locate the target object in the surround view image, wherein the target object is of at least one type.

[0122] Furthermore, the target region is located on both sides of the target vehicle; the segmentation of the target region in the target direction to obtain multiple grids includes: segmenting the target region in the target direction to obtain multiple grids and image semantic information corresponding to each grid; wherein, the image semantic information corresponding to each grid includes: the type of target object contained in the image region corresponding to the grid, and the number of pixels of each type of target object; the image semantic information corresponding to the auxiliary grid includes: the type of target object contained in the image region corresponding to the auxiliary grid, and the number of pixels of each type of target object.

[0123] Further, determining the parking space based on the image semantic information corresponding to the multiple grids includes: encoding the multiple grids respectively based on the image semantic information corresponding to the multiple grids; wherein, if the image region corresponding to the grid is determined to contain at least one type of target object based on the image semantic information, and the number of pixels of at least one type of target object is not less than the corresponding pixel threshold, then the grid is encoded as a first identifier code; otherwise, the grid is encoded as a second identifier code; within the preset distance range, if the number of grids encoded as the second identifier code meets a preset condition, then the image region corresponding to the preset distance range is determined to correspond to a parking space; wherein the preset distance range is related to the actual parking space width.

[0124] Furthermore, the aforementioned processor 1301 is also specifically used for:

[0125] After determining that the image region corresponding to the preset distance range corresponds to a parking space, for a preset number of target grids located at the edge position in all grids within the preset distance range, it is determined whether the target grid corresponds to the simulated parking space line based on the image semantic information corresponding to the target grid and the image semantic information corresponding to the auxiliary grid; wherein, the auxiliary grid is the adjacent grid of the target grid.

[0126] Furthermore, determining whether the target grid corresponds to a simulated parking space line based on the image semantic information corresponding to the target grid and the image semantic information corresponding to the auxiliary grid includes: determining whether the target grid is encoded as the first identifier; determining whether there is a grid in the auxiliary grid that is encoded as the second identifier; and determining that the target grid corresponds to a simulated parking space line when the target grid is encoded as the first identifier and there is a grid in the auxiliary grid that is encoded as the second identifier.

[0127] Furthermore, the aforementioned processor 1301 is also specifically used for:

[0128] After determining that the image region corresponding to the preset distance range corresponds to a parking space, based on the image semantic information of the grid, it is determined whether the image region corresponding to the grid contains pixels of the parking line type, and the number of pixels is not less than the pixel threshold of the parking line type; if the grid contains pixels of the parking line type, and the number of pixels is not less than the pixel threshold of the parking line type, it is determined that the grid corresponds to a parking line.

[0129] Furthermore, the aforementioned processor 1301 is also specifically used for:

[0130] Before determining the parking space based on the image semantic information corresponding to the multiple grids, if the image semantic information of the panoramic image indicates the presence of a distorted image region, at least one auxiliary image is acquired again; a local image region corresponding to the distorted image region is determined from the at least one auxiliary image; and the distortion is determined based on the image semantic information of the local image region.

[0131] Further, determining the distortion variable based on the image semantic information of the aforementioned local image region includes: segmenting the aforementioned local image region along the aforementioned target direction to obtain N grids, where N is an integer greater than 1; encoding the i-th grid based on the image semantic information of the i-th grid; wherein, if the image region corresponding to the i-th grid contains pixels of the vehicle type, and the number of pixels is not less than the pixel threshold corresponding to the vehicle type, then the i-th grid is encoded as a first identifier code; otherwise, the i-th grid is encoded as a second identifier code, where i is a positive integer not greater than N; and determining the distortion variable based on the number of grids identified as the first identifier code.

[0132] Memory 1302 may include one or more computer-readable storage media, which may be non-transitory. Memory 1302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments of this disclosure, the non-transitory computer-readable storage media in memory 1302 are used to store at least one instruction for execution by processor 1301 to implement the methods in the embodiments of this disclosure.

[0133] In some embodiments, the electronic device 1300 further includes a peripheral device interface 1303 and at least one peripheral device. The processor 1301, memory 1302, and peripheral device interface 1303 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1303 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a display screen 1304, a camera 1305, and an audio circuit 1306.

[0134] Peripheral device interface 1303 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1301 and memory 1302. In some embodiments of this disclosure, processor 1301, memory 1302, and peripheral device interface 1303 are integrated on the same chip or circuit board; in other embodiments of this disclosure, any one or two of processor 1301, memory 1302, and peripheral device interface 1303 can be implemented on separate chips or circuit boards. This disclosure does not specifically limit the scope of the embodiments.

[0135] Display screen 1304 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1304 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1301 for processing. In this case, display screen 1304 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments of this disclosure, there may be one display screen 1304, which serves as the front panel of electronic device 1300; in other embodiments, there may be at least two display screens 1304, respectively disposed on different surfaces of electronic device 1300 or in a folded design; in some embodiments, display screen 1304 may be a flexible display screen, disposed on a curved or folded surface of electronic device 1300. Furthermore, display screen 1304 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1304 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0136] Camera 1305 is used to capture images or videos. Optionally, camera 1305 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the electronic device, and the rear-facing camera is located on the back of the electronic device. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusing the main camera and the depth-sensing camera, panoramic shooting by fusing the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments of this disclosure, camera 1305 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0137] The audio circuit 1306 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 1301 for processing. For stereo sound acquisition or noise reduction purposes, there may be multiple microphones, each located in a different part of the electronic device 1300. The microphone may also be an array microphone or an omnidirectional microphone.

[0138] Power supply 1307 is used to supply power to various components in electronic device 1300. Power supply 1307 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1307 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0139] The block diagram of the electronic device shown in this embodiment does not constitute a limitation on the electronic device 1300. The electronic device 1300 may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0140] In the description of this disclosure, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this disclosure based on the specific circumstances. Furthermore, in the description of this disclosure, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0141] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, equivalent variations made in accordance with the claims of this disclosure are still within the scope of this disclosure.

Claims

1. A parking space recognition apparatus characterized by comprising: The device comprises: an acquisition module configured to acquire a surround view image of a target vehicle and determine a target region in the surround view image; a segmentation module configured to segment the target region in a target direction to obtain a plurality of grids, wherein the target direction is perpendicular to a driving direction of the target vehicle, and a grid width of the grids obtained by segmentation is adapted to a width of a parking line; a first determination module configured to determine a parking space according to image semantic information corresponding to the plurality of grids respectively; the first determination module comprises an encoding unit and a determination unit; the encoding unit is configured to encode the plurality of grids respectively according to the image semantic information corresponding to the plurality of grids respectively; wherein if it is determined according to the image semantic information that at least one type of target object is contained in an image region corresponding to a grid, and a pixel number of the at least one type of target object is not less than a corresponding pixel threshold, the grid is encoded as a first identification code, otherwise, the grid is encoded as a second identification code; the determination unit is configured to, within a preset distance range, if a number of grids encoded as the second identification code satisfies a preset condition, determine that an image region corresponding to the preset distance range corresponds to a parking space; wherein the preset distance range is related to an actual parking space width; and a first positioning module configured to, for a preset number of target grids at edge positions in all grids within the preset distance range, determine whether the target grids correspond to a target parking line according to image semantic information corresponding to the target grids and image semantic information corresponding to auxiliary grids; wherein the auxiliary grids are adjacent grids of the target grids.

2. The identification device of claim 1, wherein The grid width is determined according to a resolution of the surround view image.

3. The identification device according to claim 1 or 2, characterized in that The acquisition module is further configured to perform semantic segmentation processing on the surround view image to locate target objects in the surround view image, wherein the target objects are of at least one type.

4. The identification device of claim 3, wherein Pixels of the surround view image are associated with labels or categories.

5. The identification device according to claim 3 or 4, characterized in that The acquisition module is specifically configured to perform semantic segmentation processing on the surround view image to locate pixel combinations corresponding to vehicles, or pixel combinations corresponding to pedestrians, or pixel sets corresponding to traffic signal signs, or pixel sets corresponding to sidewalks in the surround view image.

6. The identification device of claim 3, wherein, The acquisition module is further configured to: split the surround view image into a plurality of channels according to a number of types of the target objects in the surround view image, and perform binaryzation processing on the channels.

7. The identification device of claim 1, wherein The image semantic information comprises types of target objects contained in image regions, and pixel numbers of each target object.

8. The identification device of claim 1, wherein The acquisition module is further configured to, according to image semantic information of the surround view image, if it is determined that there is a distorted image region in the surround view image, acquire at least one auxiliary image again; determine a local image region corresponding to the distorted image region in the at least one auxiliary image; determine a distortion amount according to image semantic information of the local image region.

9. A method of identifying a parking space, characterized by, The method comprises: acquiring a surround view image of a target vehicle and determining a target region in the surround view image; Split the target region in a target direction to obtain a plurality of grids, wherein the target direction is perpendicular to a driving direction of the target vehicle, and a grid width of the grids is adapted to a width of a parking line; Determine a parking space according to image semantic information corresponding to the plurality of grids respectively; The determining of the parking space according to the image semantic information corresponding to the plurality of grids respectively comprises: Encode the plurality of grids respectively according to the image semantic information corresponding to the plurality of grids respectively; If the image region corresponding to a grid is determined to contain at least one type of target object according to the image semantic information, and the number of pixels of the at least one type of target object is not less than a corresponding pixel threshold, then the grid is encoded as a first identification code, otherwise, the grid is encoded as a second identification code; If the number of grids encoded as the second identification code in a preset distance range satisfies a preset condition, then it is determined that an image region corresponding to the preset distance range corresponds to a parking space; The preset distance range is related to an actual parking space width; For a preset number of target grids at edge positions in all grids in the preset distance range, it is determined whether the target grids correspond to a tentative parking line according to image semantic information corresponding to the target grids and image semantic information corresponding to auxiliary grids; The auxiliary grids are adjacent grids of the target grids.

Images