Parking space monitoring method, device and equipment based on image recognition

Abstract

The present disclosure relates to a kind of parking space monitoring method, device and equipment based on image recognition, comprising: the depth image that the image acquisition device set in parking space is collected, the distance of target vehicle and image acquisition device is determined according to depth image, if distance is less than or equal to preset distance threshold, with the collection time of this depth image as benchmark, extract multiple frames target depth image from subsequently collected depth image;With parking line as boundary, the depth image to be identified including target vehicle is intercepted from target depth image, the depth value of target vehicle in the depth image to be identified and in the pixel coordinate of the plane where this depth value is located, determine overhead view image;Calculate the area of the occupied area of target vehicle in parking space in overhead view image;If the ratio of occupied area and parking space area exceeds preset area ratio threshold, determine that parking space is in occupied state, if the ratio does not exceed preset area ratio threshold, determine that parking space is in idle state.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a parking space monitoring method, apparatus and equipment based on image recognition. Background Technology

[0002] Parking spaces in parking lots are usually equipped with image acquisition devices such as cameras. The images captured by these devices are used to identify whether a parking space is occupied. However, if a vehicle turns around or avoids other vehicles in an empty parking space, it will also be identified as an occupied parking space, resulting in low accuracy in monitoring whether a parking space is occupied. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a parking space monitoring method, device and equipment based on image recognition.

[0004] According to a first aspect of the present disclosure, a parking space monitoring method based on image recognition is provided, comprising:

[0005] Acquire a depth image captured by an image acquisition device set in the parking space, and determine the distance between the target vehicle and the image acquisition device based on the depth image. If the distance is less than or equal to a preset distance threshold, extract multiple target depth images from the subsequently acquired depth images, using the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold as a reference.

[0006] Using the parking line of the parking space as the boundary, extract the depth image to be identified from each of the multiple frames of the target depth image, including the target vehicle, and determine the top view image of the target vehicle in the parking space based on the depth value of the target vehicle in each depth image to be identified and the pixel coordinates of the plane where the depth value is located.

[0007] Calculate the area occupied by the target vehicle in the parking space in the top-view image;

[0008] If the ratio of the occupied area to the parking space area exceeds a preset area ratio threshold, the parking space is determined to be occupied. If the ratio of the occupied area to the parking space area does not exceed the preset area ratio threshold, the parking space is determined to be vacant.

[0009] In a preferred embodiment, determining the top-view image of the target vehicle in the parking space based on the depth value of the target vehicle in each of the depth images to be identified and the pixel coordinates of the plane containing the depth value includes:

[0010] Using the depth value of the target vehicle in each of the depth images to be identified, each depth image to be identified is mapped to the coverage area of ​​the virtual parking space corresponding to the parking space according to the depth value.

[0011] Based on the pixel coordinates of the target vehicle in the plane where the depth value is located in each depth image to be identified, the extreme pixel coordinates of each depth value on the plane are determined, wherein the extreme pixel coordinates are the pixel coordinates of the target vehicle in the depth image to be identified corresponding to each depth value that are closest to the left and right parking lines of the parking space.

[0012] Based on the extreme pixel coordinates corresponding to each depth value, a top-down view image of the target vehicle at the parking space is generated.

[0013] In a preferred embodiment, generating a top-view image of the target vehicle in the parking space based on the corresponding extreme pixel coordinates at each depth value includes:

[0014] Based on the relationship between the depth values, adjacent depth values ​​are sequentially connected to the extreme pixel coordinates of the parking space corresponding to the same parking line to obtain a connected component image;

[0015] The viewpoint of the connected component image is transformed to generate a top-down view of the target vehicle at the parking space.

[0016] In a preferred embodiment, when the distance is less than or equal to a preset distance threshold, extracting multiple frames of target depth images from subsequently acquired depth images, based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold, includes:

[0017] When the distance is less than or equal to a preset distance threshold, the number of image frames of the depth image subsequently acquired within a preset time period is counted, based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold.

[0018] If the number of image frames exceeds a preset threshold, then based on the pixel coordinates of each depth image in the plane where the depth value is located, the unqualified depth images of the target vehicle that exceed the parking line of the parking space are removed.

[0019] If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images still exceeds the preset number threshold, then depth images are randomly removed from the remaining backup depth images until the number of image frames in the remaining backup depth images equals the preset number threshold.

[0020] If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images is less than the preset number threshold, then the depth difference between the depth values ​​of the target vehicle in the adjacent backup images after sorting by acquisition time is calculated.

[0021] The depth differences are sorted by size and, in descending order, it is determined whether there are any unqualified depth images that have been rejected between adjacent backup images.

[0022] If there are rejected unqualified depth images among adjacent backup images, one of the rejected unqualified depth images is restored as the backup depth image until the number of image frames of the backup depth image is equal to the preset number threshold. Then, the restoration of one of the rejected unqualified depth images as the backup depth image is stopped, and the backup depth image is used as multiple frames of the target depth image.

[0023] In a preferred embodiment, the method further includes:

[0024] If, after restoring one of the rejected unqualified depth images as a backup depth image, the number of image frames in the backup depth image is still less than the preset threshold, then the process of sequentially determining whether there are rejected unqualified depth images between adjacent backup images is repeated in descending order. If there are rejected unqualified depth images between adjacent backup images, then the step of restoring one of the rejected unqualified depth images as the backup depth image is continued until the number of image frames in the backup depth image equals the preset threshold, at which point the process of restoring one of the rejected unqualified depth images as a backup depth image is stopped.

[0025] In a preferred embodiment, the step of restoring one of the rejected unqualified depth images as the backup depth image if there are rejected unqualified depth images among adjacent backup images includes:

[0026] If there are rejected unqualified depth images between adjacent backup images, then the number of rejected image frames of the rejected unqualified depth images between adjacent backup images is determined.

[0027] If the number of rejected image frames is 1, then the unqualified depth image is directly restored as the backup depth image;

[0028] If the number of rejected image frames is greater than 1, then the unqualified depth image with the smallest difference between the depth value and half of the depth difference is restored as the backup depth image.

[0029] In a preferred embodiment, the method further includes:

[0030] If the number of image frames does not exceed the preset number threshold, then the foreground region and background region of each depth image are identified. The foreground region is the area where the target vehicle is located in the depth image, and the background region is other areas in the depth image besides the foreground region.

[0031] Based on the foreground region corresponding to each depth image, the change value of the foreground region of depth images acquired at adjacent times is calculated, and based on the background region corresponding to each depth image, the change value of the background region of depth images acquired at adjacent times is calculated.

[0032] If the change value in the foreground region exceeds a preset change threshold, and / or if the change value in the background region exceeds the preset change threshold, the background region of the depth image acquired earlier in the adjacent depth images is replaced with the preset image, and a replacement depth image is generated based on the preset image and the corresponding foreground region.

[0033] Each depth image and each replacement depth image are divided into different image groups. Each image group includes a forward image and a backward image. The interpolation time phase corresponding to each image group is determined, and the images in the corresponding image group are interpolated based on the interpolation time phase corresponding to each image group to obtain the interpolated image corresponding to each image group.

[0034] The multi-frame target depth image is obtained based on the interpolated image and the depth image.

[0035] In a preferred embodiment, the method further includes:

[0036] If the change value of the foreground region is less than the preset change threshold, and the change value of the background region is less than the preset change threshold, the background region of each depth image in the adjacent depth images is retained.

[0037] According to a second aspect of the present disclosure, a parking space monitoring device based on image recognition is provided, comprising:

[0038] The acquisition module is configured to acquire a depth image captured by an image acquisition device set in the parking space, and determine the distance between the target vehicle and the image acquisition device based on the depth image. If the distance is less than or equal to a preset distance threshold, multiple frames of target depth images are extracted from the subsequently acquired depth images, with the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold as a reference.

[0039] The cropping module is configured to crop out a depth image to be identified from multiple frames of the target depth image, using the parking line of the parking space as the boundary, and to determine the top view image of the target vehicle in the parking space based on the depth value of the target vehicle in each depth image to be identified and the pixel coordinates of the plane where the depth value is located.

[0040] The calculation module is configured to calculate the area occupied by the target vehicle in the parking space in the top view image;

[0041] The determination module is configured to determine that the parking space is occupied when the ratio of the occupied area to the parking space area exceeds a preset area ratio threshold, and to determine that the parking space is vacant when the ratio of the occupied area to the parking space area does not exceed the preset area ratio threshold.

[0042] In a preferred embodiment, the interception module is configured to:

[0043] Using the depth value of the target vehicle in each of the depth images to be identified, each depth image to be identified is mapped to the coverage area of ​​the virtual parking space corresponding to the parking space according to the depth value.

[0044] Based on the pixel coordinates of the target vehicle in the plane where the depth value is located in each depth image to be identified, the extreme pixel coordinates of each depth value on the plane are determined, wherein the extreme pixel coordinates are the pixel coordinates of the target vehicle in the depth image to be identified corresponding to each depth value that are closest to the left and right parking lines of the parking space.

[0045] Based on the extreme pixel coordinates corresponding to each depth value, a top-down view image of the target vehicle at the parking space is generated.

[0046] In a preferred embodiment, the interception module is configured to:

[0047] Based on the relationship between the depth values, adjacent depth values ​​are sequentially connected to the extreme pixel coordinates of the parking space corresponding to the same parking line to obtain a connected component image;

[0048] The viewpoint of the connected component image is transformed to generate a top-down view of the target vehicle at the parking space.

[0049] In a preferred embodiment, the acquisition module is configured to:

[0050] When the distance is less than or equal to a preset distance threshold, the number of image frames of the depth image subsequently acquired within a preset time period is counted, based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold.

[0051] If the number of image frames exceeds a preset threshold, then based on the pixel coordinates of each depth image in the plane where the depth value is located, the unqualified depth images of the target vehicle that exceed the parking line of the parking space are removed.

[0052] If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images still exceeds the preset number threshold, then depth images are randomly removed from the remaining backup depth images until the number of image frames in the remaining backup depth images equals the preset number threshold.

[0053] If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images is less than the preset number threshold, then the depth difference between the depth values ​​of the target vehicle in the adjacent backup images after sorting by acquisition time is calculated.

[0054] The depth differences are sorted by size and, in descending order, it is determined whether there are any unqualified depth images that have been rejected between adjacent backup images.

[0055] If there are rejected unqualified depth images among adjacent backup images, one of the rejected unqualified depth images is restored as the backup depth image until the number of image frames of the backup depth image is equal to the preset number threshold. Then, the restoration of one of the rejected unqualified depth images as the backup depth image is stopped, and the backup depth image is used as multiple frames of the target depth image.

[0056] In a preferred embodiment, the acquisition module is configured to:

[0057] If, after restoring one of the rejected unqualified depth images as a backup depth image, the number of image frames in the backup depth image is still less than the preset threshold, then the process of sequentially determining whether there are rejected unqualified depth images between adjacent backup images is repeated in descending order. If there are rejected unqualified depth images between adjacent backup images, then the step of restoring one of the rejected unqualified depth images as the backup depth image is continued until the number of image frames in the backup depth image equals the preset threshold, at which point the process of restoring one of the rejected unqualified depth images as a backup depth image is stopped.

[0058] In a preferred embodiment, the acquisition module is configured to:

[0059] If there are rejected unqualified depth images between adjacent backup images, then the number of rejected image frames of the rejected unqualified depth images between adjacent backup images is determined.

[0060] If the number of rejected image frames is 1, then the unqualified depth image is directly restored as the backup depth image;

[0061] If the number of rejected image frames is greater than 1, then the unqualified depth image with the smallest difference between the depth value and half of the depth difference is restored as the backup depth image.

[0062] In a preferred embodiment, the acquisition module is configured to:

[0063] If the number of image frames does not exceed the preset number threshold, then the foreground region and background region of each depth image are identified. The foreground region is the area where the target vehicle is located in the depth image, and the background region is other areas in the depth image besides the foreground region.

[0064] Based on the foreground region corresponding to each depth image, the change value of the foreground region of depth images acquired at adjacent times is calculated, and based on the background region corresponding to each depth image, the change value of the background region of depth images acquired at adjacent times is calculated.

[0065] If the change value in the foreground region exceeds a preset change threshold, and / or if the change value in the background region exceeds the preset change threshold, the background region of the depth image acquired earlier in the adjacent depth images is replaced with the preset image, and a replacement depth image is generated based on the preset image and the corresponding foreground region.

[0066] Each depth image and each replacement depth image are divided into different image groups. Each image group includes a forward image and a backward image. The interpolation time phase corresponding to each image group is determined, and the images in the corresponding image group are interpolated based on the interpolation time phase corresponding to each image group to obtain the interpolated image corresponding to each image group.

[0067] The multi-frame target depth image is obtained based on the interpolated image and the depth image.

[0068] In a preferred embodiment, the acquisition module is configured to:

[0069] If the change value of the foreground region is less than the preset change threshold, and the change value of the background region is less than the preset change threshold, the background region of each depth image in the adjacent depth images is retained.

[0070] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0071] processor;

[0072] Memory used to store processor-executable instructions;

[0073] The processor is configured to execute the executable instructions stored in the memory to implement the method of any one of the first aspects.

[0074] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0075] The system acquires depth images from image acquisition devices installed in parking spaces. Based on these depth images, the distance between the target vehicle and the acquisition device is determined. If the distance is less than or equal to a preset distance threshold, multiple target depth images are extracted from subsequently acquired depth images, using the acquisition time of that depth image as a baseline. This avoids automatically classifying a parking space as occupied upon vehicle entry, reducing false identifications when a vehicle is avoiding other vehicles or making a U-turn. Using the parking line as a boundary, a depth image including the target vehicle is extracted from the target depth image. A top-view image is determined based on the target vehicle's depth value and its pixel coordinates on the plane containing that depth value. The area occupied by the target vehicle within the parking space is calculated in the top-view image. If the ratio of the occupied area to the parking space area exceeds a preset area ratio threshold, the parking space is determined to be occupied; otherwise, it is determined to be vacant. This improves the accuracy of monitoring whether a parking space is occupied.

[0076] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0077] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0078] Figure 1 This is a flowchart illustrating an image recognition-based parking space monitoring method according to an exemplary embodiment.

[0079] Figure 2 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart for step S12.

[0080] Figure 3 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart for step S11.

[0081] Figure 4 This is a block diagram illustrating an image recognition-based parking space monitoring device according to an exemplary embodiment.

[0082] Figure 5 This is a block diagram illustrating an apparatus for image recognition-based parking space monitoring according to an exemplary embodiment. Detailed Implementation

[0083] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0084] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0085] Figure 1 This is a flowchart illustrating an image recognition-based parking space monitoring method according to an exemplary embodiment. This image recognition-based parking space monitoring method can be applied to a parking lot management server, such as... Figure 1 As shown, the method includes the following steps.

[0086] In step S11, a depth image is acquired by an image acquisition device set in the parking space, and the distance between the target vehicle and the image acquisition device is determined based on the depth image. If the distance is less than or equal to a preset distance threshold, multiple target depth images are extracted from the subsequently acquired depth images, based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold.

[0087] In this embodiment of the disclosure, the image acquisition device may be a camera installed behind the parking space, wherein the area behind the parking space is a direction in which vehicles cannot drive out.

[0088] In this embodiment, an infrared device can be installed in each parking space. The image acquisition device is in a closed state when the infrared device is not triggered. When a vehicle enters and triggers the infrared device, the image acquisition device is turned on. After the vehicle is parked in the parking space, the image acquisition device is turned off, thereby reducing the power consumption of the image acquisition device.

[0089] The preset distance threshold can be set according to the width of the parking space. For example, the preset distance threshold is equal to the width of the parking lot, where the width refers to the length of the parking line on the shorter side of the parking space.

[0090] In this embodiment, the image acquisition device can be a three-color channel depth camera. This allows for the acquisition of the target vehicle's planar pixel coordinates and depth information relative to the image acquisition device.

[0091] In step S12, using the parking line of the parking space as the boundary, a depth image to be identified, including the target vehicle, is extracted from each of the multiple frames of the target depth image. Based on the depth value of the target vehicle in each depth image to be identified and the pixel coordinates of the plane where the depth value is located, a top-view image of the target vehicle in the parking space is determined.

[0092] It is understandable that parking lines can include the left and right lines of a parking space. The left and right lines are relative to the image acquisition device. For example, in the case of a reverse parking space, the image acquisition device is located at the rear of the vehicle in the parking space, and the left and right lines actually refer to the parking lines on the left and right sides of the vehicle after it is parked. In the case of a parallel parking space, the image acquisition device is located on the side of the parking space, and the left and right lines actually refer to the parking lines in front of and behind the vehicle after it is parked.

[0093] It is understandable that the depth value can represent the distance of the target vehicle relative to the plane where the image acquisition device is located, while the pixel coordinates are the two-dimensional coordinates on the plane where the depth value is located.

[0094] In step S13, the area occupied by the target vehicle in the parking space in the top view image is calculated.

[0095] In step S14, if the ratio of the occupied area to the parking space area exceeds a preset area ratio threshold, the parking space is determined to be occupied; if the ratio of the occupied area to the parking space area does not exceed the preset area ratio threshold, the parking space is determined to be vacant.

[0096] The above technical solution acquires depth images from image acquisition devices installed in parking spaces. Based on these depth images, the distance between the target vehicle and the image acquisition device is determined. If the distance is less than or equal to a preset distance threshold, multiple target depth images are extracted from subsequently acquired depth images, using the acquisition time of the depth image as a reference. This avoids automatically classifying the parking space as occupied upon detecting a vehicle entering, reducing false identifications when the vehicle is avoiding other vehicles or making a U-turn. Using the parking line as a boundary, a depth image including the target vehicle is extracted from the target depth image. Based on the depth value of the target vehicle in the depth image and its pixel coordinates relative to the image acquisition device, a top-view image is determined. The area occupied by the target vehicle in the parking space is calculated in the top-view image. If the ratio of the occupied area to the parking space area exceeds a preset area ratio threshold, the parking space is determined to be occupied; otherwise, the parking space is determined to be vacant. This improves the accuracy of monitoring whether a parking space is occupied.

[0097] In a preferred embodiment, see Figure 2 As shown, in step S12, determining the top-view image of the target vehicle in the parking space based on the depth value of the target vehicle in each depth image to be identified and the pixel coordinates of the plane containing that depth value includes:

[0098] In step S121, each depth image to be identified is mapped to the coverage area of ​​the virtual parking space corresponding to the parking space according to the depth value of the target vehicle in each depth image to be identified.

[0099] The virtual parking space is obtained by projecting a parking space. For example, a virtual parking space is obtained by projecting a top-down view of a parking space, and the virtual parking space has the same size and shape as the parking space.

[0100] In step S122, the limiting pixel coordinates of each depth value on the plane are determined based on the pixel coordinates of the target vehicle in each depth image to be identified on the plane where the depth value is located.

[0101] Wherein, the extreme pixel coordinates are the pixel coordinates of the target vehicle relative to the left and right parking lines of the parking space in the depth image to be identified for each depth value.

[0102] The extreme pixel coordinates typically include the pixel coordinates closest to the left parking line and the pixel coordinates closest to the right parking line. Taking a reverse parking space as an example, each depth value corresponds to an extreme pixel coordinate with both the closest left and right parking lines.

[0103] In step S123, a top-view image of the target vehicle in the parking space is generated based on the extreme pixel coordinates corresponding to each depth value.

[0104] It can be explained that the extreme pixel coordinates can be used to represent the distance between the part of the target vehicle closest to the parking line and the parking line.

[0105] In a preferred embodiment, step S123, generating a top-view image of the target vehicle in the parking space based on the corresponding extreme pixel coordinates at each depth value, includes:

[0106] Based on the relationship between the depth values, adjacent depth values ​​are sequentially connected to the limiting pixel coordinates of the parking space corresponding to the same parking line to obtain a connected component image.

[0107] For example, the extreme pixel coordinates of adjacent depth values ​​on the left parking line side are connected, and the extreme pixel coordinates of adjacent depth values ​​on the right parking line side are connected. Then, based on the depth image with the smallest depth value and the depth image with the largest depth value, a connected component image is constructed. The depth image with the largest depth value is the depth image acquired when the distance between the target vehicle and the image acquisition device is equal to a preset distance threshold.

[0108] The viewpoint of the connected component image is transformed to generate a top-down view of the target vehicle at the parking space.

[0109] In a preferred embodiment, see Figure 3 As shown, in step S11, when the distance is less than or equal to a preset distance threshold, extracting multiple frames of target depth images from subsequently acquired depth images based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold includes:

[0110] In step S111, when the distance is less than or equal to a preset distance threshold, the number of image frames of the depth image subsequently acquired within a preset time period is counted based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold.

[0111] In step S112, if the number of image frames exceeds a preset threshold, then based on the pixel coordinates of each depth image in the plane where the depth value is located, the unqualified depth images of the target vehicle that exceed the parking line of the parking space are removed.

[0112] An unqualified depth image is one in which any part of the target vehicle extends beyond the parking area of ​​the parking space.

[0113] In step S113, if the number of image frames in the remaining backup depth images still exceeds the preset number threshold after removing the unqualified depth images, then depth images are randomly removed from the remaining backup depth images until the number of image frames in the remaining backup depth images is equal to the preset number threshold.

[0114] In step S114, if the number of image frames in the remaining backup depth images is less than the preset number threshold after removing the unqualified depth images, then the depth difference between the depth values ​​of the target vehicle in the adjacent backup images after sorting by acquisition time is calculated.

[0115] In step S115, the depth differences are sorted by size and, in descending order, it is determined whether there are any unqualified depth images that have been rejected between adjacent backup images.

[0116] In step S116, if there are rejected unqualified depth images among adjacent backup images, one of the rejected unqualified depth images is restored as the backup depth image until the number of image frames of the backup depth image is equal to the preset number threshold. Then, the restoration of one of the rejected unqualified depth images as the backup depth image is stopped, and the backup depth image is used as multiple frames of the target depth image.

[0117] In a preferred embodiment, the method further includes:

[0118] If, after recovering one of the rejected unqualified depth images as a backup depth image, the number of image frames in the backup depth image is still less than the preset threshold, then the process of sequentially determining whether there are rejected unqualified depth images between adjacent backup images is repeated in descending order. If there are rejected unqualified depth images between adjacent backup images, then the step of recovering one of the rejected unqualified depth images as the backup depth image continues until the number of image frames in the backup depth image equals the preset threshold, at which point the process of recovering one of the rejected unqualified depth images as a backup depth image stops.

[0119] In a preferred embodiment, the step of restoring one of the rejected unqualified depth images as the backup depth image if there are rejected unqualified depth images among adjacent backup images includes:

[0120] If there are rejected unqualified depth images between adjacent backup images, then the number of rejected image frames of the rejected unqualified depth images between adjacent backup images is determined.

[0121] If the number of rejected image frames is 1, then the unqualified depth image is directly restored as the backup depth image.

[0122] If the number of rejected image frames is greater than 1, then the unqualified depth image with the smallest difference between the depth value and half of the depth difference is restored as the backup depth image.

[0123] In a preferred embodiment, the method further includes:

[0124] If the number of image frames does not exceed the preset number threshold, then the foreground region and background region of each depth image are identified. The foreground region is the area where the target vehicle is located in the depth image, and the background region is other areas in the depth image other than the foreground region.

[0125] Based on the foreground region corresponding to each depth image, the change value of the foreground region of depth images acquired at adjacent times is calculated, and based on the background region corresponding to each depth image, the change value of the background region of depth images acquired at adjacent times is calculated.

[0126] If the change value in the foreground region exceeds a preset change threshold, and / or if the change value in the background region exceeds the preset change threshold, the background region of the earlier acquired depth image in the adjacent depth images is replaced with the preset image, and a replacement depth image is generated based on the preset image and the corresponding foreground region.

[0127] Each depth image and each replacement depth image are divided into different image groups. Each image group includes a forward image and a backward image. The interpolation time phase corresponding to each image group is determined, and the images in the corresponding image group are interpolated based on the interpolation time phase corresponding to each image group to obtain the interpolated image corresponding to each image group.

[0128] The multi-frame target depth image is obtained based on the interpolated image and the depth image.

[0129] In a preferred embodiment, the method further includes:

[0130] If the change value of the foreground region is less than the preset change threshold, and the change value of the background region is less than the preset change threshold, the background region of each depth image in the adjacent depth images is retained.

[0131] This disclosure also provides a parking space monitoring device based on image recognition, see [link to relevant documentation]. Figure 4 As shown, the image recognition-based parking space monitoring device 400 includes:

[0132] The acquisition module 410 is configured to acquire a depth image captured by an image acquisition device set in the parking space, and determine the distance between the target vehicle and the image acquisition device based on the depth image. If the distance is less than or equal to a preset distance threshold, multiple frames of target depth images are extracted from the subsequently acquired depth images based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold.

[0133] The cropping module 420 is configured to crop out a depth image to be identified from multiple frames of the target depth image, using the parking line of the parking space as the boundary, and to determine the top view image of the target vehicle in the parking space based on the depth value of the target vehicle in each depth image to be identified and the pixel coordinates of the plane where the depth value is located.

[0134] Calculation module 430 is configured to calculate the area occupied by the target vehicle in the parking space in the top view image;

[0135] The determination module 440 is configured to determine that the parking space is occupied when the ratio of the occupied area to the parking space area exceeds a preset area ratio threshold, and to determine that the parking space is vacant when the ratio of the occupied area to the parking space area does not exceed the preset area ratio threshold.

[0136] In a preferred embodiment, the interception module 420 is configured to:

[0137] Using the depth value of the target vehicle in each of the depth images to be identified, each depth image to be identified is mapped to the coverage area of ​​the virtual parking space corresponding to the parking space according to the depth value.

[0138] Based on the pixel coordinates of the target vehicle in the plane where the depth value is located in each depth image to be identified, the extreme pixel coordinates of each depth value on the plane are determined, wherein the extreme pixel coordinates are the pixel coordinates of the target vehicle in the depth image to be identified corresponding to each depth value that are closest to the left and right parking lines of the parking space.

[0139] Based on the extreme pixel coordinates corresponding to each depth value, a top-down view image of the target vehicle at the parking space is generated.

[0140] In a preferred embodiment, the interception module 420 is configured to:

[0141] Based on the relationship between the depth values, adjacent depth values ​​are sequentially connected to the extreme pixel coordinates of the parking space corresponding to the same parking line to obtain a connected component image;

[0142] The viewpoint of the connected component image is transformed to generate a top-down view of the target vehicle at the parking space.

[0143] In a preferred embodiment, the acquisition module 410 is configured to:

[0144] When the distance is less than or equal to a preset distance threshold, the number of image frames of the depth image subsequently acquired within a preset time period is counted, based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold.

[0145] If the number of image frames exceeds a preset threshold, then based on the pixel coordinates of each depth image in the plane where the depth value is located, the unqualified depth images of the target vehicle that exceed the parking line of the parking space are removed.

[0146] If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images still exceeds the preset number threshold, then depth images are randomly removed from the remaining backup depth images until the number of image frames in the remaining backup depth images equals the preset number threshold.

[0147] If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images is less than the preset number threshold, then the depth difference between the depth values ​​of the target vehicle in the adjacent backup images after sorting by acquisition time is calculated.

[0148] The depth differences are sorted by size and, in descending order, it is determined whether there are any unqualified depth images that have been rejected between adjacent backup images.

[0149] If there are rejected unqualified depth images among adjacent backup images, one of the rejected unqualified depth images is restored as the backup depth image until the number of image frames of the backup depth image is equal to the preset number threshold. Then, the restoration of one of the rejected unqualified depth images as the backup depth image is stopped, and the backup depth image is used as multiple frames of the target depth image.

[0150] In a preferred embodiment, the acquisition module 410 is configured to:

[0151] If, after restoring one of the rejected unqualified depth images as a backup depth image, the number of image frames in the backup depth image is still less than the preset threshold, then the process of sequentially determining whether there are rejected unqualified depth images between adjacent backup images is repeated in descending order. If there are rejected unqualified depth images between adjacent backup images, then the step of restoring one of the rejected unqualified depth images as the backup depth image is continued until the number of image frames in the backup depth image equals the preset threshold, at which point the process of restoring one of the rejected unqualified depth images as a backup depth image is stopped.

[0152] In a preferred embodiment, the acquisition module 410 is configured to:

[0153] If there are rejected unqualified depth images between adjacent backup images, then the number of rejected image frames of the rejected unqualified depth images between adjacent backup images is determined.

[0154] If the number of rejected image frames is 1, then the unqualified depth image is directly restored as the backup depth image;

[0155] If the number of rejected image frames is greater than 1, then the unqualified depth image with the smallest difference between the depth value and half of the depth difference is restored as the backup depth image.

[0156] In a preferred embodiment, the acquisition module 410 is configured to:

[0157] If the number of image frames does not exceed the preset number threshold, then the foreground region and background region of each depth image are identified. The foreground region is the area where the target vehicle is located in the depth image, and the background region is other areas in the depth image besides the foreground region.

[0158] Based on the foreground region corresponding to each depth image, the change value of the foreground region of depth images acquired at adjacent times is calculated, and based on the background region corresponding to each depth image, the change value of the background region of depth images acquired at adjacent times is calculated.

[0159] If the change value in the foreground region exceeds a preset change threshold, and / or if the change value in the background region exceeds the preset change threshold, the background region of the depth image acquired earlier in the adjacent depth images is replaced with the preset image, and a replacement depth image is generated based on the preset image and the corresponding foreground region.

[0160] Each depth image and each replacement depth image are divided into different image groups. Each image group includes a forward image and a backward image. The interpolation time phase corresponding to each image group is determined, and the images in the corresponding image group are interpolated based on the interpolation time phase corresponding to each image group to obtain the interpolated image corresponding to each image group.

[0161] The multi-frame target depth image is obtained based on the interpolated image and the depth image.

[0162] In a preferred embodiment, the acquisition module 410 is configured to:

[0163] If the change value of the foreground region is less than the preset change threshold, and the change value of the background region is less than the preset change threshold, the background region of each depth image in the adjacent depth images is retained.

[0164] Regarding the image recognition-based parking space monitoring device 400 in the above embodiments, the specific methods by which each module performs its operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0165] This disclosure also provides an electronic device, including:

[0166] processor;

[0167] Memory used to store processor-executable instructions;

[0168] The processor is configured to execute the executable instructions stored in the memory to implement any of the methods described in the foregoing embodiments.

[0169] Figure 5 This is a block diagram illustrating an apparatus 1900 for image recognition-based parking space monitoring according to an exemplary embodiment. For example, apparatus 1900 may be provided as a parking space management server. (Refer to...) Figure 5 The device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the aforementioned image recognition-based parking space monitoring method.

[0170] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958. Device 1900 can operate on an operating system stored in memory 1932.

[0171] In another exemplary embodiment, a computer program product is also provided, comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described image recognition-based parking space monitoring method when executed by the programmable device.

[0172] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0173] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A parking space monitoring method based on image recognition, characterized in that, The image recognition-based parking space monitoring method includes: Acquire a depth image captured by an image acquisition device set in the parking space, and determine the distance between the target vehicle and the image acquisition device based on the depth image. If the distance is less than or equal to a preset distance threshold, extract multiple target depth images from the subsequently acquired depth images, using the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold as a reference. Using the parking line of the parking space as the boundary, extract the depth image to be identified from each of the multiple frames of the target depth image, including the target vehicle, and determine the top view image of the target vehicle in the parking space based on the depth value of the target vehicle in each depth image to be identified and the pixel coordinates of the plane where the depth value is located. Calculate the area occupied by the target vehicle in the parking space in the top-view image; If the ratio of the occupied area to the parking space area exceeds a preset area ratio threshold, the parking space is determined to be occupied; if the ratio of the occupied area to the parking space area does not exceed the preset area ratio threshold, the parking space is determined to be vacant. When the distance is less than or equal to a preset distance threshold, the step of extracting multiple frames of target depth images from subsequently acquired depth images, based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold, includes: When the distance is less than or equal to a preset distance threshold, the number of image frames of the depth image subsequently acquired within a preset time period is counted, based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold. If the number of image frames exceeds a preset threshold, then based on the pixel coordinates of each depth image in the plane where the depth value is located, the unqualified depth images of the target vehicle that exceed the parking line of the parking space are removed. If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images still exceeds the preset number threshold, then depth images are randomly removed from the remaining backup depth images until the number of image frames in the remaining backup depth images equals the preset number threshold. If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images is less than the preset number threshold, then the depth difference between the depth values ​​of the target vehicle in the adjacent backup images after sorting by acquisition time is calculated. The depth differences are sorted by size and, in descending order, it is determined whether there are any unqualified depth images that have been rejected between adjacent backup images. If there are rejected unqualified depth images among adjacent backup images, one of the rejected unqualified depth images is restored as the backup depth image until the number of image frames of the backup depth image is equal to the preset number threshold. Then, the restoration of one of the rejected unqualified depth images as the backup depth image is stopped, and the backup depth image is used as multiple frames of the target depth image.

2. The method according to claim 1, characterized in that, The step of determining the top-view image of the target vehicle in the parking space based on the depth value of the target vehicle in each depth image to be identified and the pixel coordinates of the plane containing the depth value includes: Using the depth value of the target vehicle in each of the depth images to be identified, each depth image to be identified is mapped to the coverage area of ​​the virtual parking space corresponding to the parking space according to the depth value. Based on the pixel coordinates of the target vehicle in the plane where the depth value is located in each depth image to be identified, the extreme pixel coordinates of each depth value on the plane are determined, wherein the extreme pixel coordinates are the pixel coordinates of the target vehicle in the depth image to be identified corresponding to each depth value that are closest to the left and right parking lines of the parking space. Based on the extreme pixel coordinates corresponding to each depth value, a top-down view image of the target vehicle at the parking space is generated.

3. The method according to claim 2, characterized in that, The step of generating a top-view image of the target vehicle in the parking space based on the extreme pixel coordinates corresponding to each depth value includes: Based on the relationship between the depth values, adjacent depth values ​​are sequentially connected to the extreme pixel coordinates of the parking space corresponding to the same parking line to obtain a connected component image; The viewpoint of the connected component image is transformed to generate a top-down view of the target vehicle at the parking space.

4. The method according to claim 1, characterized in that, The method further includes: If, after restoring one of the rejected unqualified depth images as a backup depth image, the number of image frames in the backup depth image is still less than the preset threshold, then the process of sequentially determining whether there are rejected unqualified depth images between adjacent backup images is repeated in descending order. If there are rejected unqualified depth images between adjacent backup images, then the step of restoring one of the rejected unqualified depth images as the backup depth image is continued until the number of image frames in the backup depth image equals the preset threshold, at which point the process of restoring one of the rejected unqualified depth images as a backup depth image is stopped.

5. The method according to claim 1, characterized in that, If there are rejected unqualified depth images among adjacent backup images, one of the rejected unqualified depth images is restored as the backup depth image, including: If there are rejected unqualified depth images between adjacent backup images, then the number of rejected image frames of the rejected unqualified depth images between adjacent backup images is determined. If the number of rejected image frames is 1, then the unqualified depth image is directly restored as the backup depth image; If the number of rejected image frames is greater than 1, then the unqualified depth image with the smallest difference between the depth value and half of the depth difference is restored as the backup depth image.

6. The method according to claim 1, characterized in that, The method further includes: If the number of image frames does not exceed the preset number threshold, then the foreground region and background region of each depth image are identified. The foreground region is the area where the target vehicle is located in the depth image, and the background region is other areas in the depth image besides the foreground region. Based on the foreground region corresponding to each depth image, the change value of the foreground region of depth images acquired at adjacent times is calculated, and based on the background region corresponding to each depth image, the change value of the background region of depth images acquired at adjacent times is calculated. If the change value in the foreground region exceeds a preset change threshold, and / or if the change value in the background region exceeds the preset change threshold, the background region of the depth image acquired earlier in the adjacent depth images is replaced with the preset image, and a replacement depth image is generated based on the preset image and the corresponding foreground region. Each depth image and each replacement depth image are divided into different image groups. Each image group includes a forward image and a backward image. The interpolation time phase corresponding to each image group is determined, and the images in the corresponding image group are interpolated based on the interpolation time phase corresponding to each image group to obtain the interpolated image corresponding to each image group. The multi-frame target depth image is obtained based on the interpolated image and the depth image.

7. The method according to claim 6, characterized in that, The method further includes: If the change value of the foreground region is less than the preset change threshold, and the change value of the background region is less than the preset change threshold, the background region of each depth image in the adjacent depth images is retained.

8. A parking space monitoring device based on image recognition, characterized in that, include: The acquisition module is configured to acquire a depth image captured by an image acquisition device set in the parking space, and determine the distance between the target vehicle and the image acquisition device based on the depth image. If the distance is less than or equal to a preset distance threshold, multiple frames of target depth images are extracted from the subsequently acquired depth images, with the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold as a reference. The cropping module is configured to crop out a depth image to be identified from multiple frames of the target depth image, using the parking line of the parking space as the boundary, and to determine the top view image of the target vehicle in the parking space based on the depth value of the target vehicle in each depth image to be identified and the pixel coordinates of the plane where the depth value is located. The calculation module is configured to calculate the area occupied by the target vehicle in the parking space in the top view image; The determination module is configured to determine that the parking space is occupied when the ratio of the occupied area to the parking space area exceeds a preset area ratio threshold, and to determine that the parking space is vacant when the ratio of the occupied area to the parking space area does not exceed the preset area ratio threshold. The acquisition module is configured to: when the distance is less than or equal to a preset distance threshold, based on the acquisition time of the depth image corresponding to the distance being less than or equal to the preset distance threshold, count the number of image frames of the depth image subsequently acquired within a preset time period. If the number of image frames exceeds a preset threshold, then based on the pixel coordinates of each depth image in the plane where the depth value is located, the unqualified depth images of the target vehicle that exceed the parking line of the parking space are removed. If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images still exceeds the preset number threshold, then depth images are randomly removed from the remaining backup depth images until the number of image frames in the remaining backup depth images equals the preset number threshold. If, after removing the unqualified depth images, the number of image frames in the remaining backup depth images is less than the preset number threshold, then the depth difference between the depth values ​​of the target vehicle in the adjacent backup images after sorting by acquisition time is calculated. The depth differences are sorted by size and, in descending order, it is determined whether there are any unqualified depth images that have been rejected between adjacent backup images. If there are rejected unqualified depth images among adjacent backup images, one of the rejected unqualified depth images is restored as the backup depth image until the number of image frames of the backup depth image is equal to the preset number threshold. Then, the restoration of one of the rejected unqualified depth images as the backup depth image is stopped, and the backup depth image is used as multiple frames of the target depth image.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions stored in the memory to implement the method of any one of claims 1-7.

Images