Idle point location identification method and device, medium, electronic equipment and program product

Abstract

The invention discloses an idle point location identification method and device, a medium, electronic equipment and a program product, and relates to the technical field of image processing, target object detection is carried out on a shot image to obtain an object detection frame corresponding to a target object included in the shot image, then fitting is carried out on the object detection frame in the shot image to obtain a fitting point location, and the fitting point location is identified. The method comprises the following steps: taking a shot image, determining a first depth corresponding to a fitting point location and a second depth corresponding to an object detection frame based on the shot image, then determining whether the fitting point location belongs to an idle point location or not according to the first depth and the second depth corresponding to the fitting point location, and detecting whether the fitting point location belongs to the idle point location or not through the depths. According to the method, the influence of illumination variation, shadow interference and appearance difference can be avoided, the idle point location can still be accurately and stably detected even under the complex environment condition, and the method has good robustness and universality.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to a method, apparatus, medium, electronic device, and program product for identifying vacant locations. Background Technology

[0002] In related technologies, image recognition technology is increasingly widely used in smart parking, intelligent warehousing, and retail management scenarios because it can identify the occupancy status of parking spaces in parking lots and the slots on warehouse shelves, thereby determining whether there are vacant spaces. Currently, it typically relies on object detection or segmentation algorithms to judge each frame of a single image. However, when there are changes in lighting, occlusion, perspective distortion, or detection gaps, false detections and missed detections are extremely likely to occur, leading to a decrease in the accuracy of vacant space recognition. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] Firstly, this disclosure provides a method for identifying vacant locations, including: Acquire images corresponding to the target scene; Target object detection is performed on the captured image to obtain the object detection box corresponding to the target object included in the captured image; The object detection box is fitted in the captured image to obtain fitted points, which are used to store the target object. Based on the captured image, determine the first depth corresponding to the fitted point and the second depth corresponding to the object detection box; For each fitted point, it is determined whether the fitted point is an idle point based on the first depth and the second depth corresponding to the fitted point.

[0005] Secondly, this disclosure provides a device for identifying vacant locations, including: The acquisition module is configured to acquire images corresponding to the target scene. The detection module is configured to perform target object detection on the captured image to obtain an object detection box corresponding to the target object included in the captured image; The fitting module is configured to fit the object detection box in the captured image to obtain fitting points, which are used to store the target object. The first determining module is configured to determine a first depth corresponding to the fitted point and a second depth corresponding to the object detection box based on the captured image. The second determining module is configured to determine whether a fitted point is an idle point based on a first depth and a second depth corresponding to each fitted point.

[0006] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0007] Fourthly, this disclosure provides an electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0008] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] Based on the above technical solution, by acquiring the captured image corresponding to the target scene, target object detection is performed on the captured image to obtain the object detection box corresponding to the target object included in the captured image. Then, the object detection box is fitted in the captured image to obtain the fitted points. Based on the captured image, the first depth corresponding to the fitted point and the second depth corresponding to the object detection box are determined. Then, based on the first depth and the second depth corresponding to the fitted point, it is determined whether the fitted point belongs to an empty point. The detection of whether the fitted point belongs to an empty point can be achieved by using depth, which can avoid the influence of changes in lighting, shadow interference, and appearance differences. Even under complex environmental conditions, it can still accurately and stably detect empty points, and has good robustness and versatility. For example, the empty point recognition method provided by the embodiments of this disclosure can be compatible with application scenarios with different appearances of vehicles, goods, and placement differences, and realize more universal empty space recognition. It can be flexibly applied to the detection of empty spaces in parking lots, shelf positions, factory pallets, and warehouse storage troughs.

[0010] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 This is a flowchart illustrating a method for identifying vacant locations according to some embodiments.

[0012] Figure 2 This is a schematic diagram of an object detection box according to some embodiments.

[0013] Figure 3 This is a schematic diagram of the fitted lines shown according to some embodiments.

[0014] Figure 4 This is a schematic diagram of sampling points shown according to some embodiments.

[0015] Figure 5 This is a schematic diagram illustrating the fitting points according to some embodiments.

[0016] Figure 6 This is a schematic diagram of the structure of a vacant location identification device according to some embodiments.

[0017] Figure 7 This is a schematic diagram of the structure of an electronic device according to some embodiments. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0020] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0022] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0024] Figure 1 This is a flowchart illustrating a method for identifying vacant locations according to some embodiments. For example... Figure 1 As shown, this disclosure provides a method for identifying vacant locations, which can be implemented using a vacant location identification device, which can be implemented in software and / or hardware. Figure 1 As shown, the method may include the following steps.

[0025] In step 110, the captured image corresponding to the target scene is obtained.

[0026] Here, the target scenario can refer to a parking lot, warehouse shelves, factory assembly line, or similar environment. For example, images of the target scenario can be acquired using cameras installed within it. For instance, in a smart parking application scenario, cameras installed in the parking lot can periodically or in real-time acquire images of the parking lot.

[0027] It should be understood that the acquired images can be single-frame images or videos. Furthermore, after acquiring the images, distortion correction, ROI (Region of Interest Cropping), and other processing can be performed to facilitate better object detection in subsequent steps.

[0028] In step 120, target object detection is performed on the captured image to obtain object detection boxes corresponding to the target objects included in the captured image.

[0029] Here, the target object varies depending on the target scenario. For example, in a parking lot, the target object could be vehicles. In a warehouse, the target object could be goods on shelves. An object detection box is a method in computer vision used to identify the location of detected objects in an image. Object detection boxes are typically rectangular boxes used to enclose and label objects of interest in an image.

[0030] Figure 2 This is a schematic diagram of an object detection box according to some embodiments. For example... Figure 2As shown, the object detection boxes identified in the captured image 200 include a first object detection box 201, a second object detection box 202, a third object detection box 203, a fourth object detection box 204, a fifth object detection box 205, a sixth object detection box 206, and a seventh object detection box 207.

[0031] In this embodiment of the disclosure, an object detection algorithm can be used to detect target objects in the captured image to obtain object detection boxes corresponding to each target object included in the captured image.

[0032] For example, the object detection algorithm can be CountGD (Multimodal Open World Counting Model). CountGD can detect regularly arranged target objects in a captured image and return the coordinate information of the object detection bounding boxes corresponding to each target object. Of course, other types of object detection algorithms can also be used, and this disclosure does not impose specific limitations on them.

[0033] For example, the object detection bounding box output by the object detection algorithm can be represented by (x, y, w, h, score, label), where x is the pixel coordinate of the object detection bounding box in the horizontal direction of the captured image, y is the pixel coordinate of the object detection bounding box in the vertical direction of the captured image, w is the width of the object detection bounding box, h is the height of the object detection bounding box, score is the confidence score of the object detection bounding box, and label is the label to which the target object belongs.

[0034] In step 130, the object detection box is fitted in the captured image to obtain the fitted points.

[0035] Here, after obtaining the object detection boxes corresponding to each target object in the captured image, the object detection boxes can be fitted in the coordinate system of the captured image to obtain fitted points. These fitted points are used to store the target objects; they can be understood as points obtained by fitting the object detection boxes onto the captured image that can be used to store the target objects. In other words, the fitted points can be points that can be inferred from the captured image using the object detection boxes to store the target objects.

[0036] In step 140, based on the captured image, the first depth corresponding to the fitted point and the second depth corresponding to the object detection box are determined.

[0037] Here, for each fitted point, the first depth corresponding to that fitted point can be determined by taking an image. For each object detection box, the second depth corresponding to that object detection box can be determined by taking an image.

[0038] For example, the captured image can be input into a depth estimation model to obtain the corresponding first depth and second depth.

[0039] It's important to note that the first depth can be understood as the depth of field of the image region corresponding to the fitted point in the captured image. Similarly, the second depth can be understood as the depth of field of the image region corresponding to the object detection box in the captured image.

[0040] It should be understood that in images, depth of field is often represented by different colors; the darker the color, the farther the depth of field. Therefore, the first and second depths can be determined by the colors of corresponding image regions.

[0041] In step 150, for each fitted point, it is determined whether the fitted point is an idle point based on the first depth and the second depth corresponding to the fitted point.

[0042] Here, after obtaining the second depth corresponding to each object detection box and the first depth corresponding to each fitted point, for each fitted point, the first depth corresponding to the fitted point can be compared with the second depth corresponding to the object detection box to determine whether the fitted point is an idle point. For example, the first depth corresponding to the fitted point can be compared with the second depth corresponding to any object detection box to determine whether the fitted point is an idle point. An idle point refers to a fitted point that does not contain a target object and is in an idle state. It should be understood that since the image region corresponding to the object detection box already contains the target object, the second depth corresponding to the object detection box is actually a depth indicating the presence of a target object. Therefore, by comparing the first depth and the second depth, it can be determined whether a target object exists in the fitted point. If no target object exists, the fitted point is an idle point; if a target object exists, the fitted point is not an idle point.

[0043] For example, in a captured image, the depth of field of an image area containing a target object will be less than the depth of field of an image area where no target object exists. Therefore, by comparing the first depth with the second depth, it is possible to accurately determine whether a fitted point is an idle point.

[0044] It is worth noting that the vacant location identification method provided in this disclosure can be used to detect vacant parking spaces in parking lots, vacant warehouse locations on shelves, vacant pallets on factory production lines, and vacant material troughs in warehouse storage troughs.

[0045] Therefore, by acquiring the captured image corresponding to the target scene, performing target object detection on the captured image, and obtaining the object detection box corresponding to the target object included in the captured image, the object detection box is then fitted in the captured image to obtain the fitted points. Based on the captured image, a first depth corresponding to the fitted point and a second depth corresponding to the object detection box are determined. Then, based on the first and second depths corresponding to the fitted point, it is determined whether the fitted point belongs to an empty point. Detecting whether a fitted point belongs to an empty point by using depth can avoid the influence of changes in lighting, shadow interference, and appearance differences. Even under complex environmental conditions, it can still accurately and stably detect empty points, exhibiting good robustness and versatility. For example, the empty point recognition method provided in this disclosure embodiment can be compatible with application scenarios with different appearances of vehicles, goods, and placement differences, achieving more universal empty space recognition. It can be flexibly applied to the detection of empty spaces in parking lots, shelf locations, factory pallets, and warehouse storage troughs.

[0046] It is worth noting that, in this embodiment of the disclosure, after detecting the point status (idle or occupied) corresponding to each fitted point, the point status corresponding to each fitted point can be rendered on the captured image to indicate idle points. Of course, information such as the point coordinates, confidence level, and timestamp corresponding to each fitted point can also be output.

[0047] In some feasible implementations, in step 130, the center point of the object detection box in the captured image can be fitted based on the target line corresponding to the target scene to obtain at least one fitted line; for each fitted line, sampling points are created on the fitted line based on a preset target step size; for each sampling point, the fitting point position corresponding to the sampling point is obtained based on the size of the object detection box and the sampling point.

[0048] Here, the target line corresponding to the target scene can be determined based on the arrangement of the points used to store the target object within the scene. Taking a shelf as an example, if the points on the shelf are arranged in straight lines, the target line can be a straight line; if the points are arranged in curves (meaning the shelf is curved), the target line can be a curve. Similarly, with a parking lot, if the parking spaces are arranged in straight lines, the target line is a straight line; if the parking spaces are arranged in arcs or other curves, the target line is a curve. Of course, the target line can also be determined by combining the captured image of the target scene. For example, when the captured image has perspective distortion, the target line can be a spline curve. Even if the target line is determined to be a straight line based on the arrangement of the points used to store the target object within the scene, it can still be a spline curve if the captured image has perspective distortion.

[0049] In other words, in this embodiment of the disclosure, the target lines used for point-to-point fitting can be different for different target scenarios. It should be understood that the target lines corresponding to the target scenario can be determined automatically by the electronic device or input by the user. For example, the user can select the target lines to use based on the different target scenarios.

[0050] By fitting the center points of all object detection boxes in the captured image using the target line, at least one fitted line is obtained. Specifically, if the target line is a straight line, the resulting fitted line is a straight line; if the target line is a curve, the resulting fitted line is a curve.

[0051] Figure 3 This is a schematic diagram of the fitted lines shown according to some embodiments. For example... Figure 3 As shown, by fitting the center points corresponding to the first object detection box 201, the second object detection box 202, the third object detection box 203, the fourth object detection box 204, the fifth object detection box 205, the sixth object detection box 206, and the seventh object detection box 207 in the captured image 200 through the target lines, the first fitted line 301 and the second fitted line 302 can be obtained.

[0052] In some embodiments, at least one fitted line can be obtained by fitting the center point of the object detection box in the captured image based on the target line using the Random Sample Consensus (RANSAC) algorithm.

[0053] The Random Sample Consensus (RSC) algorithm is a method for estimating mathematical model parameters from a sample set containing outliers. The RSC algorithm randomly selects two center points and, combined with the target line, calculates a candidate line. It then counts the interior points corresponding to this candidate line. After repeated iterations, the RSC algorithm selects the candidate line with the most interior points as the best model. Finally, it performs a least-squares operation using the candidate line and its corresponding interior points to obtain the final fitted line. An interior point is a center point whose distance from the candidate line is less than a residual threshold. In this embodiment, the residual threshold can be a preset multiple of the pixel scale. For example, the preset multiple can be 0.5 to 1.5 times.

[0054] Therefore, outliers can be identified iteratively using a random sample consensus algorithm. For example... Figure 3 As shown, the center point of the third object detection box 203 is actually an outlier. By fitting the data using the random sample consensus algorithm, the interference of the center point of the third object detection box 203 can be eliminated, and the correct fitted line can be obtained.

[0055] In other words, the Random Sample Consensus (RSC) algorithm can independently fit the implicit network structure of an image along both row and column directions. The RSC algorithm ultimately outputs the fitted line with the highest confidence level. For example, ... Figure 3 As shown, the Random Sample Consensus (RSC) algorithm fits the image 200 in both column and row directions, but the first and second fitted lines 301 and 302 have the highest confidence levels. Therefore, by using the RSC algorithm, the hidden network structure in the captured image can be reconstructed, and fitted lines that conform to the hidden network structure of the captured image can be obtained.

[0056] After obtaining the fitted lines, for each fitted line, sampling points can be created along the fitted line using a preset target step size. That is, a sampling point is created on the fitted line at preset target step sizes. It's worth noting that the fitted lines will have the center point of the object detection box. Therefore, a sampling point can be created at a position at a target step size interval from the center point, using that center point as a reference. Then, the next sampling point can be created using that created sampling point as a reference.

[0057] For example, for each fitted line, sampling points are created on the fitted line by sampling at equal intervals along the arc length direction of the fitted line based on the target step size.

[0058] When the fitted line is a straight line, sampling points at equal intervals along the fitted line with the target step size will generally not cause any problems. However, if the fitted line is a curve (such as a spline curve), directly performing equal-interval sampling will lead to uneven distribution of sampling points. For example, spline curves are used for fitting when there is curvature in the shelving or parking lot floor, or when the captured image has perspective distortion.

[0059] Therefore, sampling points can be created on the fitted line by performing equal-interval sampling along the arc length direction of the fitted line, based on the target step size. In other words, sampling points are created on the fitted line by performing equal-interval sampling along the arc length direction of the fitted line, with the target step size as the interval, to ensure that the sampling points are evenly distributed.

[0060] It should be understood that, regardless of whether the fitted line is a straight line or a curve, sampling points can be created on the fitted line by sampling at equal intervals based on the target step size along the arc length direction of the fitted line.

[0061] In some embodiments, the projected distance between the center points of two adjacent object detection boxes in the captured image is determined, and then the mode of the projected distance is determined as the target step size.

[0062] Specifically, for the center point of each object detection box, the projected distance between the center points of all adjacent object detection boxes in the captured image can be calculated. This projected distance can refer to the pixel distance between the center points of two adjacent object detection boxes projected onto a reference coordinate line in the captured image.

[0063] After determining the projected distances in the image to the center points of all adjacent object detection boxes, the mode of all projected distances can be used as the target step size. In other words, the projected distance that appears most frequently among all projected distances is determined as the target step size.

[0064] It is worth noting that by determining the mode of the projected distance as the target step size, a small number of extremely large or small outliers can be ignored, resulting in better resistance to outliers and making the created sampling points more accurate.

[0065] Figure 4 This is a schematic diagram of sampling points shown according to some embodiments. For example... Figure 4 As shown, on the first fitting line 301, a first sampling point 401, a second sampling point 402, a third sampling point 403, and a fourth sampling point 404 can be created. On the second fitting line 302, a fifth sampling point 405 and a sixth sampling point 406 can be created.

[0066] After determining the sampling points, for each sampling point, the corresponding fitting point can be obtained based on the size of the object detection box and the sampling point itself. For example, the sampling point can be used as the center point of the fitting point, and the size of the object detection box can be used as the size of the fitting point to determine the corresponding fitting point.

[0067] It should be noted that the size of the object detection box refers to its size on the captured image. Since the number of detected object detection boxes may be large, and the size of each object detection box may vary, the mode of all object detection box sizes can be used to determine the size of the object detection box used to determine the fitted point.

[0068] Figure 5 This is a schematic diagram illustrating the fitting points according to some embodiments. For example... Figure 5 As shown, the first fitting point 501 corresponding to the first sampling point 401, the second fitting point 502 corresponding to the second sampling point 402, the third fitting point 503 corresponding to the third sampling point 403, the fourth fitting point 504 corresponding to the fourth sampling point 404, the fifth fitting point 505 corresponding to the fifth sampling point 405, and the sixth fitting point 506 corresponding to the sixth sampling point 406 can be obtained.

[0069] Therefore, through the above implementation method, even if some object detection boxes are detected in the captured image, all standard points (points corresponding to object detection boxes and fitted points) contained in the target scene can still be inferred. Even if there are missed detections, the network results of the target scene can be recovered, thereby extracting all points that can be used to store target objects. This provides an accurate data foundation for subsequent detection of idle points in the target scene, significantly improving the accuracy of idle point recognition. Moreover, even in the presence of perspective distortion and camera angle changes, fitted lines can still be accurately fitted from the captured image, greatly improving the applicability and automation of the idle point recognition method.

[0070] In some feasible implementations, in step 140, a depth image corresponding to the captured image can be obtained based on the captured image, and then a first depth corresponding to the fitted point is determined based on the first image region corresponding to the depth image, and a second depth corresponding to the object detection box is determined based on the second image region corresponding to the depth image.

[0071] Here, the captured image can be input into the depth estimation model to obtain a depth image output by the depth estimation model. For example, the depth estimation model can be DepthAnything (monocular depth estimation model). DepthAnything can convert ordinary captured images into depth images. It should be understood that the depth output by DepthAnything is a relative depth. Of course, in this embodiment of the disclosure, the depth estimation model can also be other types of depth estimation models besides DepthAnything, and this embodiment of the disclosure does not limit the specific type of depth estimation model.

[0072] It should be understood that since different depth estimation models have different near and far directions, when outputting depth images, the background or ground in the captured image can be normalized so that a larger depth value represents a farther depth of field, thus unifying the depth direction.

[0073] After acquiring the depth image, for each fitted point, the first depth corresponding to the fitted point can be determined based on the first image region corresponding to the fitted point in the depth image.

[0074] For example, the average depth of each pixel in the first image region can be determined as the first depth. Alternatively, the median depth of each pixel in the first image region can also be determined as the first depth.

[0075] It should be noted that in depth images, depth of field can be represented by different colors, with darker colors indicating greater depth. Correspondingly, the first depth corresponding to a fitted point can be determined based on the color intensity within the first image region. In other words, if the depth color of a fitted point is darker than the depth color of the object detection box containing the target object, it can be determined that no target object exists at the fitted point, and the fitted point is considered an empty point.

[0076] Similarly, for each object detection box, the second depth corresponding to the object detection box can be determined based on the second image region corresponding to the object detection box in the depth image.

[0077] For example, the average depth of each pixel in the second image region can be used to determine the second depth. Alternatively, the median depth of each pixel in the second image region can also be used to determine the second depth. It should be understood that the calculation methods for the second depth and the first depth can be consistent.

[0078] Therefore, through the above implementation method, the second depth corresponding to the object detection box and the first depth corresponding to the fitted point can be accurately obtained, providing accurate data support for the accurate detection of idle points.

[0079] In some feasible implementations, in step 150, a reference depth can be determined based on the second depth corresponding to the object detection box, and then, if the first depth corresponding to the fitted point is greater than the reference depth, the fitted point is determined to be an idle point.

[0080] Here, the reference depth is used to characterize the presence of a target object. That is, in a depth image, the image region corresponding to the reference depth can be understood as containing a target object, and the reference depth serves as a reference for detecting the presence of a target object. Therefore, by comparing the first depth corresponding to the fitted point with the reference depth, it can be determined whether a target object exists at the fitted point.

[0081] When the first depth corresponding to the fitted point is greater than the reference depth, it means that the depth of field of the fitted point is farther than the depth of field of the object detection box containing the target object. Therefore, there is no target object at this fitted point, and the fitted point can be determined to be an idle point. When the first depth corresponding to the fitted point is less than or equal to the reference depth, it means that the depth of field of the fitted point is closer than the depth of field of the object detection box containing the target object. Therefore, there is a target object at this fitted point, and the fitted point can be determined to be occupied.

[0082] like Figure 5As shown, assuming the second depth of the first object detection box 201 is the reference depth, when the first depth of the first fitting point 501 is greater than the second depth of the first object detection box 201, the first fitting point 501 is an idle point. When the first depth of the second fitting point 502 is less than the second depth of the first object detection box 201, the second fitting point 502 is an occupied point.

[0083] In some embodiments, the median of the second depths can be used as the baseline depth. By using the median of all second depths as the baseline depth, the impact of a few extremely large or small outliers can be reduced. For example, in a parking lot, if parked vehicle A is significantly taller than other vehicles, its depth of field will be shallower; if parked vehicle B is significantly shorter than other vehicles, its depth of field will be deeper. If the second depth of the object detection box corresponding to vehicle A or vehicle B is used as the baseline depth, it may cause originally idle fitted points to be misdetected as occupied, or originally occupied fitted points to be misdetected as idle fitted points.

[0084] In other embodiments, the second depths can be sorted in ascending order, and the second depths at a preset percentile (greater than 50%) can be determined as the base depth. Specifically, all second depths can be sorted from smallest to largest, and the second depths at a preset percentile can be determined as the base depth. For example, the preset percentile is greater than 50%. That is, in this embodiment, the second depth at the highest percentile can be determined as the base depth.

[0085] Therefore, by using the above implementation method to detect whether the fitted point is an idle point through depth comparison, the effects of changes in lighting, shadow interference and appearance differences can be avoided. Even under complex environmental conditions, it can still accurately and stably detect idle points, and has good robustness and versatility.

[0086] Figure 6 This is a schematic diagram of a vacant location identification device according to some embodiments. For example... Figure 6 As shown, this embodiment of the disclosure provides a vacant location identification device 600, which includes: The acquisition module 601 is configured to acquire the captured image corresponding to the target scene; The detection module 602 is configured to perform target object detection on the captured image to obtain an object detection box corresponding to the target object included in the captured image; The fitting module 603 is configured to fit the object detection box in the captured image to obtain fitting points, which are used to store the target object. The first determining module 604 is configured to determine a first depth corresponding to the fitting point and a second depth corresponding to the object detection box based on the captured image. The second determining module 605 is configured to determine whether a fitted point is an idle point based on a first depth and a second depth corresponding to each fitted point.

[0087] Optionally, the first determining module 604 is specifically configured as follows: Based on the captured image, a depth image corresponding to the captured image is obtained; Based on the fitted point in the first image region corresponding to the depth image, the first depth corresponding to the fitted point is determined. Based on the object detection box in the second image region corresponding to the depth image, the second depth corresponding to the object detection box is determined.

[0088] Optionally, the second determining module 605 is specifically configured as follows: Based on the second depth corresponding to the object detection box, a reference depth is determined, and the reference depth is used to characterize the presence of the target object; If the first depth corresponding to the fitted point is greater than the reference depth, the fitted point is determined to be an idle point.

[0089] Optionally, the second determining module 605 is specifically configured as follows: The median of the second depth is determined as the reference depth; or The second depth is sorted in ascending order, and the second depth that is at a preset percentile after sorting is determined as the reference depth, wherein the preset percentile is greater than 50%.

[0090] Optionally, the fitting module 603 is specifically configured as follows: Based on the target lines corresponding to the target scene, the center point of the object detection box is fitted in the captured image to obtain at least one fitted line. For each of the fitted lines, sampling points are created on the fitted lines based on a preset target step size; For each sampling point, the fitting point corresponding to the sampling point is obtained based on the size of the object detection box and the sampling point.

[0091] Optionally, the fitting module 603 is further configured to: Determine the projection distance between the center points of two adjacent object detection boxes in the captured image; The mode of the projected distance is determined as the target step size.

[0092] Optionally, the fitting module 603 is specifically configured as follows: For each of the fitted lines, sampling points are created on the fitted lines by performing equally spaced sampling along the arc length direction of the fitted lines based on the target step size.

[0093] The functional logic executed by each functional module in the aforementioned idle location identification device 600 has been described in detail in the section on methods, and will not be repeated here.

[0094] The following is for reference. Figure 7 The diagram illustrates a structural schematic of an electronic device (e.g., a terminal device or a server) 700 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0095] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0096] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0097] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0098] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0099] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0100] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0101] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: Acquire images corresponding to the target scene; Target object detection is performed on the captured image to obtain the object detection box corresponding to the target object included in the captured image; The object detection box is fitted in the captured image to obtain fitted points, which are used to store the target object. Based on the captured image, determine the first depth corresponding to the fitted point and the second depth corresponding to the object detection box; For each fitted point, it is determined whether the fitted point is an idle point based on the first depth and the second depth corresponding to the fitted point.

[0102] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0104] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0105] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0106] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0107] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0108] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0109] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. A method for identifying vacant locations, characterized in that, The method comprises: obtaining a photographed image corresponding to a target scene; performing target object detection on the photographed image to obtain an object detection frame corresponding to the target object included in the photographed image; fitting the object detection frame in the photographed image to obtain a fitted point position, the fitted point position being used to store the target object; determining a first depth corresponding to the fitted point position and determining a second depth corresponding to the object detection frame based on the photographed image; for each fitted point position, determining whether the fitted point position belongs to an idle point position according to the first depth corresponding to the fitted point position and the second depth.

2. The method of claim 1, wherein, The method comprises: obtaining a depth image corresponding to the photographed image based on the photographed image; determining the first depth corresponding to the fitted point position based on a first image region corresponding to the fitted point position in the depth image; determining the second depth corresponding to the object detection frame based on a second image region corresponding to the object detection frame in the depth image.

3. The method of claim 1, wherein, The method comprises: determining a reference depth based on the second depth corresponding to the object detection frame, the reference depth being used to represent the presence of the target object; in a case where the first depth corresponding to the fitted point position is greater than the reference depth, determining that the fitted point position belongs to an idle point position.

4. The method of claim 3, wherein, The method comprises: determining the median of the second depths as the reference depth; or sorting the second depths in ascending order, and determining the second depth at a preset percentile in the sorted second depths as the reference depth, the preset percentile being greater than 50%.

5. The method according to any one of claims 1-4, characterized in that, The method comprises: fitting a center point of the object detection frame in the photographed image based on a target line corresponding to the target scene to obtain at least one fitted line; for each fitted line, creating a sampling point on the fitted line based on a preset target step; for each sampling point, obtaining a fitted point position corresponding to the sampling point based on the size of the object detection frame and the sampling point.

6. The method of claim 5, wherein, The target step is determined by the following steps: determining a projection distance of the center points of two adjacent object detection frames in the photographed image; determining the mode in the projection distance as the target step.

7. The method of claim 5, wherein, The method comprises: for each fitted line, equally spacing sampling points on the fitted line in the arc length direction of the fitted line based on the target step to create sampling points on the fitted line.

8. A free point identification device, characterized by The method comprises: an obtaining module configured to obtain a photographed image corresponding to a target scene; a detection module configured to perform target object detection on the photographed image to obtain an object detection frame corresponding to the target object included in the photographed image; The fitting module is configured to fit the object detection box in the captured image to obtain fitting points, which are used to store the target object. The first determining module is configured to determine a first depth corresponding to the fitted point and a second depth corresponding to the object detection box based on the captured image. The second determining module is configured to determine whether a fitted point is an idle point based on a first depth and a second depth corresponding to each fitted point.

9. A computer readable medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processing device, it implements the steps of the method according to any one of claims 1-7.

10. An electronic device, comprising: include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-7.

11. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

Images