Parking space information detection method, electronic equipment and medium

Abstract

The invention relates to the field of vehicle parking control, in particular to a parking space information detection method, electronic equipment and a medium. The parking space information detection method comprises the following steps: acquiring a bird's-eye view, wherein the bird's-eye view is obtained by adopting a fisheye view; inputting the aerial view into a parking space detection model to obtain a region of interest of the fisheye image; the region of interest of the fisheye image comprises a parking space frame, angular points and auxiliary points; inputting the region of interest of the fisheye image into a semantic segmentation model, and identifying a target region; the target area comprises a parking space line area, a vehicle area and a limiter area; and determining parking space information according to the region of interest and the target region of the fisheye image. According to the method, parking space detection of the aerial view is combined with semantic segmentation of the region of interest of the fisheye image, so that the problems of inaccurate parking space detection of the aerial view and complex processing after segmentation of the fisheye image are effectively solved, the parking space detection accuracy and the position detection accuracy of the limiter are greatly improved, the detection speed is increased, and the detection efficiency is improved. And the collision probability of the limiting stopper is reduced.

Description

Technical Field

[0001] The present application relates to the field of vehicle parking control, and in particular to a parking space information detection method, electronic equipment and medium. Background Art

[0002] The accuracy of parking space detection during automatic parking directly affects the parking effect, resulting in undesirable consequences such as uneven body and vehicle crossing the line after parking. Secondly, inaccurate limiter recognition during parking not only damages vehicle tires but also seriously affects user experience and even poses certain safety hazards.

[0003] In the related technology, there are solutions for parking space recognition based on bird's-eye view, but the parking space recognition algorithm based on bird's-eye view has factors such as splicing misalignment and distortion that affect the accuracy of parking space detection, and the stretching of vehicles occupying parking spaces in the bird's-eye view will affect the detection and occupancy judgment of adjacent parking spaces; in addition, the stretching of the limiter makes it impossible to accurately obtain the position of the limiter, which leads to the phenomenon of collision with the limiter. The solution of parking space detection based on the surround fisheye image and the four-way fisheye image consumes too much resources, and the parking space construction process is complicated.

[0004] In view of this, this application is hereby filed. Summary of the invention

[0005] The purpose of the present application is to provide a parking space information detection method, electronic device and medium to solve the problems of poor parking space detection accuracy, inability to accurately obtain the position of the limiter and excessive resource consumption in the prior art.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions: In a first aspect, the present application provides a parking space information detection method, comprising: Obtaining a bird's-eye view, wherein the bird's-eye view is obtained by using a fisheye view; Input the bird's-eye view image into a parking space detection model to obtain an area of ​​interest of the fisheye image; the area of ​​interest of the fisheye image includes a parking space frame, corner points and auxiliary points; Inputting the region of interest of the fisheye image into a semantic segmentation model to identify a target region; the target region includes a parking space line region, a vehicle region, and a stopper region; Parking space information is determined according to the region of interest of the fisheye image and the target region.

[0007] In some technical solutions, the step of inputting the bird's-eye view into a parking space detection model to obtain the region of interest of the fisheye view includes: Input the bird's-eye view into the parking space detection model, and output parking space frame information; the parking space frame information includes a front corner point frame, an auxiliary point frame, an entrance wire frame and a parking space frame; Back-projecting the parking space frame information onto the fisheye image to obtain an initial area; According to the initial area and the area expansion ratio, the region of interest of the fisheye image is obtained.

[0008] In some technical solutions, the parking space information includes a parking space type, a parking space inclination angle, and a parking space occupancy status; and determining the parking space information according to the area of ​​interest and the target area of ​​the fisheye image includes: Determine the parking space type and the parking space inclination angle according to the region of interest of the fisheye image; The parking space occupancy status is determined according to the region of interest of the fisheye image and the target region.

[0009] In some technical solutions, determining the parking space type and the parking space inclination angle according to the area of ​​interest of the fisheye image includes: Determine the parking space type according to the type of the entrance wireframe in the parking space frame; Determining a reference parking space inclination angle according to the corner point and the auxiliary point; When the parking space type is a vertical parking space and the reference parking space inclination angle is a first angle, the parking space inclination angle is determined to be 90°; the first angle is 90°±angle tolerance; When the parking space type is a vertical parking space and the reference parking space inclination angle is not the first angle, or when the parking space type is an inclined parking space, the parking space inclination angle is determined according to the parking space line area and the reference parking space inclination angle.

[0010] In some technical solutions, determining the parking space inclination angle according to the parking space line area and the reference parking space inclination angle includes: According to the reference parking space inclination angle, a target parking space line is selected from the parking space line area; the deviation between the inclination angle of the target parking space line and the inclination angle of the reference parking space is within a preset deviation range; The parking space inclination angle is determined according to the inclination angle of the target parking space line and the reference parking space inclination angle.

[0011] In some technical solutions, after determining the parking space type and the parking space inclination angle according to the area of ​​interest of the fisheye image, the method further includes: Determining the initial position of the rear corner point of the parking space according to the entrance parking space frame in the parking space frame and the parking space inclination angle; The rear corner point of the parking space is determined according to the rear edge area in the parking space line area and the initial position of the rear corner point of the parking space.

[0012] In some technical solutions, after determining the rear corner point of the parking space according to the rear edge area in the parking space line area and the initial position of the rear corner point of the parking space, the method further includes: When the length-to-width ratio of the parking space frame does not meet the length-to-width ratio standard of the parking space, adjusting the position of the rear corner point of the parking space; or, in the case where the parking space type is an inclined parking space, adjusting the inclination angle of the parking space frame according to the inclination angle of the parking space and the inclination angle of an adjacent parking space; Or, in the case where there is a deviation between the parking space line area and the edge of the parking space frame, the parking space line area is adjusted so that the parking space line area fits the edge of the parking space frame.

[0013] In some technical solutions, after the region of interest of the fisheye image is input into the semantic segmentation model and the target region is identified, the method further includes: Determine the parking safety distance based on the distance between the rear axle and the front of the vehicle and the preset safety distance; The stopper grounding point is determined according to the stopper area and the entrance line of the parking space line area; when the distance between the stopper grounding point and the entrance line of the parking space line area is greater than the parking safety distance, the stopper position is adjusted forward.

[0014] In a second aspect, the present application provides an electronic device, including: at least one processor, and a memory communicatively coupled to at least one of the processors; The memory stores instructions that can be executed by at least one of the processors, and the instructions are executed by at least one of the processors so that at least one of the processors can perform the above method.

[0015] In a third aspect, the present application provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the above method.

[0016] Compared with the prior art, the beneficial effects of this application are: The parking space information detection method provided by the present application includes obtaining a bird's-eye view, wherein the bird's-eye view is obtained by using a fisheye view; inputting the bird's-eye view into a parking space detection model to obtain an area of ​​interest of the fisheye view; the area of ​​interest of the fisheye view includes a parking space frame, corner points, and auxiliary points; inputting the area of ​​interest of the fisheye view into a semantic segmentation model to identify a target area; the target area includes a parking space line area, a vehicle area, and a limiter area; and determining parking space information based on the area of ​​interest of the fisheye view and the target area. This method combines parking space detection using a bird's-eye view with semantic segmentation of the area of ​​interest of a fisheye view, effectively solving the problems of inaccurate parking space detection using a bird's-eye view and complex post-processing of fisheye view segmentation, greatly improving the accuracy of parking space detection and limiter position detection, and speeding up the detection speed, reducing the probability of limiters being hit, and improving the driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flow chart of the parking space information detection method provided by this application; Figure 2 It is a schematic diagram of a partial area structure of an area of ​​interest of a fisheye image in the fisheye image provided by the present application; Figure 3 It is a structural schematic diagram of the parking space information detection device provided by the present application; Figure 4 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0019] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0020] As mentioned in the background technology, the existing technology has problems such as poor parking space detection accuracy, inability to accurately obtain the position of the limiter, and excessive resource consumption. In this regard, the present application combines the parking space detection of the bird's-eye view with the semantic segmentation of the region of interest of the fisheye view, effectively solving the problems of inaccurate parking space detection of the bird's-eye view and complex post-segmentation processing of the fisheye view. The present application is further described in detail below in conjunction with the embodiments.

[0021] Example 1 Figure 1 This is a flow chart of a parking space information detection method provided by this embodiment. This embodiment is applicable to parking space information detection when a vehicle intends to park. This method can be executed by a parking space information detection device, which can be composed of software and / or hardware and is generally integrated in an electronic device, which can be a vehicle computer or other mobile terminal (such as a smart phone, tablet computer, etc.). For ease of understanding, each step in the control method of this embodiment is executed by the vehicle computer.

[0022] like Figure 1 As shown, this embodiment provides a parking space information detection method, comprising the following steps: S110: Acquire a bird's-eye view, where the bird's-eye view is obtained using a fisheye view.

[0023] The fisheye image is an image captured by a fisheye camera. Optionally, the fisheye image includes four-way fisheye images of the front, rear, left and right of the vehicle.

[0024] Optionally, the bird's-eye view image is obtained in the following manner: the bird's-eye view image is determined according to a mapping relationship between the fish-eye image and the bird's-eye view image and the fish-eye image.

[0025] The mapping relationship between the fisheye image and the bird's-eye view image can be pre-calibrated. The following is an optional calibration method: a. Dedistort the fisheye image based on the camera intrinsic parameters, use the corner detection algorithm and sub-pixel calibration algorithm to obtain the pixel coordinates (u, v) of the corner points of the modified calibration plate, and measure the world coordinates (x, y, z) of the checkerboard corner points in the real world coordinate system; b. Based on the pixel coordinates of the corner points of the calibration plate and the corresponding world coordinates, use the extrinsic parameter estimation algorithm to obtain the extrinsic parameters of the fisheye camera; c. Determine the size and field of view of the bird's-eye view and construct a mapping coordinate matrix; d. Use the corresponding fisheye camera extrinsic parameters according to different areas of the bird's-eye view to obtain the dedistorted fisheye camera coordinates corresponding to the bird's-eye view coordinates; e. Use the fisheye camera intrinsic parameters to calculate the original fisheye camera coordinates, and then determine the mapping table between the fisheye image and the bird's-eye view.

[0026] In this step, a bird's-eye view can be obtained based on the above mapping relationship and the fisheye image actually collected. For example, when the fisheye image is a four-way fisheye image, the bird's-eye view obtained from each fisheye image can be spliced ​​together in combination with an image splicing and fusion algorithm to obtain a complete bird's-eye view.

[0027] S120, inputting the bird's-eye view into a parking space detection model to obtain a region of interest of the fisheye view; the region of interest of the fisheye view includes a parking space frame, corner points, and auxiliary points.

[0028] In an optional implementation, the step of inputting the bird's-eye view into a parking space detection model to obtain the region of interest of the fisheye view includes: Input the bird's-eye view into the parking space detection model, and output parking space frame information; the parking space frame information includes a front corner point frame, an auxiliary point frame, an entrance wire frame and a parking space frame; Back-projecting the parking space frame information onto the fisheye image to obtain an initial area; The region of interest of the fisheye image is obtained according to the initial region and the region expansion ratio.

[0029] The above parking space detection model refers to a model that can detect parking spaces based on an input bird's-eye view. For example, a trained YoloV5n model can be used as a parking space detection model. The area expansion ratio refers to a preset ratio of expanding the initial area, for example, 10%.

[0030] Parking spaces are generally parallelograms formed by four vertices, but the rear corner points of parking spaces are usually invisible; for this reason, the data only annotates the front corner point frame, auxiliary point frame, entrance wireframe and parking space frame of the parking space, among which the entrance wireframe includes a vertical entrance wireframe and an inclined entrance wireframe. The parking space frame information includes the front corner point frame, auxiliary point frame, entrance wireframe and parking space frame. After obtaining the above parking space frame information, the parking space frame information is back-projected onto the fisheye image to obtain the corresponding initial area on the fisheye image (which can be understood as the area of ​​interest of the initial fisheye image). In order to ensure the integrity of the area, the initial area is appropriately expanded to obtain the area of ​​interest of the fisheye image.

[0031] Among them, Figure 2 As shown, the front corner point box contains the front corner point of the parking space, the auxiliary point box contains the auxiliary points of the parking space (i.e., points located in the extension direction of the parking space for assisting in determining the inclination angle of the parking space), the entrance line box contains the entrance line of the parking space, and the parking space box contains the parking space.

[0032] Optionally, the size of the bird's-eye view input to the parking space detection model is 320x320 pixels.

[0033] Optionally, data enhancement methods such as random image flipping, random adjustment of image saturation and exposure are used to improve the performance of the parking space detection model.

[0034] S130, inputting the region of interest of the fisheye image into a semantic segmentation model to identify a target region; the target region includes a parking line region, a vehicle region, and a limiter region.

[0035] Since the region of interest of the above-mentioned fisheye image is located in the fisheye image, compared with the method of using a bird's-eye view for semantic segmentation, inputting the region of interest of the fisheye image into the semantic segmentation model can improve the accuracy of target area recognition. In this embodiment, the region of interest of the fisheye image is input into the semantic segmentation model instead of the entire fisheye image, thereby improving recognition efficiency.

[0036] Optionally, in order to further improve the training effect of the semantic segmentation model, data enhancement methods such as randomly changing the brightness and contrast of the image, randomly rotating and randomly adding noise are used to scale the region of interest of the fisheye image to 128ⅹ96 pixels as the model input.

[0037] Optionally, in addition to the above parking line area, vehicle area and limiter area, the semantic segmentation model can also identify the background area, that is, other areas in the area of ​​interest of the fisheye image except the above parking line area, vehicle area and limiter area.

[0038] S140: Determine parking space information according to the region of interest of the fisheye image and the target region.

[0039] In an optional implementation manner, the parking space information includes a parking space type, a parking space inclination angle, and a parking space occupancy status; and determining the parking space information according to the region of interest and the target region of the fisheye image includes: Determine the parking space type and the parking space inclination angle according to the region of interest of the fisheye image; The parking space occupancy status is determined according to the region of interest of the fisheye image and the target region.

[0040] The parking space information in this embodiment includes parking space type, parking space inclination angle and parking space occupancy status, wherein the parking space type includes vertical parking space and inclined parking space, the parking space inclination angle refers to the inclination angle of the parking space relative to the road direction, and the parking space occupancy status refers to whether the parking space is occupied.

[0041] In an optional implementation manner, determining the parking space type and the parking space inclination angle according to the region of interest of the fisheye image includes: Determine the parking space type according to the type of the entrance wireframe in the parking space frame; Determining a reference parking space inclination angle according to the corner point and the auxiliary point; When the parking space type is a vertical parking space and the reference parking space inclination angle is a first angle, the parking space inclination angle is determined to be 90°; the first angle is 90°±angle tolerance; When the parking space type is a vertical parking space and the reference parking space inclination angle is not the first angle, or when the parking space type is an inclined parking space, the parking space inclination angle is determined according to the parking space line area and the reference parking space inclination angle.

[0042] As mentioned above, the entrance wireframe includes a vertical entrance wireframe and an inclined entrance wireframe. If the entrance wireframe type is a vertical entrance wireframe, the parking space type is a vertical parking space. If the entrance wireframe type is an inclined entrance wireframe, the parking space type is an inclined parking space.

[0043] The reference parking space inclination angle can also be understood as the initial parking space inclination angle, which refers to the initial parking space inclination angle calculated based on the corner point and the auxiliary point. Optionally, determining the reference parking space inclination angle according to the corner point and the auxiliary point includes: determining the reference parking space inclination angle according to the coordinates of the corner point (x1, y1) and the coordinates of the auxiliary point (x2, y2). The calculation formula can be: For example, when the coordinates of the corner point are (10, 20) and the coordinates of the auxiliary point are (11, 25), the reference parking space inclination angle is .

[0044] If the parking space type is a vertical parking space and the reference parking space inclination angle is the first angle, the parking space inclination angle is determined to be 90°, and the first angle is 90°±angle tolerance, and the angle tolerance is, for example, 1~5°. In this case, it means that the reference parking space inclination angle is slightly different from 90°, so it can be considered that the inclination angle of the parking space is 90°.

[0045] If the parking space type is a vertical parking space, and the reference parking space tilt angle is not the first angle, it means that the reference parking space tilt angle is much different from 90°. In this case, the parking space tilt angle needs to be determined by combining the parking space line area and the reference parking space tilt angle. Similarly, if the parking space type is an inclined parking space, the parking space tilt angle also needs to be determined by combining the parking space line area and the reference parking space tilt angle.

[0046] In an optional implementation manner, determining the parking space inclination angle according to the parking space line area and the reference parking space inclination angle includes: According to the reference parking space inclination angle, a target parking space line is selected from the parking space line area; the deviation between the inclination angle of the target parking space line and the inclination angle of the reference parking space is within a preset deviation range; The parking space inclination angle is determined according to the inclination angle of the target parking space line and the reference parking space inclination angle.

[0047] The preset deviation range may be ±5° or ±10°. In this embodiment, the target parking space line is first selected from the parking space line area according to the reference parking space inclination angle, and then the parking space inclination angle is determined by combining the inclination angle of the target parking space line and the reference parking space inclination angle, so that the obtained parking space inclination angle is more accurate.

[0048] Optionally, after the target parking space line is screened, the target parking space line can also be subjected to noise elimination and morphological processing, such as morphological operations such as dilation and erosion to smooth the line edge or median filtering to filter out isolated pixels, which helps to ensure that the line is smoother and more regular and reduce errors; and the best straight line can be obtained by using fitting algorithms such as the least squares method for the screened and processed parking space lines.

[0049] Optionally, a weighted average method is used to process the inclination angle of the target parking space line and the reference parking space inclination angle to obtain the parking space inclination angle.

[0050] In an optional implementation manner, after determining the parking space type and the parking space inclination angle according to the area of ​​interest of the fisheye image, the method further includes: Determining the initial position of the rear corner point of the parking space according to the entrance parking space frame in the parking space frame and the parking space inclination angle; The rear corner point of the parking space is determined according to the rear edge area in the parking space line area and the initial position of the rear corner point of the parking space.

[0051] Through this implementation, the accurate position of the rear corner point can be further obtained. This method first determines the initial position of the rear corner point of the parking space based on the entrance parking space frame and the parking space inclination angle in the parking space frame, and then determines the rear corner point of the parking space based on the rear edge area in the parking space line area after semantic segmentation and the initial position of the rear corner point of the parking space.

[0052] Optionally, the rear corner point may be determined by vertically projecting the direction of the entrance parking space frame and the rear edge area, and finally aligned with the inclination angle of the entrance line of the parking space to ensure that the rear corner point conforms to the overall direction of the parking space.

[0053] In an optional implementation, after determining the rear corner point of the parking space according to the rear edge area in the parking space line area and the initial position of the rear corner point of the parking space, the method further includes: When the length-to-width ratio of the parking space frame does not meet the length-to-width ratio standard of the parking space, adjusting the position of the rear corner point of the parking space; or, in the case where the parking space type is an inclined parking space, adjusting the inclination angle of the parking space frame according to the inclination angle of the parking space and the inclination angle of an adjacent parking space; Or, in the case where there is a deviation between the parking space line area and the edge of the parking space frame, the parking space line area is adjusted so that the parking space line area fits the edge of the parking space frame.

[0054] After obtaining the rear corner point of the parking space, this implementation can further improve the accuracy of the rear corner point position, the accuracy of the tilt angle of the parking space frame, and the fit between the parking space line and the edge of the parking space frame, thereby further ensuring the accuracy of the parking space frame. If the calculated length-to-width ratio of the parking space frame does not conform to the parking space length-to-width ratio standard (i.e., the length-to-width ratio of the standard parking space), the extension position of the rear corner point can be further fine-tuned; the tilted parking spaces are usually arranged neatly, and the tilt angles of adjacent parking spaces are similar. Therefore, the tilt angle of the current parking space frame can be determined based on the tilt angles of adjacent parking spaces; if a deviation is detected between the parking space line area and the edge of the parking space frame, the parking space line and the edge of the parking space frame can be made to fit more closely through morphological operations or boundary adjustments.

[0055] In an optional implementation, after the region of interest of the fisheye image is input into the semantic segmentation model to identify the target region, the method further includes: Determine the parking safety distance based on the distance between the rear axle and the front of the vehicle and the preset safety distance; The stopper grounding point is determined according to the stopper area and the entrance line of the parking space line area; when the distance between the stopper grounding point and the entrance line of the parking space line area is greater than the parking safety distance, the stopper position is adjusted forward.

[0056] In step S130 of this embodiment, a reliable limiter area has been obtained, which helps the driver to accurately determine the limiter position and prevent the limiter from being hit during parking. The rear wheels of the vehicle need not touch the limiter during parking. If the parking safety distance is determined only by the distance between the rear axle and the front of the vehicle, the limiter may be hit. Therefore, this embodiment further improves the collision resistance by jointly determining the parking safety distance by using the distance between the rear axle and the front of the vehicle and the preset safety distance. Among them, the preset safety distance is the distance between the center of the rear wheel and the grounding point of the limiter after the vehicle is parked.

[0057] This embodiment determines the limiter grounding point based on the limiter area and the entrance line of the parking space line area. Optionally, the minimum circumscribed rectangle of the limiter area is first calculated, and the midpoint of the minimum circumscribed rectangle edge that is closest to the entrance line is regarded as the limiter grounding point. Experience has shown that the distance between the limiter and the entrance line must be greater than 3 meters. To reduce misidentification, this embodiment can filter out the limiter grounding point within 2.5 meters from the parking space entrance line. When the distance between the determined limiter grounding point and the entrance line is greater than the parking safety distance, it means that the limiter area is incorrectly identified, so the limiter in the figure needs to be advanced by a corresponding distance to improve the accuracy of the limiter position in the figure.

[0058] The above-mentioned parking space information detection method includes obtaining a bird's-eye view, which is obtained by using a fisheye view; inputting the bird's-eye view into a parking space detection model to obtain the region of interest of the fisheye view; the region of interest of the fisheye view includes a parking space frame, corner points and auxiliary points; inputting the region of interest of the fisheye view into a semantic segmentation model to identify a target area; the target area includes a parking space line area, a vehicle area and a limiter area; and determining parking space information based on the region of interest of the fisheye view and the target area. This method combines parking space detection using a bird's-eye view with semantic segmentation of the region of interest of a fisheye view, effectively solving the problems of inaccurate parking space detection using a bird's-eye view and complex post-segmentation processing of fisheye views, greatly improving the accuracy of parking space detection and limiter position detection, and speeding up the detection speed, reducing the probability of limiters being hit, and improving the driving experience.

[0059] Example 2 like Figure 3 As shown, this embodiment provides a parking space information detection device, including: A bird's-eye view acquisition module 201 is used to acquire a bird's-eye view, where the bird's-eye view is obtained using a fisheye view; The fisheye image region of interest output module 202 is used to input the bird's-eye view image into the parking space detection model to obtain the region of interest of the fisheye image; the region of interest of the fisheye image includes a parking space frame, corner points and auxiliary points; The target area recognition module 203 is used to input the region of interest of the fisheye image into the semantic segmentation model to identify the target area; the target area includes the parking line area, the vehicle area and the stopper area; The parking space information determination module 204 is used to determine the parking space information direction according to the area of ​​interest of the fisheye image and the target area.

[0060] The device is used to execute the above method, and thus has at least functional modules and beneficial effects corresponding to the above method.

[0061] Example 3 like Figure 4 As shown, this embodiment provides an electronic device, including: at least one processor; and a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions executable by at least one of the processors, and the instructions are executed by at least one of the processors to enable at least one of the processors to perform the above method. At least one processor in the electronic device can perform the above method, and thus has at least the same advantages as the above method.

[0062] Optionally, the electronic device also includes interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on a memory to display graphical information of a GUI (Graphical User Interface) on an external input / output device (such as a display device coupled to an interface). In other embodiments, if necessary, multiple processors can be used together with multiple memories, and / or multiple buses can be used together with multiple memories. Similarly, multiple electronic devices can be connected (for example, as a server array, a group of blade servers, or a multi-processor system), and each device provides some necessary operations. Figure 4 A processor 301 is taken as an example.

[0063] The memory 302, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the parking space information detection method in the embodiment of the present application (for example, the bird's-eye view acquisition module, the fisheye image region of interest output module, the target area recognition module, and the parking space information determination module in the parking space information detection device). The processor 301 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 302, that is, realizes the above-mentioned parking space information detection method.

[0064] The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 302 may further include a memory remotely arranged relative to the processor 301, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0065] The electronic device may further include: an input device 303 and an output device 304. The processor 301, the memory 302, the input device 303 and the output device 304 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0066] The input device 303 can receive input digital or character information, and the output device 304 can include a display device, an auxiliary lighting device (e.g., LED), and a tactile feedback device (e.g., a vibration motor), etc. The display device can include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device can be a touch screen.

[0067] Example 4 This embodiment provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the above method. The computer instructions on the computer-readable storage medium are used to enable a computer to execute the above method, and thus have at least the same advantages as the above method.

[0068] The medium in this application may adopt any combination of one or more computer-readable media. The medium may be a computer-readable signal medium or a computer-readable storage medium. The medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of the medium (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, the medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device, or device.

[0069] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0070] The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF (Radio Frequency), etc., or any suitable combination of the above.

[0071] Computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0072] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0073] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A parking space information detection method, characterized in that: include: Obtaining a bird's-eye view, wherein the bird's-eye view is obtained by using a fisheye view; Inputting the bird's-eye view into a parking space detection model to obtain an area of ​​interest of the fisheye view; The region of interest of the fisheye image includes a parking space frame, corner points and auxiliary points; Inputting the region of interest of the fisheye image into a semantic segmentation model to identify a target region; the target region includes a parking space line region, a vehicle region, and a stopper region; Parking space information is determined according to the region of interest of the fisheye image and the target region.

2. The parking space information detection method according to claim 1, characterized in that: The step of inputting the bird's-eye view image into the parking space detection model to obtain the region of interest of the fisheye image includes: Input the bird's-eye view into the parking space detection model, and output parking space frame information; the parking space frame information includes a front corner point frame, an auxiliary point frame, an entrance wire frame and a parking space frame; Back-projecting the parking space frame information onto the fisheye image to obtain an initial area; The region of interest of the fisheye image is obtained according to the initial region and the region expansion ratio.

3. The parking space information detection method according to claim 1, characterized in that: The parking space information includes a parking space type, a parking space inclination angle, and a parking space occupancy status; the determining the parking space information according to the area of ​​interest and the target area of ​​the fisheye image includes: Determine the parking space type and the parking space inclination angle according to the region of interest of the fisheye image; The parking space occupancy status is determined according to the region of interest of the fisheye image and the target region.

4. The parking space information detection method according to claim 3, characterized in that: The determining of the parking space type and the parking space inclination angle according to the region of interest of the fisheye image includes: Determine the parking space type according to the type of the entrance wireframe in the parking space frame; Determining a reference parking space inclination angle according to the corner point and the auxiliary point; When the parking space type is a vertical parking space and the reference parking space inclination angle is a first angle, the parking space inclination angle is determined to be 90°; the first angle is 90°±angle tolerance; When the parking space type is a vertical parking space and the reference parking space inclination angle is not the first angle, or when the parking space type is an inclined parking space, the parking space inclination angle is determined according to the parking space line area and the reference parking space inclination angle.

5. The parking space information detection method according to claim 4, characterized in that: The determining of the parking space inclination angle according to the parking space line area and the reference parking space inclination angle comprises: According to the reference parking space inclination angle, a target parking space line is selected from the parking space line area; the deviation between the inclination angle of the target parking space line and the inclination angle of the reference parking space is within a preset deviation range; The parking space inclination angle is determined according to the inclination angle of the target parking space line and the reference parking space inclination angle.

6. The parking space information detection method according to claim 3, characterized in that: After determining the parking space type and the parking space inclination angle according to the area of ​​interest of the fisheye image, the method further includes: Determining the initial position of the rear corner point of the parking space according to the entrance parking space frame in the parking space frame and the parking space inclination angle; The rear corner point of the parking space is determined according to the rear edge area in the parking space line area and the initial position of the rear corner point of the parking space.

7. The parking space information detection method according to claim 6, characterized in that: After determining the rear corner point of the parking space according to the rear edge area in the parking space line area and the initial position of the rear corner point of the parking space, the method further includes: When the length-to-width ratio of the parking space frame does not meet the length-to-width ratio standard of the parking space, adjusting the position of the rear corner point of the parking space; or, in the case where the parking space type is an inclined parking space, adjusting the inclination angle of the parking space frame according to the inclination angle of the parking space and the inclination angle of an adjacent parking space; Or, in the case where there is a deviation between the parking space line area and the edge of the parking space frame, the parking space line area is adjusted so that the parking space line area fits the edge of the parking space frame.

8. The parking space information detection method according to claim 1, characterized in that: After the region of interest of the fisheye image is input into the semantic segmentation model to identify the target region, the method further includes: Determine the parking safety distance based on the distance between the rear axle and the front of the vehicle and the preset safety distance; The stopper grounding point is determined according to the stopper area and the entrance line of the parking space line area; when the distance between the stopper grounding point and the entrance line of the parking space line area is greater than the parking safety distance, the stopper position is adjusted forward.

9. An electronic device, characterized in that: include: at least one processor, and a memory communicatively coupled to at least one of the processors; The memory stores instructions executable by at least one of the processors, and the instructions are executed by at least one of the processors so that the at least one processor can execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 8.

Images