Method, computer program product, parking assistance system, vehicle, and system for operating a parking assistance

Abstract

(Method for operating a parking assistance system for a vehicle, computer program product, parking assistance system, vehicle, and system) A method is proposed for operating a parking assistance system (110) for a vehicle (100), the method comprising: a) receiving (S1) an optical image (IMG) of the surroundings of the vehicle (100) and a digital surroundings map (MAP) representing the surroundings of the vehicle (100), the digital surroundings map (MAP) including at least one designated parking area (P1-P7); b) for each of at least one of the designated parking areas (P1-P7), selecting (S2) a received region of interest (ROI1-ROI7) in the optical image (IMG) corresponding to the respective designated parking area (P1-P7); c) detecting lines (PAT) and / or symbols (SYM1 to SYM3) on the ground in each of the selected regions of interest (ROI1 to ROI7); d) assigning one of a plurality of parking space classes (CLS) to each selected region of interest (ROI1-ROI7) based on the detected lines (PAT) and / or symbols (SYM1-SYM3) (S4); e) for each of at least one of the designated parking areas (P1 to P7), determining whether each of the designated parking areas (P1 to P7) corresponding to the selected region of interest (ROI1 to ROI7) is a parking space candidate based on the parking space class (CLS) assigned to the corresponding region of interest (ROI1 to ROI7); Equipped with.

Description

Technical Field

[0001] The present invention relates to a method for operating a parking assistance system of a vehicle, a computer program product, a parking assistance system for a vehicle, a vehicle equipped with the parking assistance system, and a system including the vehicle and an external unit.

Background Art

[0002] Parking assistance systems that use camera images to detect parking spaces around a vehicle are known. For example, such a system detects and analyzes lines drawn on the ground for detecting parking spaces.

[0003] There are increasing numbers of parking spaces for specific purposes. A parking space for a specific purpose is, for example, a parking space dedicated to a specific group of people and vehicles, such as a parking space for disabled persons, families with children, women, emergency situations, electric vehicles, etc. These parking spaces are often designated by symbols, letters, etc. To avoid parking violation tickets and other more serious problems, the driver of the vehicle needs to understand the meaning of all kinds of symbols, which can be complicated. This is because the use of symbols can vary from country to country, state to state, or region to region.

[0004] DE 10 2017 007 823 A1 discloses that a surrounding image is taken by an in-vehicle camera, and a special parking space is detected based on the surrounding image. Further, special equipment for disabled persons and / or a child seat arranged in the vehicle are detected, and the characteristics of at least one person riding in the vehicle are derived from the information. A special parking space corresponding to the derived characteristics is presented as a destination.

Summary of the Invention

[0005] One object of the present invention is to improve the operation of a parking assistance system of a vehicle.

[0006] According to a first aspect, a method for operating a parking assistance system for a vehicle is proposed. In a first step a), an optical image around the vehicle and a digital surrounding map representing the surroundings of the vehicle are received. The digital surrounding map includes at least one designated parking area. In a second step b), a region of interest corresponding to each of the designated parking areas is selected within the received optical image for each of the at least one designated parking areas. In a third step c), lines and / or symbols provided on the ground are detected in each selected region of interest. In a fourth step d), based on the detected lines and / or symbols, one of a plurality of parking section classes is assigned to each selected region of interest. In a fifth step e), for each of the at least one designated parking areas, it is determined, based on the parking section class assigned to the corresponding region of interest, whether each of the designated parking areas corresponding to the selected region of interest is a parking section candidate.

[0007] This method has the advantage that parking sections can be identified by the parking assistance system based on lines and / or symbols provided on the ground. In particular, parking sections for special purposes can be distinguished from normal parking sections, and parking section candidates marked as no-parking areas can be detected. Thus, the operation of the parking assistance system is improved.

[0008] The optical image around the vehicle may be detected by a camera of the vehicle. Preferably, the optical image corresponds to a surround view image of the vehicle. The surround view image may show an overhead view from above the vehicle. The surround view image may be obtained by combining and / or stitching together a plurality of images each covering a specific field of view.

[0009] The digital surrounding map corresponds to a digital representation of the actual surroundings of the vehicle. The surrounding map can be determined based on sensor signals detected by surrounding sensors arranged on the vehicle, such as an ultrasonic sensor unit and / or an optical sensor unit, by a processing unit of the vehicle, such as an ECU (engine control unit).

[0010] The processing unit that provides the digital surrounding map is preferably also configured to detect the boundary lines of the parking spaces. This detection is based on, for example, an optical image detected by an optical sensor unit such as a camera. Based on the detected boundary lines of the parking spaces, the processing unit is further configured to specify candidate parking areas around the vehicle. A candidate parking area is, for example, an area defined by boundary lines on the ground and is an empty area, that is, an area not blocked by an object or another vehicle, etc. For example, the candidate parking areas can be in the roadside parking strips or parking lots, etc. When specifying the parking area, the processing unit does not consider lines and / or symbols provided on the ground within each area that are not boundary lines.

[0011] Each designated parking area corresponds to an area in the surrounding optical image. Therefore, for each designated parking area, a corresponding region of interest is selected within the optical image.

[0012] Next, the region of interest is analyzed for the lines and / or symbols present therein. Based on the detected lines and / or symbols, one of a plurality of parking section classes is assigned to each selected region of interest. For example, if no lines or symbols are detected, the region of interest may be assigned to the "standard parking section" class. If a wheelchair symbol is detected, the region of interest may be assigned to the "parking section for disabled persons" class. Since each region of interest corresponds to a designated parking area, the parking section class assigned to the region of interest is similarly applied to the designated parking area. Here, the "line" specifically refers to a boundary line provided on the ground. For example, the boundary line may have an appearance that is all similar in line width and color. That is, a line having an appearance different from the appearance assumed as the boundary line may be regarded as representing a symbol in this sense.

[0013] Based on the assigned parking section class, for each of at least one designated parking area, it is determined whether each designated parking area corresponding to the selected region of interest is a parking section candidate. A parking section candidate is a parking section suitable for a vehicle, for example, corresponding to a parking section that the driver of the vehicle has permission to use. This step may involve applying specific filtering criteria and / or priority criteria. For example, parking sections for disabled persons may be set to be filtered out in this step. That is, they are not regarded as parking section candidates. Such settings may be defined by the user in particular, and / or may be based on the configuration of the vehicle such as the vehicle being a plug-in electric vehicle, and / or may be based on detected additional equipment such as a baby seat.

[0014] As a result, this method has the advantage that it can detect all kinds of different special-purpose parking areas and / or no-parking areas by means of a parking assistance system and can filter them by the parking assistance system. Thereby, better operation of the parking assistance system can be realized. This is because the user of a vehicle equipped with a parking assistance system does not need to reconfirm, for example, whether he has obtained permission to use the parking area proposed by the parking assistance system for parking the vehicle.

[0015] According to an embodiment, the method further comprises the steps of collating all the determined parking area candidates, selecting one of the collated parking area candidates, and providing the selected parking area to an autonomous driving unit for controlling the vehicle to park in the selected parking area.

[0016] In this embodiment, one of a plurality of parking area candidates is selected. This is particularly useful when there are a plurality of parking area candidates around the vehicle. The selection may be based on user interaction or may be performed completely automatically.

[0017] The collating step includes the step of ordering the parking area candidates according to a certain ranking criterion. For example, the ordering step may be performed based on the distance to the parking area candidate of the vehicle, based on the length and / or width of the parking area candidate, based on the assigned parking area class corresponding to the parking area candidate, and / or based on other user-defined criteria as the case may be.

[0018] The step of selecting one of the parking area candidates may include the step of outputting a graphic depiction of the surroundings of the vehicle including the parking area candidates to the user of the vehicle via, for example, a display. The user may select one of the parking area candidates by means of a voice command, a gesture, a touch command, or the like.

[0019] The autonomous driving unit may be configured to control the vehicle fully automatically and / or semi-automatically. Full automatic control may include, for example, automatic control of the accelerator, brakes, steering wheel, gears, etc. Semi-automatic may include, for example, control of the steering wheel and gears. The user needs to manually control the accelerator and brakes.

[0020] According to a further embodiment, it further comprises the step of receiving determination information regarding at least one of the plurality of parking section classes, and step e) additionally is based on the determination information.

[0021] The determination information includes information on how to determine whether the designated parking area is a parking section candidate. In particular, the determination information relates to a plurality of parking section classes. For example, the determination information may include information describing that one of the plurality of parking section classes, for example, "section for the disabled" or "parking section for electric vehicles", should not be regarded as a parking section candidate. The determination information may be provided and / or input by the user of the vehicle. Alternatively, the vehicle manufacturer may configure the parking assistance system in response to information provided by the customer.

[0022] According to this embodiment, the user of the vehicle can accurately define, according to their personal situation and needs, which parking section classes are parking section candidates for themselves and should be determined as parking section candidates by the parking assistance system. Thereby, the user has the option of using these parking sections to park the vehicle using the parking assistance system. Each step including a selection step where the user selects their needs, for example, via a user interface, may be provided, for example.

[0023] According to a further embodiment of the method, the determination information includes priority information regarding at least one of the plurality of parking section classes, and step e) further includes the step of assigning the priorities to each of the determined parking section candidates, and the step of ordering the determined parking section candidates according to the assigned priorities.

[0024] For example, the priority information may include information that the "normal parking section" having the widest width has the highest priority, and the "normal parking section" arranged as a vertical parking section along the road has a lower priority. Preferably, the priority information is given as a numerical value, for example, from 0 to 100, corresponding to specific characteristics of the parking section. Specific characteristics of the parking section include the assigned parking section class, the geometric characteristics of the parking section, the relative arrangement of the parking section with respect to other surrounding features such as parallel parking, diagonal parking, or head-on parking, the available space next to the parking section, and the like. When assigning priorities, the parking assistance system may be configured to automatically determine a priority score based on the priority information and the individual characteristics of each parking section candidate.

[0025] According to a further embodiment of the method, step c) is a step of detecting whether the detected line is arranged in the same way as one of a plurality of predetermined patterns, and each predetermined pattern further includes a step of corresponding to at least one parking section class.

[0026] In this embodiment, the detected line can be compared with a predetermined pattern. Such a predetermined pattern can be, for example, a pattern provided by a local government or a government intended to be used for designating a parking section in a unified manner within a sphere of influence. The predetermined pattern may also include a pattern that is customarily used without being officially defined.

[0027] The predetermined pattern can be defined using geometric descriptions, for example, the length, width, and / or curvature of the line, the relative arrangement of a plurality of lines, for example, the angle between two lines, the intersection of two lines, the length of the line to the intersection, and the like. It should be noted here that a dot is also regarded as a "line".

[0028] The term "similarly arranged" should be understood as being able to be determined that the detected lines each represent a respective predetermined pattern. It can also be said that it is inferred that the detected lines each represent a respective predetermined pattern.

[0029] Detecting that the detected lines are arranged similarly to one of a plurality of predetermined patterns may be based on a similarity score calculated based on a comparison between the actual arrangement of the detected lines and the arrangement of the lines in the predetermined pattern.

[0030] In this embodiment, the detection may be performed using a deterministic algorithm, or a rule-based algorithm including a heuristic approach, etc.

[0031] According to a further embodiment of the method, the predetermined pattern is defined by geometric features of the detected line and / or a section of the detected line. The geometric features of the detected line and / or a section of the detected line include the absolute length and / or absolute width of the detected line, the length ratio between the detected line and / or a section of the detected line, the width ratio between the detected line and / or a section of the detected line, the angle formed between the detected line and / or a section of the detected line, the curvature of the detected line and / or a section of the detected line, and / or the distance between the detected line and / or a section of the detected line.

[0032] For example, an "X" pattern can be defined as having two lines of essentially the same length that intersect each other at essentially the midpoint of the lines. As another example, a "zigzag" line can be defined as having at least three linearly extending lines connected at the ends and having an angle in the range of, for example, 60° to 120° between them.

[0033] According to a further embodiment of the method, each one of the plurality of said predetermined patterns comprises a set of values or value intervals corresponding to at least some of said geometric features.

[0034] According to a further embodiment of the method, step c) further comprises extracting the detected symbol of each region of interest as an image, and classifying the image using an artificial intelligence trained to classify images.

[0035] In this embodiment, the symbol provided on the ground is classified by an artificial intelligence such as a neural network trained for the classification of known symbols. In particular, the artificial intelligence is trained for symbols known to be used to mark parking spaces. The artificial intelligence can classify the symbols into a number of predetermined classes. Preferably, each class corresponds to at least one parking space class.

[0036] First, the symbol is extracted as an image from the region of interest. This may include applying steps to prepare the image containing the symbol for analysis by the artificial intelligence by resizing and / or converting it.

[0037] The artificial intelligence can be trained to recognize (classify) figurative symbols, numbers, letters, words, etc. The artificial intelligence can also take into account the color of the structures in the image.

[0038] According to a second aspect, a computer program product is proposed. The computer program product comprises instructions which, when executed by a computer of the program, cause the computer to perform the method according to the first aspect.

[0039] A computer program product, such as a computer program medium, can be provided in the form of a memory device such as a memory card, USB stick, CD-ROM, DVD, etc., and / or in the form of a digital data file downloadable from a server or the like within a computer network. For example, this can be achieved by transferring the corresponding file via a wireless network.

[0040] According to a third aspect, a parking assistance system for a vehicle is proposed. The parking assistance system includes a receiving unit for receiving an optical image around the vehicle and a digital surrounding map representing the surroundings of the vehicle, the digital surrounding map including at least one designated parking area; a selection unit for selecting, for each of at least one of the designated parking areas, a region of interest in the received optical image corresponding to the respective designated parking area; a detection unit for detecting lines and / or symbols provided on the ground in each of the selected regions of interest; an assignment unit for assigning, based on the detected lines and / or symbols, one of a plurality of parking section classes to each of the selected regions of interest; and a determination unit for determining, for each of at least one of the designated parking areas, whether the respective designated parking area corresponding to the selected region of interest is a parking section candidate, based on the parking section class assigned to the corresponding region of interest.

[0041] This parking assistance system has the same advantages as those described for the method according to the first aspect. The embodiments and features suggested for the method according to the first aspect can similarly form the features and embodiments of the parking assistance system in the corresponding aspects.

[0042] Each unit of the parking assistance system, for example, the receiving unit, the selection unit, the detection unit, the allocation unit, and the determination unit, can be implemented in hardware and / or software. When implemented in hardware, each unit can be implemented as a computer, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or a PLC (Programmable Logic Controller). When implemented in software, each unit can be configured as a computer program product, a function, an algorithm, a routine, a part of programming code, or an executable object. Further, each unit can be implemented as a part of a vehicle control unit such as an ECU (Engine Control Unit).

[0043] According to a fourth aspect, a vehicle is proposed that includes at least one camera for detecting an optical image around the vehicle, a processing unit for providing a digital surrounding map representing the surroundings of the vehicle based at least in part on the detected optical image, and a parking assistance system according to the third aspect.

[0044] According to an embodiment, the vehicle further includes a communication unit configured to transmit the assigned parking section class corresponding to the designated parking area to an external unit.

[0045] This embodiment has the advantage that the detection result by the parking assistance system can be provided to other nearby users or vehicles, and / or users or vehicles planning to move to the same location. Specifically, vehicles that do not have the advanced function of detecting the parking section class can also utilize this information when searching for an appropriate parking section. Preferably, the vehicle position information such as GPS coordinates is transmitted together with the assigned parking section class.

[0046] The communication unit is specifically configured to transmit information by modulated electromagnetic signals via, for example, Bluetooth(R), Wi-Fi, infrared, 3G / 4G / 5G technology, etc.

[0047] According to a fifth aspect, a system is proposed that includes at least one vehicle according to an embodiment of the fifth aspect and an external unit outside the vehicle. The external unit is configured to receive the assigned parking section class transmitted from the vehicle and to transmit the received parking section class corresponding to the designated parking area to a further device, and / or to transmit a control command for controlling the vehicle to the vehicle according to the received assigned parking section class.

[0048] The external unit may be realized as a mobile device such as a smartphone, as a fixed device such as a desktop computer or a server, or as a communication unit such as another vehicle.

[0049] In a preferred embodiment, the communication unit may further transmit a list of parking section candidates to the external unit. Then, the user may select one of the parking section candidates. The external unit transmits this selection as a control command for controlling the vehicle to autonomously park in the selected parking section.

[0050] The present invention has been described from the perspective of various embodiments. It should be understood that the singular or plural features of any one embodiment may be combined with the singular or plural features of other embodiments. Furthermore, any single feature or combination of features in any embodiment may constitute additional embodiments.

[0051] Further embodiments or aspects of the present invention are the subject of the examples described below with reference to the dependent claims and the drawings.

Brief Description of the Drawings

[0052]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

[0053] In the drawings, unless otherwise specified, the same elements are denoted by the same reference numerals.

[0054] FIG. 1 is a schematic diagram of a vehicle 100, which is a car for example. The vehicle 100 includes a processing unit 105 and a parking assistance system 110. The processing unit 105 can be implemented as an engine control unit (ECU). Although the processing unit 105 and the parking assistance system 110 are shown as separate units in FIG. 1, they may be integrally implemented in a single integrated circuit and / or share resources such as a CPU and a RAM. The vehicle 100 further includes a plurality of sensors 120, 130. For example, the sensor 120 is implemented as an optical sensor and may include a camera, a radar, and / or a lidar, etc. The optical sensor 120 is configured to detect an image preferably including depth data about the surroundings of the vehicle 100 and output the detected image as an optical sensor signal. The sensor 130 is implemented as, for example, an ultrasonic sensor and is configured to detect the distance to an object 200 (see FIGS. 2 or 3) near the vehicle 100 and output the detected distance as an ultrasonic sensor signal. In addition to the sensors 120, 130 shown in FIG. 1, the vehicle 100 may include additional and / or other sensors such as a microphone, an acceleration sensor, and an antenna for receiving electromagnetic data signals.

[0055] The processing unit 105 is configured to determine a surrounding map MAP (FIG. 2) based on sensor signals, for example, by using sensor fusion technology. The surrounding map MAP corresponds to a digital depiction of the actual surroundings of the vehicle 100. The processing unit 105 may be further configured to detect the boundary lines 210 (see FIG. 2) of the parking spaces in the optical sensor signals and to specify the parking areas P1 to P7 in the surrounding map MAP based on the detected boundary lines 210. In this way, the processing unit 105 determines whether the space between the two boundary lines 210 is empty or blocked, for example, by an object 200 such as another vehicle. If the space is blocked or obstructed, the respective area is not designated as a parking area.

[0056] The parking assistance system 110 can be implemented as shown in FIG. 6 and is configured to perform the processes described below corresponding to FIGS. 2 to 5.

[0057] FIG. 2 shows an example of a digital surrounding map MAP of the vehicle 100. The digital surrounding map MAP includes geometric objects representing the objects around the vehicle 100. In particular, the digital surrounding map MAP includes the center line CL of the road and the dashed lines CL, SL representing the side lines of the road. It includes the geometric object 100 representing the vehicle 100, which includes geometric objects 200 representing the surrounding vehicles or other obstacles, which includes further lines (not shown) detected as defining the drivable area, which includes the parking lines PL detected as boundary lines defining the parking areas, which includes the designated parking areas P1 to P7. The designated parking areas P1 to P7 represent the parking space candidates for the vehicle 100. It should be noted that the designated parking areas P1 to P7 are preselected based simply on the detection of whether each area is empty or blocked. Only the empty areas are designated as the parking areas P1 to P7. For example, the parking assistance system 110 (FIGS. 1, 6, or 7) receives the digital surrounding map MAP from the processing unit 105 (see FIG. 1).

[0058] Figure 3 shows an optical image IMG around the vehicle 100. The optical image IMG corresponds to the digital surrounding map MAP of FIG. 2. For example, the digital surrounding map MAP can be determined at least in part based on the optical image IMG shown in FIG. 3. For example, the parking assistance system 110 (FIGS. 1, 6, or 7) receives the optical image IMG from the processing unit 105. The parking assistance system 110 selects regions of interest ROI1-ROI7 in the optical image IMG that correspond to the designated parking areas P1-P7 (see FIG. 2) of the digital surrounding map MAP (see FIG. 2). As shown in FIG. 3, each of the regions of interest ROI1-ROI7 includes a portion of the optical image IMG that essentially indicates an area where the vehicle 100 can potentially park.

[0059] For each of the regions of interest ROI1-ROI7, the parking assistance system 110 detects lines PAT and / or symbols SMY1-SYM3 provided on the ground. This detection is performed, for example, using image recognition technology. In the example of FIG. 3, the first symbol SYM1 is detected in the region of interest ROI2, the second symbol SYM2 is detected in the region of interest ROI3, the third symbol SYM3 is detected in the region of interest ROI4, and the line PAT is detected in the region of interest ROI7. The first symbol SYM1 represents a "no parking" symbol, the second and third symbols SYM2, SYM3 indicate a "parking for disabled" symbol, and the line PAT forms a zigzag pattern representing a "no parking" space. For example, since there is an entrance across the area corresponding to the region of interest ROI7, it is necessary to prevent parked vehicles.

[0060] Based on the detected lines PAT and / or symbols SYM1-SYM3, the parking assistance system 110 assigns a parking section class CLS (see FIG. 4) to each of the regions of interest ROI1-ROI7. The number of various parking section classes CLS is not limited in itself. The step S4 of assigning the parking section class CLS corresponds to the step of classifying the designated parking areas P1-P7 according to a valid parking section class CLS.

[0061] FIG. 4 shows a table as an example of thus classifying line patterns and symbols into specific parking section classes CLS. In this example, each class CLS is identified by a numerical value. The table includes, in this example, three columns, namely, line patterns and / or symbols, parking section class CLS, and a priority PRIO that can be given to each parking section class CLS.

[0062] The first column of the table in FIG. 4 includes known line patterns and symbols used to mark parking sections. The list in FIG. 4 does not cover everything and may include many other line patterns or symbols and / or variations of line patterns or symbols. As is clear, the same class CLS is assigned to each of the line patterns and symbols representing the "no parking" areas illustrated in rows 3 to 7 of the table in this example. This is the same for various line patterns or symbols representing other types of specific parking sections. For example, all the various symbols representing a "parking for disabled persons" section (not shown) can be assigned to the same class CLS. However, it should be noted that this is not essential, and each different line pattern / symbol can be assigned its own class CLS.

[0063] In the example of FIG. 4, priority information PRIO is further assigned to each parking section class CLS. The priority information PRIO can be provided, for example, by the user of vehicle 100 (see FIG. 1). The priority information PRIO is useful when providing the user with a list sorted based on the priority information PRIO and / or presenting a suitable parking section in a situation where there are a plurality of parking section candidates identified by the parking support system 110 (see FIG. 1). In this example, all "no parking" areas are automatically screened out and cannot be selected as parking sections for vehicle 100 to park in. For example, vehicle 100 is a plug-in electric vehicle and the user of vehicle 100 is a pregnant woman with children. The highest priority PRIO is assigned to the parking sections designated for electric vehicles and pregnant women. The parking sections designated for families have a lower priority PRIO than that. However, this priority is higher than the priority of conventional parking sections (parking sections without line patterns / symbols). The parking sections for the disabled have the lowest priority PRIO. This is because the user of vehicle 100 is not disabled and should not park in such parking sections.

[0064] FIG. 5 is a schematic block diagram showing an example of a method for operating the parking support system 110, for example, the parking support system 110 of vehicle 100 in FIG. 1. In a first step S1, an optical image IMG (see FIG. 3) around vehicle 100 and a digital surrounding map MAP (see FIG. 2) representing the surroundings of vehicle 100 are received. The digital surrounding map MAP includes at least one designated parking area P1 to P7 (see FIG. 2). In a second step S2, for each of the at least one designated parking area P1 to P7 included in the surrounding map MAP, regions of interest ROI1 to ROI7 (see FIG. 3) corresponding to the respective designated parking areas P1 to P7 are selected within the received optical image IMG. In a third step S3, lines provided on the ground (see FIG. 3) and / or symbols SYM1 to SYM3 (see FIG. 3) are detected in each of the selected regions of interest ROI1 to ROI7.

[0065] In this example, the third step S3 includes two steps S31 and S32. Step S31 includes a step of detecting whether the detected line PAT is arranged in the same way as one of a plurality of predetermined patterns, for example, the line patterns shown in the first column of the table in FIG. 4, and each predetermined pattern corresponds to at least one parking section class CLS. Step S31 can be implemented by a classical deterministic algorithm, or a rule-based algorithm that may further include a discovery approach, etc. Step S32 may include a step of extracting the detected symbols SYM1 to SYM3 of each region of interest ROI1 to ROI7 as images, and a step of classifying the images using an artificial intelligence trained to classify the images. For example, step S32 can be implemented by an artificial neural network trained to classify the symbols SYM1 to SYM3 used for parking section marks. The artificial neural network is preferably trained with a supervised learning approach. The training data used to train the artificial neural network may vary depending on various regions, countries, and / or counties around the world. Note that step S3 does not necessarily include the two described steps S31 and S32, and may be a single step performed by a single entity.

[0066] In the fourth step S4, one of the plurality of parking section classes CLS (see FIG. 4) is assigned to each selected region of interest ROI1 to ROI7 based on the detected line PAT and / or symbols SYM1 to SYM3. In the fifth step S5, for each of at least one designated parking area P1 to P7, based on the parking section class CLS assigned to the corresponding region of interest ROI1 to ROI7, it is determined whether each corresponding designated parking area corresponding to the selected region of interest ROI1 to ROI7 is a parking section candidate.

[0067] FIG. 6 is a schematic block diagram of an example of the parking assistance system 110, for example, the parking assistance 110 of the vehicle 100 in FIG. 1. The parking assistance system 110 can operate according to the method described in correspondence with FIG. 5. The parking assistance system 110 receives an optical image IMG (see FIG. 3) around the vehicle 100 and a digital surrounding map MAP (see FIG. 2) representing the surrounding of the vehicle 100, the digital surrounding map MAP including at least one designated parking area P1 to P7 (see FIG. 2). The parking assistance system 110 includes a receiving unit 111 for receiving the optical image IMG and the digital surrounding map MAP, a selection unit 112 for selecting, for each of at least one designated parking area P1 to P7, regions of interest ROI1 to ROI7 of the received optical image IMG corresponding to the respective designated parking areas P1 to P7, a detection unit 113 for detecting lines PAT (see FIG. 3) and / or symbols SYM1 to SYM3 (see FIG. 3) arranged on the ground in each of the selected regions of interest ROI1 to ROI7, an assignment unit 114 for assigning, based on the detected lines PAT and / or symbols SYM1 to SYM3, one of a plurality of parking section classes CLS (see FIG. 4) to each of the selected regions of interest ROI1 to ROI7, and a determination unit 115 for determining, for each of at least one designated parking area P1 to P7, whether the respective designated parking areas P1 to P7 corresponding to the selected regions of interest ROI1 to ROI7 are parking section candidates based on the parking section classes CLS assigned to the corresponding regions of interest ROI1 to ROI7.

[0068] In particular, the detection unit 113 may include a deterministic and / or heuristic determination unit for detecting whether the detected line PAT is arranged in the same manner as one of a plurality of predetermined patterns. Further, the detection unit 113 may include an artificial intelligence such as an artificial neural network trained to classify symbols into one of a plurality of classes. Each class corresponds to the parking section class CLS.

[0069] FIG. 7 shows an example of a system 300 having a vehicle 100 and an external unit 310. The vehicle 100 has the features of the vehicle 100 in FIG. 1 and further includes a communication unit 107. The communication unit 107 can be implemented as a mobile radio modem or a cellular modem. The external unit 310 may be implemented as a mobile device such as a smartphone, or may be a communication unit of another vehicle (not shown) or a fixed device (not shown).

[0070] The communication unit 107 is configured to transmit the assigned parking section class CLS to the external unit 310. For example, the communication unit 107 can connect to the external unit to configure a peer-to-peer connection, or can transmit the assigned parking section class CLS using a predetermined frequency band and / or coding. Then, the external unit 310 can receive the assigned parking section class CLS by listening to a predetermined frequency band.

[0071] The external unit may be configured to transmit a control command (not shown) to the communication unit 107 of the vehicle 100. The control unit may include selecting one of a plurality of parking section candidates determined by the parking assistance system. Also, the control command may include determination information for at least one of a plurality of parking section classes CLS, and may also include determination information that can be used by the parking assistance system 110 when determining whether the designated parking areas P1 to P7 (see FIG. 2) are parking section candidates.

[0072] Although several practical examples of the present technology have been described, it should be understood that the present technology is not limited to the disclosed embodiments, and is also intended to cover various modifications and equivalent arrangements included in the spirit and scope of the present technology.

Description of Reference Numerals

[0073] 100 Vehicle 105 Processing Unit 107 Communication Unit 110 Parking Assistance System 111 Receiver unit 112 Selection unit 113 Detection unit 114 Allocation unit 115 Judgment unit 120 Sensor 130 Sensor 200 Object 210 Boundary line 300 System 310 External unit CL Center line CLS Parking area class IMG Image MAP Map P1 Designated parking area P2 Designated parking area P3 Designated parking area P4 Designated parking area P5 Designated parking area P6 Designated parking area P7 Designated parking area PAT Line PL Parking line PRIO Priority ROI1 Region of interest ROI2 Region of interest ROI3 Region of interest ROI4 Region of interest ROI5 Region of interest ROI6 Region of interest ROI7 Region of interest S1 Method step S2 Method step S3 Method step S4 Method step S5 Method step SL Sideline SYM1 Symbol SYM2 Symbol SYM3 Symbol

Claims

1. A method for operating a parking assistance system (110) for a vehicle (100), comprising: a) receiving (S1) an optical image (IMG) around the vehicle (100) and a digital surrounding map (MAP) representing the surroundings of the vehicle (100), the digital surrounding map (MAP) including at least one designated parking area (P1 - P7); b) for each of at least one of the designated parking areas (P1 - P7), selecting (S2) a region of interest (ROI1 - ROI7) corresponding to each of the designated parking areas (P1 - P7) within the received optical image (IMG); c) detecting (S3) lines (PAT) and / or symbols (SYM1 - SYM3) provided on the ground in each selected region of interest (ROI1 - ROI7); d) based on the detected lines (PAT) and / or symbols (SYM1 - SYM3), assigning (S4) one of a plurality of parking section classes (CLS) to each selected region of interest (ROI1 - ROI7); e) for each of at least one of the designated parking areas (P1 - P7), determining (S5) whether each of the designated parking areas (P1 - P7) corresponding to the selected regions of interest (ROI1 - ROI7) is a parking section candidate based on the parking section class (CLS) assigned to the corresponding region of interest (ROI1 - ROI7); receiving determination information regarding at least one of the plurality of parking section classes (CLS); wherein step e) further comprises, based on the determination information, the determination information including priority information (PRIO) regarding at least one of the plurality of parking section classes (CLS), and being transmitted from an external unit to a communication unit, step e) assigning (S5) the priority (PRIO) to each of the parking section candidates; and ordering (S5) the determined parking section candidates according to the assigned priority (PRIO); step c) further comprises detecting whether the detected line (PAT) is arranged in the same way as one of a plurality of predetermined patterns, each predetermined pattern corresponding to at least one parking section class (CLS).

2. The step of collating all of the determined parking section candidates; The step of selecting one of the collated parking section candidates; The method according to claim 1, further comprising the step of providing the selected parking section to an autonomous driving unit for controlling the vehicle (100) to park in the selected parking section.

3. The method according to claim 1 or 2, wherein the predetermined pattern is defined by geometric features of the detected line and / or detected sections of the line, including the absolute length and / or absolute width of the detected line, the length ratio between the detected line and / or detected sections of the line, the width ratio between the detected line and / or detected sections of the line, the angle formed between the detected line and / or detected sections of the line, the curvature of the detected line and / or detected sections of the line, and / or the distance between the detected line and / or detected sections of the line.

4. The method according to claim 3, wherein each of the plurality of predetermined patterns includes a set of values or value intervals corresponding to at least some of the geometric features.

5. The step c) comprises: The step of extracting the detected symbols (SYM1 to SYM3) of each region of interest (ROI1 to ROI7) as an image; The method according to any one of claims 1 to 4, further comprising the step of classifying the image using an artificial intelligence trained to classify the image.

6. A computer program product comprising instructions which, when executed by a computer of the program, cause the computer to perform the method according to any one of claims 1 to 5.

7. A parking assistance system (110) for a vehicle (100), comprising: A receiving unit (111) for receiving an optical image (IMG) around the vehicle (100) and a digital surrounding map (MAP) representing the surroundings of the vehicle (100), the digital surrounding map (MAP) including at least one designated parking area (P1 to P7). For each of at least one of the specified parking areas (P1 to P7), a selection unit (112) that selects in the received optical image (IMG) a region of interest (ROI1 to ROI7) corresponding to each of the specified parking areas (P1 to P7), In each of the selected regions of interest (ROI1 to ROI7), a detection unit (113) that detects lines (PAT) and / or symbols (SYM1 to SYM3) provided on the ground, Based on the detected lines (PAT) and / or symbols (SYM1 to SYM3), an assignment unit (114) that assigns one of a plurality of parking section classes (CLS) to each of the selected regions of interest (ROI1 to ROI7), For each of at least one of the specified parking areas (P1 to P7), a determination unit (115) that determines whether each of the specified parking areas (P1 to P7) corresponding to the selected regions of interest (ROI1 to ROI7) is a parking section candidate, based on the parking section class (CLS) assigned to the corresponding region of interest (ROI1 to ROI7), and comprising: The receiving unit (111) is configured to receive determination information regarding at least one of a plurality of the parking section classes (CLS), The determination unit (115) is additionally configured to determine based on the determination information, The determination information includes priority information (PRIO) regarding at least one of a plurality of the parking section classes (CLS), and is transmitted from an external unit to the communication unit, The determination unit (115) is configured to assign the priority (PRIO) to each of the parking section candidates, and to order the determined parking section candidates according to the assigned priority (PRIO), A parking support system (110) in which the assigned parking section class (CLS) corresponding to the specified parking area (P1 to P7) is transmitted to an external unit (310).

8. A vehicle (100), comprising at least one camera (120) for detecting an optical image (IMG) around the vehicle (100), a processing unit (105) for providing a digital surrounding map (MAP) representing the surroundings of the vehicle (100) based at least in part on the detected optical image (IMG), a parking assistance system (110) according to claim 7, and a communication unit (107) configured to transmit the assigned parking section class (CLS) corresponding to the designated parking area (P1 to P7) to an external unit (310).

9. In a system (300) comprising at least one vehicle (100) according to claim 8 and an external unit (310) outside the vehicle (100), the external unit (310) is configured to receive the assigned parking section class (CLS) transmitted from the vehicle (100), and to transmit the received parking section class (CLS) corresponding to the designated parking area (P1 to P7) to a further device, and / or is configured to transmit a control command to the vehicle (100) for controlling the vehicle (100) according to the received assigned parking section class (CLS).

Images