ROAD MARKING DETECTION METHOD

Abstract

Method comprising: detecting one or more markings with an asymmetric shape (502) by multiple cameras; and changing an operating mode from a human-operated mode to an autonomous mode for a vehicle, wherein the autonomous mode is configured to set a position, orientation and / or speed of the vehicle based on the one or more detected markings for trajectory correction of the vehicle during navigation.

Description

BACKGROUND

[0001] The present disclosure relates to autonomous vehicle systems and in particular to systems and methods that include road marking detection. State of the art

[0002] The localization of autonomous vehicles presents two primary challenges. The first is identifying a suitable, absolutely reliable sensor for map comparison while ensuring that costs do not increase dramatically. The second is the reliance on wireless communication for map generation, such as satellite information. Generally, sensors like radar, sonar, LiDAR (light detection and distance measurement), or stereo cameras are used for map comparison to identify predetermined features in the environment, which are then compared to the map typically obtained from a GPS (Global Positioning System) unit. However, the use of such sensors eliminates the possibility of detecting two-dimensional colored features, which are the most common in the environment.Furthermore, satellite communication can be inconsistent in the presence of large buildings, leading to unstable and inaccurate vehicle location data. To address the aforementioned problems, several state-of-the-art implementations have been proposed.

[0003] In an example implementation of the prior art, there is a parking assistance system that uses a marking. Such systems employ a symmetrical marking, which is detected by a forward-facing camera mounted on the vehicle, to park in and out of a garage. The difficulty with this method, however, lies in the fact that a vehicle can only park forwards into a parking space, since the camera is facing forwards. Furthermore, because the marking is symmetrical, at least three feature points must be detected for the system to successfully recognize it.

[0004] Another example implementation of the prior art involves camera system calibration for determining the vehicle's position. Such prior art implementations use predefined markers whose shape parameters are stored in computer-readable memory. The shapes are extracted from the camera frames and then compared with the stored shapes. Using rotation and translation, the vehicle's position can be determined. The difficulty with such methods lies in the requirement for additional computer RAM resources.

[0005] In another prior art implementation, a method exists in which markings are arranged at the four corners of the designated parking space. These markings are used to assist in parking the vehicle by utilizing features of the marking pattern. However, this method limits vehicle localization to when the vehicle is close to the parking space, due to the detection range of any camera. This invention incorporates a method in which the markings are arranged at the entrance to the parking space and along the trajectory followed by the vehicle, thus ensuring localization and trajectory correction of the vehicle throughout the entire parking space. SUMMARY

[0006] The example implementations described here involve the use of an asymmetric marker detected by multiple cameras (e.g., surround-view cameras) and are therefore not limited to detection only in front of the vehicle. Furthermore, due to the use of a map positioning unit, the system in these example implementations does not depend on any vehicle guidance system, which can reduce costs.

[0007] Example implementations, as described here, use a template comparison method to detect the marker. In these example implementations, the template can be generated in real time and does not need to be pre-stored.

[0008] In autonomous parking (AVP) systems, the vehicle trajectory during navigation depends on localization efficiency. To counteract the problems with the methods described above, a marker detection method for localizing the autonomous vehicle in the parking space is proposed. This method utilizes an asymmetric marker, which is detected by a surround-view camera system to achieve localization without the use of GPS.

[0009] In a sample implementation that includes AVP, markers are positioned at the entrance to the parking lot and along the trajectory. The location of the marker detected by the cameras is compared to that on a map provided by an offline mapping unit. The cameras use image processing functions to calculate the vehicle's position and orientation relative to these markers. This information is delivered to the vehicle's electronic control unit for autonomous driving (AD-ECU) via CAN (controller area network). Prior to this, the electronic control unit (ECU) also receives the expected initial position and orientation of the vehicle from the mapping unit. The difference between the expected and actual coordinates of the vehicle is then calculated by the ECU and used for trajectory correction.This technique is useful for locating an independent vehicle; however, in the presence of another vehicle ahead, a shared control center is necessary to manage the movement of all vehicles in the parking lot and prevent collisions. Such implementations involve the use of a telematics control unit that transmits the vehicle's position to the control center in real time. The control center, in turn, ensures that no two vehicles are in close proximity to each other by sending commands to the following vehicle to stop it in the event of a potential collision.

[0010] Aspects of the present disclosure include a method that involves detecting one or more markings with an asymmetrical shape using multiple cameras; and changing an operating mode from a human-operated mode to an autonomous mode for a vehicle, wherein the autonomous mode is configured to set a position, orientation and / or speed of the vehicle based on the one or more detected markings.

[0011] Further aspects of the present disclosure include a computer program that involves detecting one or more asymmetrically shaped markings using multiple cameras; changing the operating mode from a human-operated mode to an autonomous mode for a vehicle, wherein the autonomous mode is configured to set the vehicle's position, orientation, and / or speed based on the one or more detected markings. The computer program may be stored on a non-transient, computer-readable medium and executed by one or more hardware processors.

[0012] Aspects of the present disclosure may include a system comprising a means for detecting one or more markings with an asymmetrical shape using multiple cameras; and a means for changing an operating mode from a human-operated mode to an autonomous mode for a vehicle, wherein the autonomous mode is configured to set a position, orientation and / or speed of the vehicle based on the one or more detected markings.

[0013] Aspects of the present disclosure may include a system comprising multiple cameras; and a processor configured to detect one or more markings of an asymmetrical shape from the multiple cameras; and an operating mode to change from a human-operated mode to an autonomous mode for a vehicle, wherein the autonomous mode is configured to set a position, orientation and / or speed of the vehicle based on the detected one or more markings. List of characters Fig. 1(a) represents a vehicle system according to an example implementation. Fig. 1(b) represents several vehicle systems and a management device according to an example implementation. Fig. 2(a) to Fig. 2(e) present examples of asymmetric markings according to an example implementation. Fig. 2(f) and Fig. 2(g) present a comparison between asymmetric markings and symmetric markings. Fig. 3(a) to Fig. 3(d) presents example use cases of asymmetric markers according to an example implementation. Fig. 4(a) to Fig. 4(d) represent example flowcharts that can be executed by the vehicle system or the management unit. Fig. 5(a) to Fig. 5(b) presents example uses of asymmetric markers according to an example implementation. Fig. Section 6 presents example management information that can be used by a management unit and / or the vehicle system. Fig. Section 7 presents a sample computing environment with a sample computer device suitable for use in some sample implementations. DETAILED DESCRIPTION

[0014] The following detailed description provides further details of the figures and example implementations of the present application. Reference numerals and descriptions of redundant elements between figures are omitted for clarity. Terms used throughout the description are provided as examples and are not intended to be limiting. For example, the use of the term "automatic" may include fully automatic or semi-automatic implementations that involve user or administrator control over certain aspects of the implementation, depending on the desired implementation by a person skilled in the field performing the implementations of the present application. The selection may be made by a user through a user interface or other input means, or it may be implemented by a desired algorithm.Example implementations, as described here, can be used either individually or in combination, and the functionality of the example implementations can be implemented by any means according to the desired implementations. The terms "vehicle" and "vehicle system" can also be used interchangeably.

[0015] Fig. 1(a) represents a vehicle system according to an example implementation. In particular, it represents Fig. 1(a) represents a human-operated example vehicle system configured to operate in a human-operated mode and an autonomous mode. The AD-ECU 1 is equipped with a map positioning unit 6 connected to receive signals from the map positioning unit 6to receive. These signals represent a defined route, map data, the vehicle's position on a map, the vehicle's direction, lane information such as the number of lanes, a speed limit, and types of roads / vehicle locations (e.g., highways and main roads, secondary roads, toll booths, parking lots or garages, etc.).

[0016] The vehicle is equipped with an operating parameter measuring unit for measuring values ​​of parameters that indicate the operating conditions of the vehicle, including a wheel speed measuring device. 7 and a vehicle behavior measuring device 8 These devices may include signals. Signals provided by these devices are sent to the AD-ECU. 1 sent. The vehicle behavior measurement device 8 It measures longitudinal acceleration, lateral acceleration, and yaw rate.

[0017] The vehicle is equipped with environmental condition measuring devices for measuring conditions of the environment around the vehicle, including a front camera. 10f , a front radar 11f , a rear camera 10r , a rear radar 11r , a left front camera 12L , a right front camera 12R , a left rear camera 13L and a right rear camera 13R These environmental condition measuring devices send information about lane markings, obstacles, and asymmetrical markings around the vehicle to the AD-ECU. 1 .

[0018] The vehicle's cameras can be in the form of surround-view cameras or other types, depending on the desired implementation. The front camera is part of the vehicle's camera system. 10fThe front radar is equipped with an image acquisition unit for obtaining an image of one or more asymmetric markers around the vehicle and an output unit that provides signals representing the positional relationship between the vehicle and the one or more asymmetric markers. 11f It detects and locates other vehicles and pedestrians and provides signals that represent the positional relationship between the vehicle and these objects. The rear camera 10r , the left front camera 12L , the right front camera 12R , the left rear camera 13L and the right rear camera 13R are in terms of functionality compared to the front camera 10f and to the front radar 11f and to the rear radar 11r similar.

[0019] The vehicle is equipped with a power engine 21 , an electronically controlled braking system 22, an electronically controlled differential mechanism 23 and an electronically controlled steering system 24 equipped. The AD-ECU 1 It sends drive signals to actuators used in these systems. 22 , 23 and 24 These are based on values ​​of influenced variables provided by the driver and / or environmental conditions, such as the detection of asymmetric markings or the activation of various autonomous modes for the vehicle system, as described here. If the vehicle needs to accelerate, the controller 1 an acceleration signal to the power unit 21 If the vehicle needs to be slowed down, the controller sends a deceleration signal to the electronically controlled braking system. 22 If the vehicle needs to be turned around, the AD-ECU will indicate this. 1 a turning signal to the electronically controlled braking system 22, the electronically controlled differential mechanism 23 and / or the electronically controlled steering system 24 .

[0020] The electronically controlled braking system 22 It is a hydraulic braking system capable of controlling individual braking forces applied to each wheel. The electronically controlled braking system applies braking forces to either the right or left wheels in response to a turning request to induce a yaw moment in the vehicle. The electronically controlled differential mechanism 23 In response to a turning request, it drives an electric motor or a clutch to generate a torque difference between the right and left axles, thus applying a yaw moment to the vehicle. The electronically controlled steering system 24For example, a steer-by-wire steering system is able to correct the steering angle independently of the steering wheel's turning angle in response to a turning request in order to apply a yaw moment to the vehicle.

[0021] The vehicle is equipped with an information output unit 26 provided. The information output unit 26 It displays images, generates sounds, and activates warning lights that represent information about assistance operations according to the type of driver assistance operation. The information output unit 26 An example is a monitor equipped with a built-in speaker. Multiple information output units can be installed in the vehicle.

[0022] The system, as in Fig. Figure 1(a) is an example implementation of the vehicle system as described herein, but other configurations are also possible and fall within the scope of protection of the example implementations, and the present disclosure is not limited to the configuration as shown in Figure 1(a). Fig. 1(a) shown, limited. Cameras may, for example, be installed on the top or roof of the vehicle for the purpose of detecting asymmetric markings arranged on walls, road signs, billboards, etc.

[0023] Fig. 1(b) represents several vehicle systems and a management device according to an example implementation. One or more vehicle systems 101-1 , 101-2 , 101-3 and 101-4 , as with reference to Fig. 1(a) described, are connected to a net 100 communicatively coupled, which is connected to a management facility 102 is connected. The management institution 102manages a database 103 , which contains data feedback from the vehicle systems in the network 100 The data is aggregated. In alternative example implementations, the data feedback from the vehicle systems can be... 101-1 , 101-2 , 101-3 and 101-4 are aggregated into a central repository or central database, such as company-owned databases, which aggregate data from systems such as enterprise resource planning systems, and the management facility 102 It can access or retrieve data from the central storage or database. Such vehicle systems can include human-driven vehicles such as passenger cars, trucks, tractors, vans, etc., depending on the desired implementation.

[0024] The management institution 102 It can be configured to receive position information from the vehicle systems 101-1 ,101-2 , 101-3 and 101-4 to receive, which transmit the position of the corresponding vehicle system relative to a marker, as in Fig. 4(a) described, and furthermore be configured to provide instructions to the vehicle systems 101-1 , 101-2 , 101-3 and 101-4 to transmit, specifying a trajectory for the vehicle to the next marker, and / or to set an operating mode, position, speed and / or orientation.

[0025] Fig. 2(a) to Fig. Figure 2(d) presents examples of asymmetric markings according to a sample implementation. The unique features of the selected checkerboard pattern markings make them suitable for detection using cameras in the vehicle systems, as shown in Fig. As shown in Figure 1(a), the difference in intensity of the white and black squares not only makes it easy to detect the transition point as a corner using detection algorithms such as the Harris corner detector, but also allows the marker to be detected under highly variable lighting conditions. Markers can be physically painted or otherwise arranged on roads, signs, or walls, or they can be projected / displayed by projection systems, monitors, etc., for modification depending on the desired implementation.

[0026] Example implementations use asymmetric markers such as T-shaped markers, as in Fig. 2(a) shown, when determining whether the vehicle is traveling in the expected direction (i.e. the vehicle is traveling in the direction opposite to the T of type ID 1 of Fig. 2(b) drives), used. Each of the T-shaped markings can also be associated with an identifier or ID representing different functionalities. However, the present disclosure is not limited to the use of a T-shaped marking; other asymmetric markings, such as an L-shaped marking, as in Fig. 2(c) shown, can also be used. Although example dimensions in Fig. 2(a) to Fig. As shown in Figure 2(c), such markers can also be resized according to the desired implementation. Thus, any asymmetrical checkerboard pattern marker can be used in example implementations.

[0027] If detection information received from the cameras indicates that the vehicle is misoriented relative to the marker, the AD-ECU corrects the trajectory as deemed necessary. The current system uses local coordinates detected by the cameras and converts them into global coordinates by comparing them to a map of the location (or parking area). Depending on the desired implementation, the system can also operate with a QR code (Quick Response Code) placed on the marker, as shown in [reference to relevant diagram]. Fig. 2(d) shown. The QR code can, for example, contain the latitude and longitude information of the marker. Once detected by the cameras, the vehicle's global coordinates can be available immediately.

[0028] Fig. 2(e) presents an example implementation of markings for use at the entrance of a parking lot according to an example implementation. At the entrance of the parking lot, four markings are arranged, one along each side of the vehicle, each progressively rotated 90 degrees clockwise, as shown in Fig. 2(e) shown. The position of the markers is such that, under a rotation condition of 0 degrees, the axis of the marker and the optical axis of the camera align. A peripheral area of ​​the camera view may be less sharp compared to the center of the camera image. The placement of the markers ensures that such sharpness is utilized. Each marker is assigned an ID, as shown in Fig. Figure 2(b) shows how this enables the ECU to determine which camera identified the marker and thus verify its accuracy. The markers are also arranged along the route that the vehicle follows to the parking space.

[0029] Fig. 2(f) and Fig. 2(g) presents a comparison between an asymmetric marking and a symmetric marking. In particular, Fig. 2(f) represents an asymmetric marking and Fig. 2(g) represents a symmetrical marking example. Example implementations use asymmetrical markings because the direction of approach can be distinguished from an orientation of 0 degrees versus an orientation of 180 degrees, as in Fig. 2(f) is shown. If, on the other hand, the marking is symmetrical, as in Fig. As shown in Figure 2(g), the direction of approach cannot be distinguished between an orientation of 0 degrees and one of 180 degrees. Consequently, example implementations use asymmetric markers for easy detection and to determine the vehicle's orientation relative to the marker.

[0030] Although checkerboard patterns are used in the example implementations, the present disclosure is not limited to them, and any other patterns can be used according to the desired implementation, as long as the mark is asymmetrical. For example, zigzag patterns, a mark that includes a circle or other shape indicating the center of the mark, QR codes, or barcodes embedded in the mark can also be used.

[0031] Fig. Figure 3(a) presents an example implementation that includes a parking area. These markers are also assigned an ID and are compared with those stored in the mapping positioning unit. These markers help the vehicle continuously correct its trajectory if the deviation from the desired path exceeds a certain threshold. Not only the mapping positioning unit receives these coordinates, but also the control center, which then decides whether it is safe for the vehicle to proceed, based on a comparison with coordinates received from other vehicles in the parking area, as shown in Figure 3(a). Fig. 3(a) shown.

[0032] As in Fig. As shown in Figure 3(a), the parking lot can be divided into several sections by the map positioning unit based on the location of the markers. In the example of Fig. 3(a) the parking lot will be divided into sections 300-1, 300-2 , 300-3 , 300-4 and 300-5 looking at the section 300-1 The entry point to the parking lot is where a vehicle can be parked and AVP mode can be activated. In one example implementation, AVP mode can be activated upon receiving an instruction from a management unit or from instructions stored in the vehicle system, after the vehicle system detects an initial marker. In another example implementation, the human driver of the vehicle system can manually activate AVP mode, after which the vehicle autonomously navigates the parking lot. As in Fig. As shown in 3(a), the vehicle navigates along a trajectory based on instructions through various sections of the parking lot until it encounters a marker that instructs the vehicle to either adjust the trajectory as described in sections300-2 and 300-3 shown, or to use automatic parking, as described in the section 300-4 As shown. If the parked vehicle needs to be retrieved, instructions can be obtained from the management facility. 102 from Fig. 1(b) be transferred to the parked vehicle, whereby the vehicle can be switched into a retrieval mode in which the vehicle is guided to a marker indicating a pick-up location, as described in section 300-5 The vehicle then stops near the marker indicating a pick-up location, after which a human driver can retrieve the vehicle.

[0033] Fig. 3(b) provides an example of the vehicle system in section 300-1 from Fig. 3(a). In this example configuration of a vehicle system, as in Fig. As shown in Figure 1(a), the vehicle is equipped with a surround-view camera system comprising four monocular cameras, one on each bumper and one on each side mirror. Each frame of the live video captured by the cameras is stitched together by a microprocessor on the surround-view ECU to provide a 360-degree top-view image of the vehicle, as shown in Figure 1(a). Fig. 3(b) shown. The center point of the vehicle is already known by the AD-ECU and is treated as the origin of the coordinate system used to find the distance from the vehicle to the center of the marker.

[0034] Fig. 4(a) to Fig. Section 4(d) presents example flowcharts that can be executed by the vehicle system or the management unit. The flowcharts, as described, can be executed by the vehicle system's AD-ECU or in conjunction with the management unit, depending on the desired implementation.

[0035] Fig. 4(a) presents an example process for a vehicle system according to an example implementation. In particular, it presents Fig. 4(a) presents an example sequence for road marking detection, in which the sequence process of 401 until 405 This represents an example process for finding a marker, the sequence of events. 407 until 409 This presents an example procedure for finding the center of the marker and the procedure at 410 This represents an example process for finding the distance from the vehicle system to the marker. In an example implementation, as with reference to Fig. As described in 3(b), all four monocular cameras sequentially follow a recognition process to provide the position and orientation as a final output via CAN; however, the following sequence can be adapted for any desired configuration of the vehicle system, as described in Fig. 1(a) described, to be carried out.

[0036] The process continues with the camera system (e.g., a 360° camera system) taking a top-down view image at 400 provides, whereby the process attempts to create a marker within a given frame by extracting a region from the frame for processing at 401 to find. Each picture frame obtained from the omnidirectional ECU will be at 402The image is subjected to smoothing using a low-pass core filter {7,7,7,7,8,7,7,7,7}, which helps reduce noise in the image. This smoothed image is then converted into a binary image for further processing at 403 using some technique according to the desired implementation (such as adaptive Bradley thresholding). Consequently, a template comparison is performed on this binary image at 404 The process is carried out using a template image that is generated in parallel. The match obtained from the template comparison is treated as a potential marker candidate within the framework or is not based on a predetermined threshold.

[0037] At 405 A determination is made as to whether a potential tag candidate is found. If no tag is found (No), then the process proceeds to the next frame at 406 further and returns to the process at 401back. If a potential marker candidate is found within the framework (Yes), then the process proceeds to 407 furthermore, whereby the expiration detection region is restricted within the framework (template region) to limit the possibility of false detection, as in Fig. 3(c) shown.

[0038] In this restricted detection region, the desired corner detection algorithm, such as the Harris corner detection method at 408, is used to find all corners. In an example implementation, the use of the Harris corner detection method ensures that a transition point qualifies as a corner based on how it scores against a predetermined threshold. The number of corners obtained in the binary object is compared to the threshold number of corners expected in the T-shaped marker (i.e., 27 ). If, in a sample implementation, the number of corners is higher than23 , but less than or equal to 27 If a certain number of vertices is detected, then the binary object qualifies as a marker. However, other thresholds can be used depending on the shape of the asymmetric marker (e.g., L-shaped, Z-shaped, etc.), and the present disclosure is not limited to these. If the number of vertices does not meet the desired threshold (No), then the process proceeds to... 406 to the next frame and returns to 401 back. Otherwise (yes), the process continues. 409 continue to find the center of the marker.

[0039] At 409 The process finds the center of the marker, for which the coordinates of the template region and the corners obtained from template comparison and Harris detection, respectively, are used. As in Fig. As shown in Figure 3(d), four points surrounding the center are obtained through mathematical operations on the dimensions of the template region window to define a square search window for the center. The corner coordinates output by Harris detection are then iterated over to find the coordinate lying within this search window, and the resulting point is the center of the marker.

[0040] At 410 The process finds the distance of the vehicle from the center, which is obtained by applying the distance formula to this corner coordinate and the vehicle's origin coordinate. Then, at 411 The distance was compared with the map to determine the difference and derive the position.

[0041] Through the process of Fig. 4(a) If the template comparison does not find an exact candidate, or if the number of corners detected by corner detection is greater than the number of expected corners, the current video frame is considered unsuitable and the AD-ECU receives feedback to proceed to the next frame. Therefore, if a marker is not detected with the desired accuracy, a non-detection result is given a higher priority than a false detection.

[0042] The vehicle position coordinates obtained from the road marking detection method are first used by the AD-ECU to determine the vehicle's orientation and then for trajectory correction. The global default coordinates of the road markings are supplied to the AD-ECU by the map positioning unit. If the vehicle follows a trajectory along points other than the previously determined coordinates, the road marking detection method can be used to identify the difference. Based on this position difference, along with the orientation difference, the ECU instructs the actuators to compensate for the difference during navigation by adjusting the vehicle's position, orientation, and / or speed as necessary.This procedure also enables the control center to ensure that two vehicles never collide in parking lots or in other scenarios, such as at toll booths or in areas where autonomous driving is used. Fig. 4(a) The vehicle can therefore set a position, orientation and / or speed based on the calculated distance and / or instructions associated with the detected marker. The instructions can be stored in the vehicle's system by Fig. 1(a) be stored in advance or by a management entity 102 be transferred, as in Fig. 1(b) shown.

[0043] Fig. 4(b) presents an example sequence for handling the non-detection of a marker after the autonomous system has been burdened by the detection of a marker. In this example sequence, the autonomous system is used when 420, when a marking is detected, either automatically or by a manual instruction within the vehicle system, to engage the autonomous system in response to the marking detection. 421 The vehicle system continues along the trajectory and at the speed specified by the initial detection marker while attempting to detect a marker along the trajectory. Detection can occur based on progress along the provided trajectory until another marker is expected. If a marker is detected (Yes), then the vehicle continues along the trajectory at 425 continues or sets the trajectory based on instructions assigned to the detected marker. Otherwise (No), the process goes to 422Furthermore, dead reckoning results are used to perform localization and to determine the error of the dead reckoning results relative to the expected marker location. In such an example implementation, the vehicle system interpolates the current position of the vehicle based on the current speed and orientation of the vehicle system. 423 A determination is made as to whether the error is less than a certain threshold (e.g., within a certain expected distance of the expected marker). If so (Yes), then the vehicle proceeds along the trajectory at 425 continues or adjusts the vehicle's orientation / speed to correct the error; otherwise (no), the process continues. 424continues and stops the autonomous mode. In such an example implementation, depending on the desired implementation, the system can be stopped by switching the vehicle to an emergency mode, which causes the vehicle to continue to a safe location and sends instructions to the management device. 102 regarding the situation, or can switch back to human operating mode with an alarm to the human driver.

[0044] Fig. 4(c) presents an example workflow for applying markers for tailgating detection according to a sample implementation. If a following vehicle can detect the marker at the rear of the vehicle in front, the conclusion is that the following vehicle is too close and the distance between vehicles should be increased. Such functionality can be useful in stop-and-go traffic. 430 The vehicle detects at least one marking that corresponds to the rear end of a preceding vehicle. In the example of Fig. 3(a) is, for example, the vehicle in section 300-2 a vehicle that is part of the vehicle of section 300-1 is moving ahead. Since both vehicles are navigating the same trajectory, there is a possibility that the vehicle in the section is moving ahead. 300-1 the vehicle in the section 300-2can drive too close.

[0045] At 431 A determination is made as to whether the marking was detected with sufficient clarity to indicate that the vehicle is following another vehicle too closely. If yes (yes), the process continues to step 432; otherwise (no), the process continues to... 435 the vehicle continues at the same speed. In an example implementation, the clarity can be determined from the resolution of the detected marker (e.g., triggering when the pixel resolution of the marker reaches a certain threshold), from the size of a marker (e.g., triggering when the size of the marker exceeds a certain threshold), or by other methods according to the desired implementation.

[0046] At 432A determination is made as to whether the distance to the next vehicle is too close (e.g., the distance is within a specified threshold safety distance). If so (Yes), then the process continues. 433 further, the AD-ECU uses the actuators to adjust the vehicle to change the distance between the vehicles. Otherwise (no), the process continues to 435 Furthermore, depending on the desired implementation, this decision-making process can be carried out in the same way as the process for... 431 and omitted. In other example implementations, the distance between radar systems, as with reference to Fig. 1(a) described, from instructions issued by the management body 102 regarding the distance between the vehicles, or determined by other implementations depending on the desired implementation.

[0047] Fig. 4(d) presents an example flowchart for AVP according to a sample implementation. As in Fig. 3(a) shown, when entering a parking space in section 300-1 The AVP mode is used by the vehicle system, whereupon the vehicle follows a trajectory assigned to the marker, based on the information provided by a management device. 102 received instructions or proceeds as pre-programmed in the AD-ECU. 440 The vehicle continues along the trajectory and detects a subsequent marker while in AVP mode. 441A determination is made as to whether the marker indicates a change for the vehicle from AVP mode to automatic parking mode. This determination can be based on pre-programmed instructions associated with the referenced marker or can be made by the management system. 102 be provided. If yes (yes), then the vehicle will be put into automatic parking mode when 443 The AD-ECU has been modified and is configured to automatically park the vehicle in an available parking space, as described in section [section number missing in original text]. 300-4 from Fig. 3(a) shown. Otherwise (No), the vehicle drives at 442 Continue to the next marker. The AD-ECU can control the vehicle based on instructions from a management unit. 102to be received to determine which parking space is available, along with the appropriate instructions to navigate to the empty parking space, park automatically.

[0048] Fig. 5(a) presents an example implementation that includes markings used as signs. In the example implementations mentioned above, markings are arranged on a 2D road surface and are detected by multiple cameras, such as surround-view cameras. However, other example implementations are also possible, and the present disclosure is not limited to them. Fig. 5(a) presents another example implementation that includes 3D road signs, which can be used independently of or in conjunction with the example implementations described herein. Such implementations can assist in vehicle localization and can also provide coordinates representing a stop sign or other signs mapped to a map. In such example implementations, any suitable camera system that also incorporates depth information can be used for the detection of such 3D road signs. Such camera systems can include stereo cameras or other cameras, depending on the desired implementation. In the example described herein, a stereo camera is used. 501 Included on the vehicle's rearview mirror to create asymmetrical markings 502to detect markings used as or on street signs or walls, or on monitors or billboards. In this way, markings can be changed as needed by altering the signs, or how the marking is displayed on the monitor.

[0049] Fig. 5(b) presents an example implementation that includes a toll barrier. When the vehicle system is about to pass through the toll barrier, its orientation must be precise to avoid scratching the sides of the vehicle system. If markers are placed in the area before or after the barrier, the vehicle can be put into autonomous mode and navigate accurately by calculating its position and performing trajectory corrections as needed. In the example of Fig. 5(b) The autonomous mode is initiated as soon as the marker at 511 is detected. The vehicle system then follows a trajectory through the toll barrier and drives according to a set of instructions issued by the management unit. 102 provided, or by instructions stored in the vehicle system regarding how to proceed through a toll barrier. Once the vehicle system has detected the marker 510 Once the target is reached, the operating mode switches to human-operated mode, with the human driver continuing to operate the vehicle system. Markings can be painted on the road or projected by light systems in the toll barrier, allowing the markings to be modified as needed.

[0050] Fig. Section 6 presents sample management information according to a sample implementation. In particular, it presents Fig. Section 6 presents a sample index of markers arranged for one or more locations to assist with the functionality of the vehicle's autonomous modes. Management information can be found on the vehicle system page in the map positioning unit. 6 or in the management facility 102 depending on the desired implementation. In the example of Fig. 6. The management information can include a marker ID, marker type, map information, operating mode, and instructions. The marker ID is the unique identifier that corresponds to the detected marker. The marker type indicates the template type to which the marker belongs (e.g., T-shaped, Z-shaped, L-shaped, etc., as described in...). Fig. 2(b)), and the template types are used to determine whether an initial marker is detected. The map information specifies the corresponding location on a map (e.g., geographic coordinates, latitude / longitude coordinates, geographic information system waypoint coordinates, etc.) for the marker ID. The operating mode specifies an operating mode for the vehicle, such as human-operated mode and various autonomous modes (e.g., AVP mode, automatic parking mode, toll gate mode, tailgating detection mode, etc.). Instructions are instructions associated with a marker ID and can include trajectory information to proceed to the next marker, instructions to transmit to and receive instructions regarding navigation, instructions to adjust a vehicle's position, orientation, and / or speed, etc.according to the desired implementation.

[0051] Fig. Section 7 presents an example computing environment with an example computer device suitable for use in some example implementations, such as to facilitate the functionality for an AD-ECU 1 and map positioning unit. 6 a vehicle system, as in Fig. 1(a) shown, or a management body 102 , as in Fig. 1(b) is shown. All functions described here can be accessed at the management facility. 102 , on the vehicle system or through a system based on a certain combination of such elements, depending on the desired implementation.

[0052] The computer device 705 in the computing environment 700 can be one or more processing units, cores, or processors 710 , a working memory 715(e.g., RAM, ROM and / or the like), an internal memory 720 (e.g. magnetic, optical, semiconductor memory and / or organic) and / or an I / O interface 725 include any of which are based on a communication mechanism or bus 730 coupled for the communication of information or integrated into the computer device 705 They can be embedded. The I / O interface 725 It is also configured to receive images from cameras or deliver images to projectors or displays, depending on the desired implementation.

[0053] The computer device 705 can be used with an input / user interface 735 and an output device / interface 740 be communicatively coupled. One or both of the input / user interface 735 and the output device / interface 740They can be a wired or wireless interface and can be detachable. The input / user interface 735 This can include any device, component, sensor, or interface, physical or virtual, that can be used to provide input (e.g., buttons, touchscreen interface, keyboard, pointer / cursor control, microphone, camera, Braille, motion sensor, optical reader, and / or the like). The output device / interface 740 This can include a display, television, monitor, printer, speaker, Braille, or similar device. In some example implementations, the input / user interface may include... 735 and output device / interface 740 into the computer device 705be embedded or physically coupled to it. In other example implementations, other computer devices can serve as the input / user interface. 735 and output device / interface 740 for a computer device 705 function or provide the functions thereof.

[0054] Examples of computer equipment 705 These may include, but are not limited to, highly mobile devices (e.g., smartphones, devices in vehicles and other machines, devices worn by humans and animals, and the like), mobile devices (e.g., tablets, notebooks, laptops, personal computers, portable televisions, radios, and the like), and devices not designed for mobility (e.g., desktop computers, other computers, information stands, televisions with one or more processors embedded in and / or coupled to them, radios, and the like).

[0055] The computer device 705 can be used with external storage 745 and a network 750 be communicatively coupled to any number of networked components, devices and systems (e.g. via the I / O interface). 725 ), including one or more computer devices of the same or a different configuration. The computer device 705 or any connected computer device may function as a server, client, thin server, general-purpose machine, special-purpose machine or any other designation, provide services thereof or be referred to as such.

[0056] The I / O interface 725May include, but are not limited to, wired and / or wireless interfaces using any communication or I / O protocols or communication or I / O standards (e.g., Ethernet, 802.11x, Universal System Bus, WiMax, Modem, Cellular Network Protocol, and the like) for communicating information to and / or from at least all of the connected components, devices, and the network in the computing environment. 700 The network 750 This can be any network or combination of networks (e.g., the Internet, a local area network, a wide area network, a telephone network, a cellular network, a satellite network, and the like).

[0057] The computer device 705It can use and / or communicate using computer-usable or computer-readable media, including transitory and non-transitory media. Transitory media include transmission media (e.g., metal cables, fiber optics), signals, carrier waves, and the like. Non-transitory media include magnetic media (e.g., disks and tapes), optical media (e.g., CD-ROM, digital video discs, Blu-ray discs), semiconductor media (e.g., RAM, ROM, flash memory, semiconductor storage), and other non-volatile memory or working memory.

[0058] The computer device 705It can be used to implement techniques, procedures, applications, processes, or computer-executable instructions in some example computing environments. Computer-executable instructions can be retrieved from transitory media and stored on and retrieved from non-transitory media. The executable instructions can originate from one or more of any programming, scripting, and machine languages ​​(e.g., C, C++, C#, Java, Visual Basic, Python, Perl, JavaScript, and others).

[0059] (One) processor(s) 710 It can run on any operating system (OS) (not shown) in a native or virtual environment. One or more applications can be used that utilize a logic unit. 760 , an application programming interface unit (API unit) 765 , an input unit 770 , an output unit 775 and a mechanism 795This includes communication between units, enabling the various units to communicate with each other, with the operating system, and with other applications (not shown). The described units and elements can be modified in design, function, configuration, or implementation and are not limited to the descriptions provided.

[0060] In some example implementations, if information or an execution instruction is provided by the API unit... 765 to be received, they are forwarded to one or more other units (e.g. logic unit). 760 , input unit 770 , Output unit 775 ) are transferred. In some cases, the logic unit 760 It must be configured to control the flow of information between the units and the services provided by the API unit. 765 , input unit 770 , Output unit 775provided to guide some of the example implementations described above. The flow of one or more processes or implementations can be controlled, for example, by the logic unit. 760 alone or in conjunction with the API unit 765 be controlled. The input unit 770 can be configured to receive an input for the calculations described in the example implementations, and the output unit 775 It can be configured to provide output based on the calculations described in the example implementations.

[0061] The RAM 715 can be used in conjunction with external storage 745 can be used as a map positioning unit 6 to function, and to be configured to manage management information, as in Fig. Figure 6 illustrates this. Such implementations allow the AD-ECU 1 to manage different operating modes of the vehicle system, as shown in Figure 6. Fig. 1(a) shown.

[0062] (The) processor(s) 710 can be configured to display the flowcharts of Fig. 4(a) to Fig. 4(d) to execute and to rely on management information in Fig. 6. In an example implementation, the processor(s) can (can) 710 be configured to detect one or more markings with an asymmetrical shape from multiple cameras; and to change an operating mode from a human-operated mode to an autonomous mode for a vehicle, the autonomous mode being configured to set a position, orientation and / or speed of the vehicle based on the one or more detected markings, as described in Fig. 4(a) described.

[0063] In an example implementation, the processor(s) can... 710 be configured for autonomous mode, which is an autonomous parking service mode (AVP mode), to set the vehicle's position, orientation, and / or speed according to a trajectory assigned to one or more detected markers; to navigate the vehicle along the trajectory until one or more secondary markers are detected; and for one or more other markers indicating a change in the trajectory, to change the vehicle's position, orientation, and / or speed according to the change in the trajectory, as in Fig. 3(a) to Fig. 3(d) and Fig. 4(a) to Fig. 4(d) described. For the one or more second markers that give instructions to park the vehicle, the processor(s) can (can) 710be configured to transmit the vehicle's position to a management unit; and upon receiving an instruction from the management unit to change the operating mode from AVP mode to automatic parking mode, to switch the operating mode from AVP mode to automatic parking mode, the automatic parking mode being configured to perform automatic parking of the vehicle in a parking space, as described in relation to Fig. 4(d) described.

[0064] In an example implementation, if the processor(s) 710 The functionality of the AD-ECU is facilitated by the processor(s). 710 configured to manage the operating mode and to adjust the position, orientation and / or speed of the vehicle by controlling one or more of the vehicle's actuators, as described in relation to the vehicle system of Fig. 1(a) described.

[0065] In an example implementation, the processor(s) can... 710 be configured to detect one or more markings of the asymmetrical shape from the multiple cameras by performing a template comparison on one or more images received from the multiple cameras; for the template comparison indicating that the one or more markings are present in the one or more images, to perform a corner detection on the one or more images; and for the corner detection indicating a number of corners that meet a certain threshold, to calculate a distance from the vehicle to the one or more markings from the one or more images, as with reference to Fig. 4(a) described.

[0066] In an example implementation, the processor(s) can... 710be configured, while the vehicle is operating in autonomous mode, to fail to detect an expected marker based on the vehicle's trajectory, to interpolate a current position of the vehicle, and to determine an error between the current position of the vehicle and the position of the expected marker; and for an error greater than a threshold, to terminate the vehicle's autonomous mode, as described in Fig. 4(b) described.

[0067] In an example implementation, the processor(s) can... 710 be configured to manage a relationship between multiple markers and a map, as in Fig. 6 shown, and furthermore be configured to transmit, for a vehicle having an autonomous mode operating mode, a position of the vehicle to a management unit, the position being calculated based on the relationship between the one or more detected markers and the map; and, upon receiving trajectory information from the management unit, wherein the instructions specify a trajectory for the vehicle, to adjust the position, orientation and / or speed of the vehicle according to the trajectory information, as with reference to Fig. 1(b) described.

[0068] Some sections of the detailed description address algorithms and symbolic representations of operations within a computer. These algorithm descriptions and symbolic representations are the means used by those skilled in the field of data processing to communicate the essence of their innovations to other experts in the field. An algorithm is a series of defined steps that lead to a desired final state or result. In example implementations, the steps performed require physical manipulations of specific quantities to achieve a particular result.

[0069] Unless specifically stated otherwise, as is evident from the discussion, it is understood that throughout the description discussions using terms such as "processing", "calculating", "calculating", "determining", "displaying" or the like may include the actions and processes of a computer system or other information processing device that affects and transforms data represented as physical (electronic) quantities within the registers and working memory of the computer system into other data likewise represented as physical quantities within the working memory or registers of the computer system or other information storage, transmission or display devices.

[0070] Example implementations may also refer to a facility for performing the operations described here. This facility may be specifically designed for the required purposes, or it may comprise one or more general-purpose computers that are selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored on a computer-readable medium, such as a computer-readable storage medium or a computer-readable signaling medium. A computer-readable storage medium may include specific media such as, but not limited to, optical disks, magnetic disks, read-only memory, random-access memory, semiconductor devices and drives, or any other type of specific or non-transient media suitable for storing electronic information. A computer-readable signaling medium may include media such as carrier waves.The algorithms and displays shown here are not inherently related to any specific computer or other device. Computer programs can consist purely of software implementations containing instructions that perform the operations of the desired implementation.

[0071] Various general-purpose systems can be used with programs and modules as shown in the examples presented here, or it may prove useful to construct a more specialized setup to perform desired procedural steps. Furthermore, the example implementations are not described with reference to any specific programming language. It is recognized that a variety of programming languages ​​can be used to implement the lessons of the example implementations as described here. The instructions of the programming language(s) can be executed by one or more processing devices, such as central processing units (CPUs), processors, or control units.

[0072] As is known in the field, the operations described above can be performed by hardware, software, or any combination of software and hardware. Various aspects of the example implementations can be implemented using circuits and logic devices (hardware), while other aspects can be implemented using instructions stored on a machine-readable medium (software) which, when executed by a processor, would cause the processor to perform a procedure to execute implementations of the present application. Furthermore, some example implementations of the present application can only be performed in hardware, whereas other example implementations can only be performed in software.Furthermore, the various described functions can be performed in a single unit or distributed across a number of components in any number of ways. If performed by software, the procedures can be executed by a processor, such as a general-purpose computer, based on instructions stored on a computer-readable medium. If desired, the instructions on the medium can be stored in a compressed and / or encrypted format.

[0073] Furthermore, other implementations of the present application are apparent to a person skilled in the art from a review of the patent description and the explanation of the teachings contained therein. Various aspects and / or components of the described example implementations can be used individually or in any combination. It is intended that the patent description and example implementations be considered merely as examples, with the actual scope of protection and concept of the present application being specified by the following claims.

Claims

[1] Procedure that includes: Detecting one or more markers with an asymmetrical shape using multiple cameras; and Changing the operating mode of a vehicle from a human-operated mode to an autonomous mode, wherein the autonomous mode is configured to set a position, orientation and / or speed of the vehicle based on one or more detected markers. [2] The method of claim 1, further comprising: for the autonomous mode, which is an autonomous parking service mode (AVP mode): Setting the position, orientation and / or speed of the vehicle according to a trajectory assigned to one or more detected markers; Navigate the vehicle along the trajectory until one or more second markers are detected; and For one or more other markings indicating a change in trajectory, change the position, orientation and / or speed of the vehicle according to the change in trajectory. [3] The method of claim 2, wherein for the one or more second markings that provide instructions to park the vehicle: Transferring the vehicle's position to a management facility; and Upon receiving an instruction from the management unit to change the operating mode from AVP mode to automatic parking mode, switching the operating mode from AVP mode to automatic parking mode, with the automatic parking mode being configured to perform automatic parking of the vehicle into a parking space. [4] Method according to claim 1, wherein the operating mode is managed by an autonomous driving electronic control unit (AD-ECU) configured to adjust the position, orientation and / or speed of the vehicle by controlling one or more actuators of the vehicle. [5] Method according to claim 1, wherein the detection of one or more markings having the asymmetrical shape comprises the multiple cameras: Performing a template comparison on one or more images received from the multiple cameras; For the template comparison, which indicates that one or more markers are present in the one or more images, a corner detection is performed on the one or more images; and For corner detection, which specifies a number of corners that meet a certain threshold, calculating a distance from the vehicle to the one or more markings of the one or more images. [6] The method of claim 1, further comprising, while the vehicle is operating in autonomous mode: In case of failure to detect an expected marker based on the vehicle's trajectory: Interpolating the current position of the vehicle and determining the error between the current position of the vehicle and the position of the expected marker; and For errors exceeding a threshold, the vehicle's autonomous mode will be terminated. [7] Method according to claim 1, wherein the vehicle is configured to manage a relationship between multiple markers and a map, the method further comprising: for the vehicle with an autonomous operating mode, transmission of the vehicle's position to a management unit, wherein the position is calculated based on a relationship between the one or more detected markers and the map; and Upon receiving trajectory information from the management unit, where the instructions specify a trajectory for the vehicle, the vehicle's position, orientation and / or speed is adjusted according to the trajectory information. [8] Non-transitory computer-readable medium that stores instructions for execution by one or more hardware processors, wherein the instructions comprise: Detecting one or more markers with an asymmetrical shape using multiple cameras; and Changing the operating mode of a vehicle from a human-operated mode to an autonomous mode, wherein the autonomous mode is configured to set a position, orientation and / or speed of the vehicle based on one or more detected markers. [9] Non-transitory computer-readable medium according to claim 8, wherein the instructions further comprise: for the autonomous mode, which is an autonomous parking service mode (AVP mode): Setting the position, orientation and / or speed of the vehicle according to a trajectory assigned to one or more detected markers; Navigate the vehicle along the trajectory until one or more second markers are detected; and For one or more other markings indicating a change in trajectory, change the position, orientation and / or speed of the vehicle according to the change in trajectory. [10] Non-transitory computer-readable medium according to claim 9, wherein for the one or more second markings that provide instructions to park the vehicle: Transferring the vehicle's position to a management facility; and Upon receiving an instruction from the management unit to change the operating mode from AVP mode to automatic parking mode, switching the operating mode from AVP mode to automatic parking mode, with the automatic parking mode being configured to perform automatic parking of the vehicle into a parking space. [11] Non-transitory computer-readable medium according to claim 8, wherein the operating mode is managed by an autonomous driving electronic control unit (AD-ECU) configured to adjust the position, orientation and / or speed of the vehicle by controlling one or more actuators of the vehicle. [12] Non-transitory computer-readable medium according to claim 8, wherein the detection of one or more markings with the asymmetric shape by the multiple cameras comprises: Performing a template comparison on one or more images received from the multiple cameras; For the template comparison, which indicates that one or more markers are present in the one or more images, a corner detection is performed on the one or more images; and For corner detection, which specifies a number of corners that meet a certain threshold, calculating a distance from the vehicle to the one or more markings of the one or more images. [13] Non-transitory computer-readable medium according to claim 8, wherein the instructions further comprise, while the vehicle is operating in autonomous mode: In case of failure to detect an expected marker based on the vehicle's trajectory: Interpolating the current position of the vehicle and determining the error between the current position of the vehicle and the position of the expected marker; and For errors exceeding a threshold, the vehicle's autonomous mode will be terminated. [14] Non-transitory computer-readable medium according to claim 8, wherein the vehicle is configured to manage a relationship between multiple markers and a map, and wherein the instructions further comprise: for the vehicle with an autonomous operating mode, transmission of the vehicle's position to a management unit, the position being calculated based on the relationship between the one or more detected markers and the map; and Upon receiving trajectory information from the management unit, where the instructions specify a trajectory for the vehicle, the vehicle's position, orientation and / or speed is adjusted according to the trajectory information. [15] System that includes: several cameras; and a processor configured to do this: to detect one or more markings with an asymmetrical shape using multiple cameras; and to change an operating mode from a human-operated mode to an autonomous mode for a vehicle, wherein the autonomous mode is configured to set a position, orientation and / or speed of the vehicle based on the one or more detected markers. [16] System according to claim 15, wherein the processor is configured to: for the autonomous mode, which is an autonomous parking service mode (AVP mode): to adjust the position, orientation and / or speed of the vehicle according to a trajectory assigned to one or more detected markers; to navigate the vehicle along the trajectory until one or more second markers are detected; and for one or more other markings indicating a change in trajectory, to change the position, orientation and / or speed of the vehicle according to the change in trajectory. [17] System according to claim 15, wherein for the one or more second markings that specify instructions to park the vehicle, the processor is configured to: to transfer the vehicle's position to a management facility; and Upon receiving an instruction from the management unit to change the operating mode from AVP mode to automatic parking mode, the operating mode will be switched from AVP mode to automatic parking mode, with the automatic parking mode being configured to perform automatic parking of the vehicle into a parking space. [18] System according to claim 15, wherein the processor is an autonomous driving electronic control unit (AD-ECU) configured to manage the operating mode of the vehicle and to adjust the position, orientation and / or speed of the vehicle by controlling one or more actuators of the vehicle. [19] System according to claim 15, wherein the processor is configured to detect one or more markings with the asymmetric shape from the multiple cameras by: Performing a template comparison on one or more images received from the multiple cameras; For the template comparison, which indicates that one or more markers are present in the one or more images, a corner detection is performed on the one or more images; and For corner detection, which specifies a number of corners that meet a certain threshold, calculating a distance from the vehicle to the one or more markings of the one or more images. [20] System according to claim 15, wherein the processor is configured to do so while the vehicle is operating in autonomous mode: In case of failure to detect an expected marker based on the vehicle's trajectory: to interpolate the current position of the vehicle and to determine the error between the current position of the vehicle and the position of the expected marker; and to terminate the vehicle's autonomous mode if the error exceeds a threshold.