Automatic parking control device

Abstract

The present invention provides an automatic parking control device, comprising: a surrounding environment recognition device for acquiring surrounding environment information of a vehicle; a parking control unit that searches for a parking-enabled area on the basis of the surrounding environment information, generates a target path in which the parking-enabled area is set as a target arrival point, and performs travel control of the vehicle along the target path; an image processing unit that estimates, on the basis of the surrounding environment information, the distance to the recognized three-dimensional object and the degree of difficulty in slipping of the road surface; an image map generation unit that generates a road surface [mu] map by combining the estimated distance information and the road surface slip difficulty level information with the surrounding environment information; a wheel selection unit that predicts paths through which each of the plurality of wheels passes on the basis of the target path, overlaps the predicted paths of each wheel with the road surface [mu] map, and selects a wheel that passes through the path that is least likely to slip; and a movement amount calculation unit that acquires the wheel speed pulse of the selected wheel and calculates the movement amount of the reference position of the vehicle.

Description

Technical Field

[0001] The present invention is an automatic parking control device that is mounted on a vehicle such as an automobile and performs driving control for guiding the vehicle to a parking area. The invention relates to a technology that can perform accurate automatic parking control or parking assistance control even on slippery road conditions. Background Art

[0002] In recent years, in vehicles such as automobiles, automatic driving control technologies that allow vehicles to automatically drive without requiring a driver's driving operation have been developed. In addition, various driving control devices that can utilize such automatic driving control technologies to perform various driving controls for assisting the driver's driving operation have been proposed and are being put into practical use.

[0003] Among such conventional travel control devices, for example, automatic parking control devices including various parking control systems that perform travel control for guiding a vehicle to a parking possible area, various types of automatic parking control devices have been proposed in recent years.

[0004] Specifically, the parking control system is referred to as, for example, an Assisted Parking System (APS) or a Partially Automated Parking System (PAPS).

[0005] The automatic parking control or parking assist control (hereinafter referred to as automatic parking control, etc.) executed in the conventional automatic parking control device includes, for example, the following control processing. That is, first, the surrounding environment information of the vehicle (such as road dividing lines, three-dimensional structures such as car barriers, etc.) is acquired using various sensors. Next, a parking area is searched based on the acquired surrounding environment information. Next, the searched parking area is set as the target arrival point, and a target path for moving the vehicle is generated. Then, various controls (steering, driving, braking, etc.) for moving the vehicle along the generated target path are appropriately executed.

[0006] Various proposals have been made for automatic parking control devices that execute such automatic parking control, for example, in Japanese Patent Application Laid-Open No. 2023-017467.

[0007] In addition, when executing such automatic parking control, it is important to accurately control the movement of the vehicle along the generated target path. Therefore, when calculating each control value of feedback control and feedforward control of steering, driving, braking, etc., it is necessary to accurately predict the movement amount of the vehicle.

[0008] In this case, conventional methods for predicting the amount of movement of a vehicle include, for example, a so-called GPS (Global Positioning System) method that calculates a moving distance based on the position of the vehicle using a GNSS (Global Navigation Satellite System) sensor, and a wheel speed pulse method that accumulates the moving distance based on wheel speed pulses.

[0009] Here, the GPS method may be in a situation where the reception of radio waves from satellites is unstable or difficult in, for example, a multi-story parking lot or a garage with a roof cover, etc. Therefore, it is considered inappropriate to use the GPS method when predicting the movement amount of the vehicle when executing automatic parking control, etc.

[0010] Therefore, currently, in order to predict the movement amount of the vehicle when executing automatic parking control or the like, a wheel speed pulse method is generally used.

[0011] Prior art literature Patent Literature Patent Document 1: Japanese Patent Application Publication No. 2023-017467 Summary of the invention Technical issues However, when the vehicle movement amount is predicted using the wheel speed pulse method, if the road conditions around the vehicle are such as snowy roads where the wheels are prone to slipping, the wheels may slip and spin. In this way, if the wheels spin, the predicted vehicle movement amount calculated based on the wheel speed pulses will have errors.

[0012] Thus, it is impossible to accurately perform control timings of steering, driving, braking, etc. based on the calculated predicted movement amount. Therefore, there arises a problem that the followability of the target path of the vehicle in automatic parking control, etc. is reduced.

[0013] In this way, when the target path following performance is reduced, there may be problems such as the parking position of the vehicle deviating from the target parking area, or unnecessary operations such as U-turn operations increasing, and the quality of vehicle control being reduced.

[0014] The present invention provides an automatic parking control device, which is mounted on a vehicle such as a car and performs driving control for guiding the vehicle to a parking area. Even on slippery road conditions such as snowy roads, the automatic parking control device can perform accurate automatic parking control or parking assistance control.

[0015] Technical Solution In order to achieve the above-mentioned object, an automatic parking control device according to one embodiment of the present invention performs driving control for guiding a vehicle to a parking available area, and the automatic parking control device comprises: a surrounding environment recognition device, which obtains surrounding environment information of the vehicle; a parking control unit, which searches for a parking available area based on the surrounding environment information obtained by the surrounding environment recognition device, generates a target path for moving the vehicle with the searched parking available area as a target arrival point, and performs driving control for moving the vehicle along the generated target path; and an image processing unit, which estimates a direct arrival point based on the surrounding environment information obtained by the surrounding environment recognition device. The invention relates to a method for determining a road surface friction coefficient map by combining the distance information and the road surface slippage difficulty information estimated by the image processing unit with the surrounding environment information to generate a road surface friction coefficient map; a wheel selection unit, which predicts the paths respectively passed by the multiple wheels of the vehicle based on the target path, makes the predicted path of each wheel overlap with the road surface friction coefficient map, and selects the wheel that passes the path least likely to slip among the multiple wheels; and a movement amount calculation unit, which obtains the wheel speed pulse of the selected wheel and calculates the movement amount of the reference position of the vehicle.

[0016] Technical Effects According to the present invention, an automatic parking control device can be provided, which is mounted on a vehicle such as a car and performs driving control for guiding the vehicle to a parking area. The automatic parking control device can perform accurate automatic parking control or parking assistance control even on slippery road conditions such as snowy roads. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a block diagram showing a schematic configuration of a vehicle travel control device including an automatic parking control device according to an embodiment of the present invention.

[0018] Figure 2 This is a display example of a planar depth map generated by the automatic parking control device according to the present embodiment.

[0019] Figure 3 This is a display example of a planar μ map generated by the automatic parking control device according to the present embodiment.

[0020] Figure 4 This is a display example of a road surface μ map generated by the automatic parking control device according to the present embodiment.

[0021] Figure 5 is Figure 4 An example of a display of the road μ map superimposed with the target path and the predicted wheel path.

[0022] Figure 6 This is a flowchart showing the operation of the vehicle travel control device including the automatic parking control device according to the present embodiment.

[0023] Figure 7 It is shown Figure 6 The road μ map generation process ( Figure 6 Flow chart of a subroutine of the process of step S4).

[0024] Explanation of symbols 1…Automatic parking control device 10…Camera unit 11…Stereo Camera 11a…Main Camera 11b…Secondary camera 12…Image processing unit 13…Image recognition unit 14…Control unit 15…Parking control unit 16…Wheel Selection Department 17…Movement calculation unit 21…Cab control unit (CP_ECU) 22…Engine control unit (E / G_ECU) 23…Transmission control unit (T / M_ECU) 24…Brake control unit (BK_ECU) 25…Power steering control unit (PS_ECU) 31…Human Machine Interface (HMI) 32…Throttle valve actuator 33…Hydraulic control circuit 34…Brake actuator 35…Electric power steering motor 36…Positioning unit 36a…GNSS sensor 36b…High-precision road map database (road map DB) 37…Vehicle-mounted radar device 38…Rear sensor 39…Near infrared sensor 40...Outside air temperature sensor 41…Wheel speed sensor 100…Parking line 101…Car blocker 111…First parking candidate area 112…Second parking candidate area M…This vehicle M1, M2...other vehicles R1…First parking target path R1L…predicted wheel path of the left wheel corresponding to the first parking target path R1R…Predicted wheel path of the right wheel corresponding to the first parking target path R2…Second parking target path R2L…predicted wheel path of the left wheel corresponding to the second parking target path R2R…Predicted wheel path of the right wheel corresponding to the second parking target path DETAILED DESCRIPTION The present invention is described below by means of the illustrated embodiments. The drawings used in the following description are schematically shown, and in order to show the components in a size that can be recognized on the drawings, sometimes different dimensional relationships, scales, etc. of the components are shown for each component. Therefore, in the present invention, the number of components, the shape of components, the ratio of the size of components, the relative positional relationship of components, etc. recorded in the drawings are not limited to the illustrated methods.

[0025] First, the following uses Figure 1 A schematic configuration of a vehicle travel control device including an automatic parking control device according to an embodiment of the present invention will be described. Figure 1 This is a block diagram showing a schematic configuration of a vehicle travel control device including an automatic parking control device according to an embodiment of the present invention.

[0026] Figure 1 The basic structure of the driving control device shown has a structure that is substantially the same as that of a conventional driving control device of the same type. Therefore, in the following description, illustrations of general structures in the driving control device and their detailed descriptions are omitted. Then, the components related to the automatic parking control device of this embodiment are described below.

[0027] The vehicle driving control device including the automatic parking control device 1 of the present embodiment includes a camera unit 10 which is a vehicle-mounted camera device fixed to the front upper center portion of a vehicle (hereinafter referred to as the vehicle, not shown) equipped with the driving control device.

[0028] The camera unit 10 is configured to include a stereo camera 11 , an image processing unit (IPU) 12 , an image recognition unit 13 , a control unit 14 , and the like.

[0029] The stereo camera 11 is formed to include two cameras, a main camera 11a and a sub-camera 11b. The main camera 11a and the sub-camera 11b are disposed, for example, at left-right symmetrical positions sandwiching the center in the vehicle width direction in the cabin of the vehicle, facing the front (travel direction) of the vehicle. The main camera 11a and the sub-camera 11b are each composed of, for example, an imaging optical system, an imaging element such as a CMOS image sensor, and a processing circuit for processing imaging signals.

[0030] With such a structure, the stereo camera 11 acquires two image data from two different viewpoints with the surrounding environment of a predetermined range in front of the vehicle as the object through the main camera 11a and the auxiliary camera 11b in a predetermined shooting cycle synchronized with each other. Then, stereo image data is generated based on the two image data thus acquired. The stereo image data is surrounding environment information showing the surrounding environment during the driving of the vehicle. The surrounding environment information (image data) generated in the stereo camera 11 is output to the image processing unit 12.

[0031] The image processing unit 12 is a component or circuit unit that performs predetermined image processing on the surrounding environment information (image data showing the surrounding environment during the driving of the vehicle) acquired by the stereo camera 11. The image processing unit 12 performs processing such as detecting the edges of various objects (objects or dividing lines, etc.) displayed on the image.

[0032] In addition, the image processing unit 12 obtains distance information based on the positional offset of the corresponding edges on the left and right images based on the stereoscopic image data, and generates image information (distance image information) including the distance information. The distance image information generated by the image processing unit 12 is output to the image recognition unit 13.

[0033] The image recognition unit 13 calculates the road curvature [1 / m] of the left and right dividing lines of the driving road (the driving road of the own vehicle) and the width between the left and right dividing lines (lane width) etc. based on the distance image information etc. received from the image processing unit 12. The method of calculating the road curvature and the lane width uses various well-known means.

[0034] In addition, the image recognition unit 13 performs predetermined pattern matching based on the distance image information acquired by the stereo camera 11, and recognizes three-dimensional objects extending along the road (such as guardrails, curbs, and other surrounding vehicles, etc.), parking frame lines marked by dividing lines on the road surface in buildings such as parking lots, or three-dimensional structures such as car barriers that set parking division areas, or gaps between other adjacent vehicles, etc. In addition, the condition of the road surface around the vehicle (hereinafter referred to as road surface condition, etc.) is also recognized.

[0035] Here, in the recognition of three-dimensional objects (object recognition) in the image recognition unit 13, for example, recognition is performed on the type of the three-dimensional object, the height of the three-dimensional object, the width of the three-dimensional object, the distance from the vehicle to the three-dimensional object, the moving speed of the three-dimensional object, the relative speed between the three-dimensional object and the vehicle, the relative distance between the three-dimensional objects (for example, the lateral distance between a curb at the end of a road and a dividing line located near it, etc.), etc.

[0036] The road conditions recognized by the image recognition unit 13 include, for example, a road wetted by rain, molten snow, etc., rainfall, snow accumulation, snow compression, frozen road conditions, etc. These road conditions are estimated based on, for example, brightness differences in images.

[0037] Various information recognized in the image recognition unit 13 is output as surrounding environment information to the control unit 14. In this case, the camera unit 10 including the image recognition unit 13 has a function as a surrounding environment recognition device that recognizes the surrounding environment of the vehicle.

[0038] In addition, the image recognition unit 13 performs image analysis using AI (Artificial Intelligence) using, for example, semantic segmentation technology, based on the surrounding environment information (image data) acquired by the stereo camera 11. Here, the AI ​​performs depth estimation (distance estimation) related to the recognized three-dimensional object based on the received image data, and generates a two-dimensional (2D) depth map showing the depth distribution on the two-dimensional image. Figure 2 This is an example of the display of the planar depth map generated at this time.

[0039] like Figure 2 As shown, the plane depth map (in Figure 2 Depth-map) is image data in the form of superimposing depth distribution information (depth information) on a normal image. Figure 2 In the display example, the depth (distance) is expressed by the type of hatching. Figure 2 In the example, an indicator showing the depth (distance) within the image is displayed outside the image bar.

[0040] In addition, the AI ​​estimates the road surface conditions (road surface μ estimation) based on the received image data, and then generates a two-dimensional (2D) μ map on the road surface, which displays areas showing the degree of slippage (μ (mu): friction coefficient). Figure 3 This is a display example of the plane μ map generated at this time.

[0041] like Figure 3 As shown, the plane μ spectrum (in Figure 3The image data (denoted as [μ-map]) is a form of image data in which information (μ information) on the degree of slippage of the road surface is superimposed on an area corresponding to the road surface on a normal image. Figure 3 In the example of the display, the area that is easy to slip is indicated by marking the hatching line. Figure 3 In the image, the description "Low_μ" is displayed outside the column. The description "Low_μ" indicates that the hatched portion indicates low μ. Figure 3 In the display example shown, the slip difficulty level is not clearly displayed, but for example, by setting the type of display (hatched display) showing the area prone to slipping to a different display type, the difference in slip difficulty level can be displayed.

[0042] In addition, the image recognition unit 13 makes the above-mentioned plane depth map ( Figure 2 )、Plane μ spectrum( Figure 3 ) and object recognition data (image data) to generate a road μ map of a predetermined pattern ( Figure 4 ). In this case, the image recognition unit 13 functions as an image atlas generating unit.

[0043] Figure 4 1 is a display example of the road surface μ map generated by the image recognition unit 13. The road surface μ map is image data including depth information (distance information) and information on the degree of slippage of the road surface (μ information).

[0044] exist Figure 4 In the display example of the road surface μ map shown, the area near the target parking position including the host vehicle is displayed in a bird's-eye view. Figure 4 The symbol M indicates the vehicle. Figure 4 Symbols M1 and M2 indicate other vehicles. Figure 4 In the process, other vehicles M1 and M2 park in a predetermined parking frame. Figure 4 The symbol 100 shows a parking frame line. Figure 4 Reference numeral 101 indicates a parking barrier provided in a parking frame.

[0045] Figure 4 The symbols 111 and 112 show the parking areas. Figure 4 The symbol 111 shows the first candidate parking area, and the symbol 112 shows the second candidate parking area. Figure 4 The example of FIG. 1 shows a situation in which two of the existing multiple parking areas are identified as candidate parking areas.

[0046] The control unit 14 is a component unit or a circuit unit included in the camera unit 10 , and controls the camera unit 10 , and centrally controls the entire travel control device including the automatic parking control device 1 of the present embodiment.

[0047] The control unit 14 is connected to various control units via in-vehicle communication lines such as CAN (Controller Area Network), such as a cab control unit (CP_ECU) 21, an engine control unit (E / G_ECU) 22, a transmission control unit (T / M_ECU) 23, a brake control unit (BK_ECU) 24, a power steering control unit (PS_ECU) 25, etc.

[0048] A human-machine interface (HMI) 31 disposed around the driver's seat is connected to the CP_ECU 21. The HMI 31 is configured to include, for example, a switch for instructing execution of various driving assistance controls, a mode switching switch for switching driving modes, a steering touch sensor for detecting the driver's grip state, a driver monitoring system (DMS) for detecting the driver's face recognition, line of sight, etc., an in-vehicle monitoring system composed of an in-vehicle camera for recognizing the riding status of people including the driver, a touch panel type display device (visual display device), a combination meter, a sound device including a speaker, etc. (auditory display device), etc.

[0049] When receiving the control signal from the control unit 14, the CP_ECU 21 appropriately reports various information related to various warnings for the preceding vehicle, the implementation status of the driving assistance control, and the surrounding environment of the vehicle to the driver by means of display, sound, etc. via the HMI 31. In addition, the CP_ECU 21 outputs various input information such as the on / off operation status of various driving assistance controls input by the driver through the HMI 31 to the control unit 14.

[0050] A throttle actuator 32 of an electronically controlled throttle valve and the like are connected to the output side of the E / G_ECU 22. In addition, various sensors such as an accelerator sensor (not shown) are connected to the input side of the E / G_ECU 22.

[0051] The E / G_ECU 22 is a driving device that controls the throttle actuator 32 to generate the driving force of the vehicle based on the control signal from the control unit 14 or the detection signal from various sensors. Thus, the E / G_ECU 22 adjusts the intake air amount of the engine to generate the desired engine output. In addition, the E / G_ECU 22 outputs the throttle opening and other signals detected by various sensors to the control unit 14.

[0052] The hydraulic control circuit 33 is connected to the output side of the T / M_ECU 23. In addition, various sensors such as a shift position sensor (not shown) are connected to the input side of the T / M_ECU 23.

[0053] The T / M_ECU 23 controls the hydraulic pressure of the hydraulic control circuit 33 based on the engine torque signal estimated by the E / G_ECU 22, the detection signals from various sensors, etc. Thus, the T / M_ECU 23 operates the friction engagement elements and pulleys provided in the automatic transmission to change the engine output at a desired speed ratio. In addition, the T / M_ECU 23 outputs signals such as the gear position detected by various sensors to the control unit 14.

[0054] A brake actuator 34 is connected to the output side of the BK_ECU 24. The brake actuator 34 is used to adjust the brake fluid pressure output to the wheel cylinders provided at each wheel. In addition, various sensors such as a brake pedal sensor, a yaw rate sensor, a front and rear acceleration sensor, and a vehicle speed sensor (not shown) are connected to the input side of the BK_ECU 24.

[0055] The BK_ECU 24 is a brake device that performs a braking control of the vehicle by driving and controlling the brake actuator 34 based on a control signal from the control unit 14 or a detection signal from various sensors. Thus, the BK_ECU 24 causes each wheel to appropriately generate a braking force for performing a forced braking control, a yaw rate control, etc. for the vehicle. In addition, the BK_ECU 24 outputs signals of the brake operation state, the yaw rate, the longitudinal acceleration, the vehicle speed (the vehicle speed), etc. detected by various sensors to the control unit 14.

[0056] An electric power steering motor 35 that applies steering torque generated by the rotational force of the motor to the steering mechanism is connected to the output side of the PS_ECU 25. Various sensors (not shown) such as a steering torque sensor and a steering angle sensor are connected to the input side of the PS_ECU 25.

[0057] The PS_ECU 25 is a steering device that controls the steering of the vehicle by driving and controlling the electric power steering motor 35 based on a control signal from the control unit 14 or detection signals from various sensors. Thus, the PS_ECU 25 causes the electric power steering motor 35 to generate a steering torque for the steering mechanism. In addition, the PS_ECU 25 outputs signals such as the steering torque and the steering angle detected by various sensors to the control unit 14.

[0058] In addition, various sensors such as a positioning unit 36 ​​, a vehicle-mounted radar device 37 , a rear sensor 38 , a near infrared sensor 39 , an outside air temperature sensor 40 , and a wheel speed sensor 41 are connected to the control unit 14 .

[0059] The positioning unit 36 ​​is configured to include a GNSS sensor 36 a and a high-precision road map database (road map DB) 36 b .

[0060] The GNSS sensor 36 a locates the position (latitude, longitude, altitude, etc.) of the vehicle by receiving positioning signals transmitted from a plurality of positioning satellites.

[0061] The road map DB 36 b is a large-capacity storage medium such as a HDD (Hard Disk Drive) device or an SSD (Solid State Drive) device, and stores high-precision three-dimensional road map information (dynamic map).

[0062] The road map DB 36b stores lane width data, lane center position coordinate data, lane travel azimuth data, speed limit, etc. as lane data required for automatic driving. The lane data is stored at intervals of several meters for each lane on the road map. In addition, the road map DB 36b contains dynamic information that changes all the time, such as accidents and congestion, in addition to traffic control and road construction.

[0063] In addition, the positioning unit 36 ​​can communicate with an external system (not shown) to obtain real-time information about the surrounding environment at the location of the vehicle located by the GNSS sensor 36a (for example, various information related to parking lots, etc. in addition to congestion information and weather information). In this case, the weather information also includes, for example, fog generation information, rainfall information, snowfall information, snow accumulation information, temperature and humidity information in the area including the location of the vehicle.

[0064] In addition, the road map DB 36b stores information on various facilities, parking lots, etc. For example, based on a request signal from the control unit 14, the road map DB 36b outputs road map information of a set range based on the vehicle position located by the GNSS sensor 36a as surrounding environment information to the control unit 14. In this way, in the present embodiment, the road map DB 36b has a function as a surrounding environment recognition device that recognizes the surrounding environment of the vehicle together with the GNSS sensor 36a.

[0065] The vehicle-mounted radar device 37 is composed of a plurality of sensors, such as a plurality of millimeter-wave radars. Here, the plurality of millimeter-wave radars receive reflected waves from objects and analyze the outputted radio waves, thereby mainly detecting three-dimensional objects such as pedestrians and parallel vehicles, and also detecting structures (such as curbs, guardrails, walls of buildings, etc., three-dimensional objects such as planting, etc.) set at the end of the road (such as the end on the shoulder side). In addition, the plurality of millimeter-wave radars also detect three-dimensional obstacles on the road. In this case, the plurality of millimeter-wave radars detect the lateral width of the three-dimensional object, the position of the representative point of the three-dimensional object (relative position and relative distance with the vehicle), and the relative speed as specific information related to the three-dimensional object.

[0066] The multiple sensors (multiple millimeter wave radars, etc.) included in the vehicle-mounted radar device 37 are arranged, for example, on the left and right sides of the front bumper (referred to as the front left and right side sensors), on the left and right sides of the rear bumper (referred to as the rear left and right side sensors), etc. Moreover, the front left and right side sensors detect three-dimensional objects in the area diagonally in front of and to the side of the vehicle that are difficult to identify in the image of the stereo camera 11 as surrounding environment information. In addition, the rear left and right side sensors detect three-dimensional objects in the area diagonally to and to the rear of the vehicle that are difficult to identify in the front left and right side sensors as surrounding environment information.

[0067] Thus, in this embodiment, the vehicle-mounted radar device 37 functions as a surrounding environment recognition device that recognizes the surrounding environment of the vehicle. In addition, information acquired by each sensor of the vehicle-mounted radar device 37 is sent to the image recognition unit 13 through the control unit 14.

[0068] The rear sensor 38 is composed of, for example, a sonar device that uses ultrasonic waves to measure the distance and shape of an object. For example, at least one (or more) rear sensors 38 are provided on the rear bumper. The rear sensor 38 detects three-dimensional objects in the area behind the vehicle that are difficult to be recognized by the rear left and right side sensors as surrounding environment information. Thus, in the present embodiment, the rear sensor 38 has a function as a surrounding environment recognition device that recognizes the surrounding environment of the vehicle.

[0069] The coordinates of each object outside the vehicle contained in the surrounding environment information recognized by the image recognition unit 13, the surrounding environment information recognized by the positioning unit 36, the surrounding environment information recognized by the vehicle-mounted radar device 37, and the surrounding environment information recognized by the rear sensor 38 are all converted into coordinates of a three-dimensional coordinate system with the center of the vehicle as the origin in the control unit 14.

[0070] The near infrared sensor 39 is a sensor for identifying the surrounding conditions of the vehicle, in particular the road surface conditions such as the road surface temperature, etc. The road surface temperature, the amount of moisture on the road surface, the snow condition on the road surface, etc. are identified based on the information obtained by the near infrared sensor 39. In addition, the outside air temperature sensor 40 is a sensor for detecting changes in the outside air temperature around the vehicle.

[0071] The information acquired by the near infrared sensor 39, the outside air temperature sensor 40 and other sensors is detected as the surrounding environment information. In addition, in the present embodiment, the near infrared sensor 39, the outside air temperature sensor 40 and other sensors function as a surrounding environment recognition device for recognizing the surrounding environment of the vehicle.

[0072] The wheel speed sensor 41 is a sensor that detects a wheel rotation speed by detecting a pulse signal (wheel speed pulse) generated in proportion to the rotation speed of each wheel (usually four wheels) in the vehicle.

[0073] The control unit 14 performs vehicle travel control based on various information acquired by various sensors. For example, in addition to engine output control and torque distribution control of each drive wheel by the E / G_ECU 22, the travel control also performs forward or reverse travel direction control by the control of the transmission by the T / M_ECU 23, and individual brake control of each wheel by the BK_ECU 24, thereby appropriately performing the required vehicle travel control.

[0074] Among various sensors for acquiring surrounding environment information, in addition to the above-mentioned sensors, for example, a laser radar (LiDAR: Light Detection And Ranging) device that measures the distance and shape to an object using laser light can also be applied.

[0075] In addition, the stereo camera 11 mainly observes the predetermined field of view in the front. In addition, it can also be configured to set a plurality of camera devices of the same type that observe the predetermined field of view in the side and rear. In this way, the entire surrounding area of ​​the vehicle can be observed.

[0076] Furthermore, the control unit 14 internally includes a parking control unit 15 , a wheel selection unit 16 , a movement amount calculation unit 17 , and the like.

[0077] The parking control unit 15 is a component or circuit unit that performs various processes performed when the automatic parking control is executed based on various information acquired by various sensors. Here, as various processes performed when the automatic parking control is executed, for example, in addition to searching for a parking position (i.e., a parking area) set as a parking target and generating a parking target path for guiding the host vehicle to the searched parking position, there is also a calculation process of vehicle control values ​​(control values ​​such as steering control, drive control, and brake control) when the host vehicle moves along the generated parking target path.

[0078] The wheel selection unit 16 is a component or circuit unit that selects and obtains the wheel position information with the highest reliability. The wheel selection unit 16 first calculates the predicted path of each wheel (usually four wheels) based on the parking target path generated by the parking control unit 15 and the specification information of the vehicle (such as the wheelbase value, the wheelbase value, etc.) pre-stored in a predetermined memory or the like (not shown). Next, the wheel selection unit 16 performs a calculation process to overlap the predicted path of each wheel calculated in this way with the road surface μ map (see Figure 5 ). Then, the wheel selection unit 16 selects, from among the predicted paths of the wheels, the predicted path of the wheel that passes through the area that is least likely to slip (high μ area).

[0079] Here, Figure 5 is Figure 4 An example of a display showing a parking target path and a predicted path for each wheel superimposed on a road surface μ map. Figure 5 Reference symbol R1 indicates a first parking target path of the host vehicle M toward the first parking candidate area 111 . Figure 5 Symbols R1R and R1L are predicted wheel paths of the wheels of the host vehicle M corresponding to the first parking target path R1. Symbol R1R shows the predicted wheel path of the right wheel, and symbol R1L shows the predicted wheel path of the left wheel.

[0080] Likewise, Figure 5 Symbol R2 indicates a second target path of the host vehicle M toward the second candidate parking area 112 . Figure 5 Symbols R2R and R2L are predicted wheel paths of the wheels of the host vehicle M corresponding to the second parking target path R2. Symbol R2R shows the predicted wheel path of the right wheel, and symbol R2L shows the predicted wheel path of the left wheel.

[0081] It should be noted that the wheel prediction path exists for each of the four wheels (front, rear, left, and right wheels), for example. Figure 5In the display example of FIG. 1 , in order to avoid making the drawing complicated, only two predicted paths of the left and right wheels are shown instead of displaying all the predicted paths. The display example is not limited to this, and the predicted paths of all the wheels may also be displayed.

[0082] The movement amount calculation unit 17 is a component unit or circuit unit that performs calculation processing of the estimated movement amount of the vehicle. The movement amount calculation unit 17 receives the output of the wheel speed sensor 41 to obtain the wheel speed pulse of each wheel, and calculates the movement amount (movement distance) of the reference position of the vehicle (for example, the center position of the vehicle or the center position of the rear axle of the vehicle) according to the rotation amount of each wheel. Then, the vehicle position estimation processing (odometry processing) is performed to estimate the driving trajectory based on the calculated cumulative movement distance to estimate the own position.

[0083] It should be noted that all or part of the image recognition unit 13, the control unit 14, the parking control unit 15, the wheel selection unit 16, the movement amount calculation unit 17, the CP_ECU 21, the E / G_ECU 22, the T / M_ECU 23, the BK_ECU 24, the PS_ECU 25, the positioning unit 36, etc. are composed of a processor including hardware.

[0084] Here, the processor, for example, is composed of a known structure including a non-transitory computerreadable medium, in addition to a central processing unit (CPU), RAM (Random Access Memory), ROM (Read Only Memory), a non-volatile memory, a non-volatile storage device, etc., and its peripheral devices.

[0085] Software programs executed by the CPU, fixed data such as data tables, etc. are stored in advance in the ROM, nonvolatile memory, nonvolatile storage device, etc. The CPU reads the software program stored in the ROM, etc., expands it in the RAM, and executes it. In addition, the software program realizes the functions of the above-mentioned components, components (13 to 17, 21 to 25, 36), etc. by appropriately referring to various data, etc.

[0086] In addition, the processor may be formed by a semiconductor chip such as an FPGA (Field Programmable Gate Array), etc. In addition, each of the above-mentioned components, components (13 to 17, 21 to 25, 36), etc. may also be formed by an electronic circuit.

[0087] Furthermore, the software program may be in the form of a computer program product recorded in whole or in part on a non-transitory computer readable medium such as a portable plate-type medium such as a floppy disk, a CD-ROM, or a DVD-ROM, a card-type memory, an HDD (Hard Disk Drive) device, or an SSD (Solid State Drive) device.

[0088] Below, use Figure 6 , Figure 7 The operation of the automatic parking control device 1 according to the present embodiment configured as above will be described. Figure 6 This is a flowchart showing the operation of the vehicle travel control device including the automatic parking control device according to the present embodiment. Figure 7 It is shown Figure 6 The road μ map generation process in the flowchart of Figure 6 It should be noted that in Figure 6 , Figure 7 In the following, the automatic parking control process is mainly described.

[0089] When the driving control device of the host vehicle M is activated and the automatic parking control process by the automatic parking control device 1 is started, first, Figure 6 In step S1, the control unit 14 performs a process of initializing the road surface μ map information and the vehicle position information. This initialization process is a normal process of erasing the information at the time of the last travel from the memory and initializing the memory.

[0090] Next, in step S2, the control unit 14 performs surrounding environment recognition processing based on the surrounding environment information acquired by various sensors such as the camera unit 10, the positioning unit 36, and the vehicle-mounted radar device 37. The surrounding environment recognition processing is a normal processing performed by the driving control device. The surrounding environment recognition processing is continuously performed during the driving of the vehicle M.

[0091] In step S3, the control unit 14 executes the host vehicle position estimation process through the movement amount calculation unit 17. The host vehicle position estimation process is, for example, a process (mileage process) of detecting output signals from the wheel speed sensor 41, the steering angle sensor (not shown), etc., estimating the movement amount of the host vehicle M from the initial position, and calculating the current estimated position. The host vehicle position estimation process is also continuously executed while the host vehicle M is traveling.

[0092] Next, in step S4, the control unit 14 controls the image recognition unit 13 to execute the road surface μ map generation process. Figure 7 The road surface μ map generation process will be described.

[0093] exist Figure 7 In step S21 , the control unit 14 outputs the image data acquired by the stereo camera 11 and subjected to predetermined processing by the image processing unit 12 to the AI ​​included in the image recognition unit 13 .

[0094] Next, in step S22, the control unit 14 controls the image recognition unit 13 to execute a planar depth map generation process. The planar depth map generation process generates a planar depth map by performing depth estimation (distance estimation) on the image data input to the AI ​​through image analysis (refer to Figure 2 ) processing.

[0095] Next, in step S23, the control unit 14 controls the image recognition unit 13 to execute a planar μ map generation process. The planar μ map generation process is to generate a planar μ map (see FIG. 1 ) by estimating the road surface condition and the like (road surface μ estimation) by image analysis of the image data input to the AI. Figure 3 ) processing.

[0096] In step S24 , the control unit 14 controls the image recognition unit 13 to perform object recognition processing by image analysis on the received image data.

[0097] In step S25, the control unit 14 controls the image recognition unit 13 to perform a fusion process. The fusion process generates a road surface μ map in a predetermined manner by fusing the generated plane depth map and plane μ map with the object recognition data (refer to Figure 4 ) processing.

[0098] In step S26 , the control unit 14 controls the image recognition unit 13 to update the generated road surface μ map and the self-position information, and then returns to the original processing step (return).

[0099] Return to Figure 6 In step S5, the control unit 14 controls the parking control unit 15 to perform a parking position search process based on the surrounding environment information acquired by various sensors such as the camera unit 10, the positioning unit 36, and the vehicle-mounted radar device 37. The parking position search process is a process of searching for a parking position (parking available area) set as a parking target of the host vehicle M based on the object recognition results of the parking frame, other vehicles, etc.

[0100] In step S6, the control unit 14 controls the parking control unit 15 to generate a parking target path connecting the vehicle position and the parking position based on the current estimated position information of the vehicle M obtained in the process of the above step S3 and the target parking position information obtained in the process of the above step S5 (parking target path generation process).

[0101] In step S7 , the control unit 14 controls the wheel selection unit 16 to calculate predicted paths of the four wheels based on the parking target path information acquired in the process of step S6 and the specification information of the host vehicle M (wheel predicted path calculation process).

[0102] In step S8, the control unit 14 performs wheel selection processing, in which the control unit 14 controls the wheel selection unit 16 to compare the predicted paths of each wheel obtained in the processing of the above step S7 with the road surface μ map obtained in the processing of the above step S4, and selects the wheel among the four wheels that passes through the path that is least likely to slip.

[0103] Here, as the wheel that passes the path least likely to slip, for example, the comprehensive values ​​of the entire paths of the wheels are compared, and one wheel that passes the path least likely to slip is selected. In addition to this, for example, a wheel that passes through a local area that is less likely to slip may be selected based on the moving distance (moving area) of the reference position of the vehicle (e.g., the center position of the vehicle or the center position of the rear axle of the vehicle), and the wheel may be switched for each area.

[0104] Specifically, for example, the left front wheel may be selected when the moving distance is 1 to 5 m, and the right rear wheel may be selected when the next moving distance is 6 to 10 m, thereby switching the selected wheel for each area in the entire path.

[0105] In step S9, the control unit 14 controls the movement amount calculation unit 17 to obtain the wheel speed pulse (output of the wheel speed sensor 41) of the wheel selected in the process of step S8. Then, based on the obtained data, the movement distance of the reference position of the vehicle (for example, the center position of the vehicle or the center position of the rear axle of the vehicle) is calculated. The calculation result is sent to the parking control unit 15.

[0106] In step S10, the control unit 14 controls the parking control unit 15 to execute a vehicle control value calculation process that calculates vehicle control values ​​(control values ​​such as steering control, drive control, and brake control) for automatically driving the host vehicle M along the set parking target path.

[0107] In step S11 , the control unit 14 executes the automatic parking control process of the host vehicle M based on the vehicle control value calculated in step S10 . Then, when the parking of the host vehicle M at the target parking position is completed, a series of processes are terminated (END).

[0108] As described above, according to the above-mentioned embodiment, in the automatic parking control device 1 mounted on a vehicle such as an automobile, Based on the image data as the surrounding environment information acquired by the camera unit 10 as the surrounding environment recognition device, the surrounding environment is recognized and the road surface condition is estimated. The wheel prediction path that is least likely to slip is selected based on the road surface condition estimated in this way, and the movement amount of the host vehicle M is predicted based on the selected wheel prediction path, so that high-precision vehicle movement amount information can be obtained, thereby achieving high-precision vehicle control.

[0109] In addition, although the structure is such that the vehicle movement amount is predicted using wheel speed pulses when executing automatic parking control, the following performance of the parking target path is not reduced even on slippery road conditions such as snowy roads. That is, the number of U-turns and the deviation of the final target position can be reduced, thereby improving the quality of automatic parking control.

[0110] It should be noted that when describing the structure and function of the automatic parking control device according to the above embodiment, the description is mainly based on the case where the automatic parking control is performed. However, the function of the automatic parking control device according to the present embodiment is not limited to the above example. The control performed by the automatic parking control device according to the present embodiment can also be applied when performing parking assist control to assist the parking operation performed by the driver.

[0111] In addition, in the above-mentioned embodiment, an example of generating and displaying a plurality of parking target paths (R1, R2) is shown. In this case, when a plurality of parking target paths are generated, it is necessary to determine which parking target path to select. In this case, for example, when displaying the generated plurality of parking target paths, it is better to set the path that is less likely to slip among the plurality of parking target paths as a recommended path and display it in a recognizable manner. In this case, for example, it is considered to adopt a display method that is visually recognizable, such as coloring the recommended path. In addition, it is also possible to adopt a method in which the driver manually operates to select a desired parking target path on such a display screen displaying a plurality of parking target paths. If such a method is adopted, the driver can easily select the target parking path that is less likely to slip. In addition, or alternatively, it is also possible to adopt a method in which a predetermined judgment criterion is set and the automatic selection control performed by the control unit 14 is performed according to the judgment criterion.

[0112] The present invention is not limited to the above-mentioned embodiments, and various modifications and / or applications can be implemented without departing from the scope of the main purpose of the invention. In addition, in the above-mentioned embodiments, inventions of various stages are included, and various inventions can be extracted by appropriate combinations of the disclosed multiple constituent elements. For example, even if several constituent elements are deleted from all the constituent elements shown in one of the above-mentioned embodiments, the structure from which the constituent elements are deleted can be extracted as an invention if it can solve the problem to be solved by the invention and obtain the effect of the invention. In addition, the constituent elements in different embodiments can also be appropriately combined. The present invention is not restricted by its specific embodiments except as limited by the attached claims.

Claims

1. An automatic parking control device, characterized in that: The automatic parking control device performs driving control for guiding the vehicle to a parking area, and includes: A surrounding environment recognition device, which obtains the surrounding environment information of the vehicle; a parking control unit that searches for a parking area based on the surrounding environment information acquired by the surrounding environment recognition device, generates a target path for moving the vehicle with the searched parking area as a target arrival point, and performs driving control for moving the vehicle along the generated target path; an image processing unit that estimates the distance to the recognized three-dimensional object and the degree of slippage of the road surface based on the surrounding environment information acquired by the surrounding environment recognition device; an image map generating unit for generating a road surface friction coefficient map by synthesizing the information of the distance and the information of the slip difficulty of the road surface estimated by the image processing unit with the surrounding environment information; a wheel selection unit, which predicts the paths that the multiple wheels of the vehicle pass through respectively based on the target path, makes the predicted paths of each wheel coincide with the road friction coefficient map, and selects the wheel that passes through the path that is least likely to slip among the multiple wheels; as well as The movement amount calculation unit acquires the wheel speed pulse of the selected wheel and calculates the movement amount of the reference position of the vehicle.

2. The automatic parking control device according to claim 1, characterized in that: The surrounding environment recognition device is a vehicle-mounted camera device that acquires surrounding environment information of the vehicle as image data.

3. The automatic parking control device according to claim 2, characterized in that: The image processing unit estimates the distance to the recognized three-dimensional object and the degree of slippage of a road surface based on the image data obtained by capturing a predetermined range in front of the vehicle.

4. The automatic parking control device according to claim 2, characterized in that: The image map generation unit generates a road friction coefficient map that is displayed by fusing a plane depth map, a plane friction coefficient map, and object recognition data of a three-dimensional object recognized based on the received image data. The planar depth map is a map that estimates the depth of the identified three-dimensional object based on the received image data, and displays the estimated depth information on the planar image as depth distribution information. The plane friction coefficient map estimates the road surface condition based on the received image data and displays the map in a manner that allows the slip difficulty of the road surface to be determined for each predetermined area.

5. The automatic parking control device according to claim 1, characterized in that: The wheel selection unit selects one wheel based on the integrated value of the entire path of each wheel, or selects a wheel that passes through an area that is less likely to slip for each area in the entire path.

Images