Parking control method and device, electronic equipment and storage medium

Abstract

The invention provides a parking control method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to the starting of a vehicle parking function, recognizing the leaving feature information of a target user, calculating the leaving confidence coefficient of the target user through the leaving feature information, carrying out the target tracking of the target user, obtaining the moving track of the target user outside a vehicle, and carrying out the parking control of the vehicle. The vehicle leaving confidence coefficient of the target user is calculated according to the moving track, the seat occupation value of the vehicle is detected, the seat occupation value, the seat leaving confidence coefficient and the vehicle leaving confidence coefficient are fused to obtain the parking feasibility, and under the condition that the parking feasibility is larger than or equal to a preset threshold value, the vehicle is controlled to execute the parking function. The vehicle leaving behavior of a driver and passengers during automatic parking of the vehicle is accurately recognized, the accurate and effective parking feasibility degree is obtained through multi-sensor data fusion and a confidence coefficient calculation mechanism, the vehicle parking function is automatically activated after it is determined that the driver is away from the vehicle and safe, and the parking safety and reliability are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle control, and particularly to a parking control method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of automotive intelligent technologies, the Automatic Parking Assist (APA) function and the out-of-vehicle parking function have gradually become standard features of intelligent vehicles. The automatic parking function is an advanced driver assistance system function in intelligent vehicles, aiming to automatically complete the parking operation through the vehicle's own sensors, control systems and algorithms, reducing the driver's operation burden. The out-of-vehicle parking function is an extension of the automatic parking function, which supports the driver to start parking using a mobile phone or key after getting out of the vehicle, and the vehicle completes the automatic parking operation, further enhancing the convenience of parking.

[0003] Currently, after the out-of-vehicle parking function is enabled, it usually requires the driver to confirm again through a mobile phone terminal, key, remote APP, physical button or blocking the camera outside the vehicle before triggering. After the system receives the driver's operation instruction, it will start the parking operation. However, the mis-touch rate of this function that requires the user to confirm again is relatively high. When the driver has not really left the vehicle or is not in a safe area, the system may be wrongly activated, resulting in potential safety hazards in vehicle parking, affecting the safety and accuracy of parking, and further affecting the user experience. Summary of the Invention

[0004] In view of this, the present invention aims to provide a parking control method, device, electronic device and storage medium to solve the problem that the current out-of-vehicle parking function may be wrongly activated when the driver has not really left the vehicle or is not in a safe area, resulting in potential safety hazards in vehicle parking and affecting the safety and accuracy of parking.

[0005] According to a first aspect of the present invention, there is provided a parking control method, the method comprising:

[0006] In response to the start of the vehicle parking function, identifying the off-seat feature information of the target user, and calculating the off-seat confidence of the target user using the off-seat feature information;

[0007] Performing target tracking on the target user to obtain the moving trajectory of the target user outside the vehicle, and calculating the off-vehicle confidence of the target user according to the moving trajectory;

[0008] Detecting the seat occupancy value of the vehicle, and fusing the seat occupancy value, the off-seat confidence and the off-vehicle confidence to obtain the parking feasibility;

[0009] When the parking feasibility is greater than or equal to a preset threshold, control the vehicle to execute the parking function.

[0010] Optionally, in response to the activation of the vehicle parking function, identifying the off-seat feature information of the target user, and calculating the off-seat confidence of the target user by using the off-seat feature information, includes:

[0011] In response to the activation of the vehicle parking function, detecting a preset area to determine the target user to be identified;

[0012] Performing behavior recognition on the target user to obtain the off-seat feature information of the target user; wherein, the off-seat feature information includes face detection results, head pose detection results, and off-seat behavior detection results;

[0013] Calculating the off-seat confidence of the target user by using the face detection results, the head pose detection results, and the off-seat behavior detection results.

[0014] Optionally, the performing behavior recognition on the target user to obtain the off-seat feature information of the target user includes:

[0015] Performing face detection on the target user to determine the face size and the first confidence level that appear in the field of view of the preset camera, and obtaining the face detection results;

[0016] Performing head pose recognition on the target user to determine the head pose offset angle and the second confidence level, and obtaining the head pose detection results;

[0017] Performing off-seat behavior detection on the target user to determine whether the target user is off-seat, and obtaining the off-seat behavior detection results.

[0018] Optionally, the performing target tracking on the target user to obtain the movement trajectory of the target user outside the vehicle, and calculating the off-vehicle confidence of the target user according to the movement trajectory, includes:

[0019] Obtaining the body characteristics of the target user, and performing target tracking on the target user according to the body characteristics;

[0020] Collecting the trajectory points of the target user, and generating the movement trajectory of the target user outside the vehicle according to the trajectory points;

[0021] Calculating the distance between the trajectory points in the movement trajectory and the center position of the vehicle to obtain the relative distance between the target user and the vehicle;

[0022] Calculating the off-vehicle confidence of the target user according to the relative distance between the target user and the vehicle.

[0023] Optionally, detecting the seat occupancy value of the vehicle, and fusing the seat occupancy value, the confidence of leaving the seat, and the confidence of getting out of the vehicle to obtain the parking feasibility, including:

[0024] Detecting the driver's seat of the vehicle to determine whether the driver's seat is occupied, and obtaining the seat occupancy value of the vehicle;

[0025] Predetermining the weight coefficients of the seat occupancy value, the confidence of leaving the seat, and the confidence of getting out of the vehicle;

[0026] Weightedly fusing the seat occupancy value, the confidence of leaving the seat, and the confidence of getting out of the vehicle according to the weight coefficients to obtain the parking feasibility.

[0027] Optionally, in the case where the parking feasibility is greater than or equal to a preset threshold, controlling the vehicle to execute the parking function, including:

[0028] Comparing the parking feasibility with a preset threshold determined in advance to obtain a comparison result;

[0029] In the case where the comparison result is that the parking feasibility is greater than or equal to the preset threshold, controlling the vehicle to execute the parking function.

[0030] Optionally, after comparing the parking feasibility with a preset threshold determined in advance to obtain a comparison result, further including:

[0031] In the case where the comparison result is that the parking feasibility is less than the preset threshold, sending a prompt message to the target user, where the prompt message is used to prompt the target user that the parking function of the vehicle cannot be executed.

[0032] According to a second aspect of the present invention, there is provided a parking control device, and the device includes:

[0033] A target recognition module, configured to, in response to the activation of the vehicle parking function, recognize the leaving-seat feature information of the target user, and calculate the confidence of the target user leaving the seat by using the leaving-seat feature information;

[0034] A target tracking module, configured to perform target tracking on the target user to obtain the moving trajectory of the target user outside the vehicle, and calculate the confidence of the target user getting out of the vehicle according to the moving trajectory;

[0035] A feasibility calculation module, configured to detect the seat occupancy value of the vehicle, and fuse the seat occupancy value, the confidence of leaving the seat, and the confidence of getting out of the vehicle to obtain the parking feasibility;

[0036] A parking control module, configured to control the vehicle to perform the parking function when the parking feasibility is greater than or equal to a preset threshold.

[0037] According to another aspect of the present invention, there is also provided an electronic device, including:

[0038] A processor;

[0039] A memory for storing instructions executable by the processor;

[0040] Wherein, the processor is configured to execute the instructions to implement the parking control method as described above.

[0041] According to another aspect of the present invention, there is also provided a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the parking control method as described above are implemented.

[0042] The parking control method provided by the embodiments of the present invention, by responding to the activation of the vehicle parking function, identifying the off-seat feature information of the target user, calculating the off-seat confidence of the target user using the off-seat feature information, performing target tracking on the target user to obtain the movement trajectory of the target user outside the vehicle, calculating the off-vehicle confidence of the target user according to the movement trajectory, detecting the seat occupancy value of the vehicle, and fusing the seat occupancy value, the off-seat confidence, and the off-vehicle confidence to obtain the parking feasibility. When the parking feasibility is greater than or equal to a preset threshold, the vehicle is controlled to perform the parking function. The embodiments of the present invention accurately identify the off-vehicle behavior of the driver and passengers during vehicle automatic parking, use multi-sensor data fusion and confidence calculation mechanisms to obtain accurate and effective parking feasibility, automatically activate the vehicle parking function after determining that the driver is far away from the vehicle and safe, avoid mis-triggering of the current off-vehicle parking function when the driver has not really left the vehicle or is not in a safe area, eliminate the need for secondary operation by the driver, significantly reduce the mis-triggering rate, reduce the safety hazards of vehicle parking, improve the safety and reliability of parking, and further enhance the user experience.

[0043] The above description is only an overview of the technical solution of the present invention. In order to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are specifically exemplified below. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0045] Figure 1 is a flowchart of the steps of a parking control method provided by an embodiment of the present invention;

[0046] Figure 2 is Figure 1 a flowchart of step 101 in a parking control method provided by an embodiment of the present invention;

[0047] Figure 3 is Figure 1 a flowchart of step 102 in a parking control method provided by an embodiment of the present invention;

[0048] Figure 4 is Figure 1 a flowchart of step 103 in a parking control method provided by an embodiment of the present invention;

[0049] Figure 5 is Figure 1 a flowchart of step 104 in a parking control method provided by an embodiment of the present invention;

[0050] Figure 6 is a schematic diagram of the scenario of a parking control method provided by an embodiment of the present invention;

[0051] Figure 7 is a schematic structural diagram of a parking control device provided by an embodiment of the present invention;

[0052] Figure 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed Embodiments

[0053] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the following will elaborate on various implementation manners of the present invention with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that in various embodiments of the present invention, many technical details are presented for the convenience of readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented. The following division of each embodiment is for convenience of description and should not constitute any limitation to the specific implementation manner of the present invention. Each embodiment can be combined and cross-referenced with each other without conflict.

[0054] Referring to Figure 1 , a flowchart of the steps of a parking control method provided by an embodiment of the present invention is shown, and the method may include:

[0055] Step 101, in response to the activation of the vehicle parking function, identify the off-seat characteristic information of the target user, and calculate the off-seat confidence level of the target user using the off-seat characteristic information.

[0056] In an embodiment of the present invention, to solve the problem that after the current off-vehicle parking function of automatic parking is enabled, it usually requires the driver to manually confirm outside the vehicle twice before starting the parking operation, with a relatively high misoperation rate. When the driver has not really left the vehicle or is not in a safe area, the system is wrongly activated, resulting in potential safety hazards in vehicle parking and affecting the safety and accuracy of parking. In this embodiment, by fusing the detection results of in-vehicle and out-vehicle sensors (seat occupancy sensors, in-vehicle cameras, out-vehicle sensors, and out-vehicle cameras), after accurately identifying that the driver has left the vehicle, the vehicle parking function is automatically activated to achieve safe and reliable automatic parking of the vehicle.

[0057] It should be noted that referring to Figure 6 , a schematic diagram of the scenario of a parking control method provided by an embodiment of the present invention is shown. The vehicle in the embodiment of the present invention includes in-vehicle sensors, in-vehicle cameras, out-vehicle sensors, and out-vehicle cameras. Among them, the in-vehicle sensors include seat occupancy sensors, and the in-vehicle cameras include DMS cameras (driver detection), PMS cameras (occupant detection), etc. The out-vehicle sensors include induction devices such as laser millimeter radars. The parking control system receives the sensor data collected by the in-vehicle and out-vehicle sensors, performs user identification, distance detection, and data fusion to obtain the analysis result that the user has left the vehicle, so as to control the vehicle to perform the automatic parking function.

[0058] Specifically, in response to the start of the vehicle parking function, the vehicle's parking control system identifies the off-seat characteristic information of the target user through sensors and calculates the off-seat confidence of the target user using the off-seat characteristic information. The vehicle's parking control system monitors the enabled state of the vehicle parking function in real time. In response to the start of the vehicle parking function, it detects a preset area to determine the target user to be identified. Among them, the start of the parking function can be manually turned on by the driver during the driving process of the vehicle, or the vehicle automatically triggers the to-be-activated state of intelligent driving after being powered on, which will not be elaborated here one by one.

[0059] In an embodiment of the present invention, the parking control system identifies the behavior of the target user through an in-vehicle camera, extracts off-seat feature information, and the off-seat feature information includes a face detection result, a head pose detection result, and an off-seat behavior detection result. Among them, the face detection result is used to determine whether the face of the target user appears in the camera's field of view; the head pose detection result is used to calculate the deviation angle of the target user's head from the due front. The deviation angle is the angle at which the target user deviates towards the outside of the vehicle or away from the seat, and is used to indicate that the driver tends to move towards the outside of the vehicle or away from the seat. The larger the deviation angle, the higher the confidence level; the off-seat behavior detection result is used to detect whether the buttocks of the target user leave the seat area. The face detection result, the head pose detection result, and the off-seat behavior detection result are used to calculate the off-seat confidence level of the target user through weighted fusion.

[0060] Step 102: Perform target tracking on the target user to obtain the movement trajectory of the target user outside the vehicle, and calculate the off-vehicle confidence level of the target user according to the movement trajectory.

[0061] In an embodiment of the present invention, to determine the behavior of the target user after getting off the seat and accurately identify whether the target user has left the vehicle and is in a safe position, the parking control system of the vehicle uses an external camera and an external sensor to perform target tracking on the target user, obtains the movement trajectory of the target user outside the vehicle, and calculates the off-vehicle confidence level of the target user according to the movement trajectory.

[0062] Specifically, the system uses a target tracking algorithm to obtain the body characteristics of the target user, performs target tracking on the target user according to the body characteristics, collects the trajectory points of the target user, generates the movement trajectory of the target user outside the vehicle according to the trajectory points, calculates the distance between the trajectory points in the movement trajectory and the center position of the vehicle to obtain the relative distance between the target user and the vehicle. Through the analysis of the distance between the person and the vehicle, the system can judge the movement direction and off-vehicle position of the target user, and thus calculate the off-vehicle confidence level of the target user according to the relative distance between the target user and the vehicle. Among them, the value range of the off-vehicle confidence level is [0, 1]. According to the off-vehicle confidence level of the target user, it can be accurately judged whether the target user is moving away from the vehicle.

[0063] Step 103: Detect the seat occupancy value of the vehicle, and fuse the seat occupancy value, the off-seat confidence level, and the off-vehicle confidence level to obtain the parking feasibility.

[0064] In the embodiment of the present invention, the driver's seat of the vehicle is detected to determine whether the driver's seat is occupied, and the seat occupancy value of the vehicle is obtained. Then, the seat occupancy value, the off-seat confidence, and the off-vehicle confidence are fused to obtain the parking feasibility. The fusion is a weighted sum according to the weight coefficients of each parameter, and the weight coefficients can be dynamically adjusted according to different scenarios and user behaviors. The seat occupancy value, the off-seat confidence, and the off-vehicle confidence are weighted and fused according to the weight coefficients to obtain the parking feasibility. The value range of the parking feasibility is [0, 1], and the closer the value is to 1, the higher the parking feasibility.

[0065] Step 104, when the parking feasibility is greater than or equal to a preset threshold, control the vehicle to execute the parking function.

[0066] In the embodiment of the present invention, a multi-sensor data fusion and weighted calculation mechanism is adopted to obtain an accurate parking feasibility. Then, the parking feasibility is compared with a preset threshold determined in advance. When the parking feasibility is greater than or equal to the preset threshold, control the vehicle to execute the parking function, and automatically activate the parking function when it is confirmed that the target user is safe, so as to realize safe and reliable automatic parking of the vehicle.

[0067] The parking control method provided by the embodiment of the present invention, by responding to the start of the vehicle parking function, identifies the off-seat characteristic information of the target user, calculates the off-seat confidence of the target user by using the off-seat characteristic information, performs target tracking on the target user to obtain the moving trajectory of the target user outside the vehicle, calculates the off-vehicle confidence of the target user according to the moving trajectory, detects the seat occupancy value of the vehicle, fuses the seat occupancy value, the off-seat confidence, and the off-vehicle confidence to obtain the parking feasibility, and when the parking feasibility is greater than or equal to a preset threshold, controls the vehicle to execute the parking function. The embodiment of the present invention accurately identifies the off-vehicle behavior of the driver and passengers during vehicle automatic parking, uses a multi-sensor data fusion and confidence calculation mechanism to obtain accurate and effective parking feasibility, automatically activates the vehicle parking function after determining that the driver is far away from the vehicle and safe, avoids mis-triggering of the current off-vehicle parking function when the driver has not really left the vehicle or is not in a safe area, eliminates the need for secondary operation by the driver, significantly reduces the mis-triggering rate, reduces the safety hazards of vehicle parking, improves the safety and reliability of parking, and further enhances the user experience.

[0068] Further, referring to Figure 2 , shows Figure 1 The flowchart of step 101 in a provided parking control method, this method is basically the same as the parking control method provided by the first embodiment of the present invention, and step 101 may include:

[0069] Step 1011, in response to the start of the vehicle parking function, detect a preset area to determine the target user to be identified;

[0070] Step 1012: Perform behavior recognition on the target user to obtain the off-seat feature information of the target user; wherein, the off-seat feature information includes the face detection result, the head pose detection result, and the off-seat behavior detection result.

[0071] Step 1013: Calculate the off-seat confidence of the target user by using the face detection result, the head pose detection result, and the off-seat behavior detection result.

[0072] In the embodiment of the present invention, the parking control system monitors the enabling state of the vehicle parking function in real time. In response to the start of the vehicle parking function, it detects a preset area to determine the target user to be recognized. Specifically, when the vehicle parking function is started, the system performs real-time detection on the preset area through the in-vehicle camera to identify whether there is a target user in the preset area, and can perform real-time analysis on the in-vehicle image to identify the target user in the driver's seat area. The preset area is the camera view area of the in-vehicle camera, which can be the driver's seat area, usually including the driver's seat and its surrounding areas, ensuring that the activity range of the driver is covered. The target user is the vehicle occupants pre-entered into the vehicle system, which is not specifically limited here.

[0073] In this embodiment, if the system detects the target user, it marks the target user as the one to be recognized and starts collecting its behavior characteristics. If the target user is not detected, the system determines that the driver has left the vehicle and can directly enter the parking function execution stage. By quickly and accurately identifying the target user in the driver's seat area, it provides a basis for subsequent behavior recognition. Specifically, the system identifies the behavior of the target user through the in-vehicle camera and extracts the off-seat feature information. The off-seat feature information includes the face detection result, the head pose detection result, and the off-seat behavior detection result. Among them, the face detection result is used to determine whether the face of the target user appears in the camera view; the head pose detection result is used to calculate the offset angle between the head of the target user and the due front. The offset angle is the angle at which the target user deviates towards the outside of the vehicle or away from the seat, which is used to indicate that the driver tends to the outside of the vehicle or away from the seat. The larger the offset angle, the higher the confidence; the off-seat behavior detection result is used to detect whether the buttocks of the target user leave the seat area.

[0074] It should be noted that in this embodiment, a deep learning-based face detection algorithm can be used to detect the face of the target user in real time to obtain the face detection result, a head pose estimation algorithm can be used to calculate the offset angle of the head of the target user to obtain the head pose detection result, and based on human key point detection, the key point position of the buttocks of the target user is identified to determine whether it leaves the seat area to obtain the off-seat behavior detection result.

[0075] Specifically, the face detection result, head pose detection result, and out-of-seat behavior detection result are all confidence levels obtained based on detection calculations. In the embodiments of the present invention, an in-vehicle camera is used to detect the target user, including face detection, head pose detection, and out-of-seat behavior detection, to capture various feature information of the target user, so as to comprehensively calculate the confidence level of the driver leaving the seat. Among them, a face detection algorithm is used to calculate whether the driver's face appears in the camera's field of view, and the face detection result F_d is obtained. The value range of the face detection result F_d is [0, 1]. If the face completely appears in the camera's field of view, the face detection result F_d takes the value of 1. If the face does not appear in the camera's field of view, the face detection result F_d takes the value of 0. The head pose detection result H_p is determined by calculating the head pose offset angle. The head pose offset angle can be a continuous value, and the value range is from 0 degrees to 90 degrees. The larger the head pose offset angle, the more inclined the driver is to the outside of the vehicle or leaving the seat, and the higher the head pose detection result H_p. The out-of-seat behavior detection result S_b is determined by detecting whether the driver's buttocks leave the seat. If the driver's buttocks leave the seat area, the out-of-seat behavior detection result S_b takes the value of 1. Otherwise, the out-of-seat behavior detection result S_b takes the value of 0.

[0076] In this embodiment, the face detection result, head pose detection result, and out-of-seat behavior detection result are used to calculate the out-of-seat confidence level of the target user. According to the face detection result F_d, head pose detection result H_p, and out-of-seat behavior detection result S_b, the out-of-seat confidence level C_in of the target user is calculated. Specifically, it is calculated through the following formula:

[0077] C_in = w1 * F_d + w2 * H_p + w3 * S_b

[0078] Among them, the value range of C_in is [0, 1]. The closer the value is to 1, the higher the confidence level that the target user leaves the seat. w1, w2, and w3 are the weight coefficients of the face detection result F_d, head pose detection result H_p, and out-of-seat behavior detection result S_b. The sum of w1, w2, and w3 is 1. The specific values of the weight coefficients are dynamically adjusted according to the importance of face detection, head detection, and out-of-seat behavior detection in the in-vehicle test dataset to adapt to different scenarios and user behaviors. For example, w1 = 0.4, w2 = 0.2, w3 = 0.4. No specific limitation is made here.

[0079] In the embodiments of the present invention, through face detection, head pose detection, and out-of-seat behavior detection, the behavior characteristics of the target user are comprehensively captured, the accuracy of out-of-seat recognition is improved, and through multi-feature weighted fusion, the out-of-seat confidence level of the target user is accurately calculated, the misjudgment rate is reduced, and the accuracy of target user behavior recognition is improved.

[0080] Specifically, in step 1012, behavior recognition is performed on the target user to obtain the off-seat feature information of the target user, which may specifically include:

[0081] Sub-step 01: Perform face detection on the target user to determine the face size and the first confidence level that appear in the preset camera field of view, and obtain the face detection result;

[0082] Sub-step 02: Perform head pose recognition on the target user to determine the head pose deviation angle and the second confidence level, and obtain the head pose detection result;

[0083] Sub-step 03: Perform off-seat behavior detection on the target user to determine whether the target user is off-seat, and obtain the off-seat behavior detection result.

[0084] It should be noted that in the above steps, for face detection of the target user, the face size and the first confidence level that appear in the preset camera field of view are determined to obtain the face detection result. The face size and the first confidence level are obtained by capturing an image of the target user through an in-vehicle camera and analyzing the image to detect the target user. The face size is the pixel size of the face in the image captured by the camera field of view, and the first confidence level is used to represent the credibility of the target user's face appearing in the camera field of view, with a value range of [0, 1].

[0085] Specifically, for head pose recognition of the target user, the head pose deviation angle and the second confidence level are determined to obtain the head pose detection result. Among them, the head pose deviation angle is the head deviation angle of the target user calculated by the head pose estimation algorithm, specifically the angle between the driver's head and the due front. The maximum deviation angle is determined in advance, and it is set that a 90-degree left deviation is the maximum angle, indicating a complete lateral or back-facing out of the vehicle. The second confidence level is determined according to the ratio of the head pose deviation angle to the maximum deviation angle. The second confidence level is used to represent the credibility of the head pose detection, with a value range of [0, 1]. The larger the head pose deviation angle, the more inclined the target user is to leave the vehicle, and the closer the second confidence level is to 1.

[0086] It should be noted that the head pose detection result H_p, that is, the second confidence level, is determined according to the ratio of the head pose deviation angle to the maximum deviation angle. The specific calculation formula of the head pose detection result H_p is as follows:

[0087]

[0088] Specifically, for the target user, off-seat behavior detection is performed to determine whether the target user gets off the seat, and the off-seat behavior detection result is obtained. The behavior of the target user is analyzed using the in-vehicle camera and human key point detection, and the off-seat behavior detection result is output. The off-seat behavior detection result is used to represent the off-seat behavior of the target user. The key point position of the target user's buttocks is identified, and it is judged whether it leaves the seat area. If the buttocks key point is outside the seat area, it is determined as an off-seat behavior; otherwise, it is determined as not getting off the seat. The off-seat behavior detection result adopts a binary value, where 1 indicates that the target user gets off the seat and 0 indicates that the target user does not get off the seat.

[0089] Through face detection, head pose recognition, and off-seat behavior detection in the embodiments of the present invention, the behavior characteristics of the target user can be comprehensively captured, the accuracy of off-seat recognition can be improved, and the off-seat state of the target user can be accurately judged by combining multi-modal data such as face size, head pose offset angle, and off-seat behavior.

[0090] Further, referring to Figure 3 shows Figure 1 The flowchart of step 102 in a parking control method provided, which is basically the same as the parking control method provided in the first embodiment of the present invention. Step 102 may include:

[0091] Step 1021, obtain the physical characteristics of the target user, and perform target tracking on the target user according to the physical characteristics;

[0092] Step 1022, collect the trajectory points of the target user, and generate the moving trajectory of the target user outside the vehicle according to the trajectory points;

[0093] Step 1023, calculate the distance between the trajectory points in the moving trajectory and the vehicle center position to obtain the relative distance between the target user and the vehicle;

[0094] Step 1024, calculate the off-vehicle confidence of the target user according to the relative distance between the target user and the vehicle.

[0095] In the embodiments of the present invention, to further determine the behavior of the target user after getting off the seat and accurately identify whether the target user gets off the vehicle and is in a safe position, the system performs trajectory tracking on the target user, obtains the physical characteristics of the target user, and performs target tracking on the target user according to the physical characteristics. Specifically, an off-vehicle camera can be used to capture the image of the target user, and the physical characteristics of the target user are identified. The physical characteristics include the position and size of the target user in the image and key point information. The key point information includes the key point positions of the head, shoulders, buttocks, etc. of the target user. The target tracking algorithm (such as DeepSORT) is used to continuously track the target user, and the target user is matched in consecutive frames according to the physical characteristics to generate a tracking trajectory.

[0096] It should be noted that a pre-trained deep learning model (such as YOLOv5) is used to detect the physical characteristics of the target user in real time, and the Kalman filter and Hungarian algorithm are combined to achieve cross-frame matching and trajectory generation of the target user. The target tracking algorithm based on physical characteristics can accurately match the target user to ensure the continuity and stability of tracking.

[0097] Specifically, during the target tracking process, the system continuously collects the trajectory points of the target user. Each trajectory point represents the position of the target user in the image. Based on the collected trajectory points, the moving trajectory of the target user outside the vehicle is generated. The interpolation algorithm (such as linear interpolation or spline interpolation) is used to smooth the trajectory points to generate a continuous moving trajectory. Through trajectory point collection and interpolation algorithm, the system can generate the continuous moving trajectory of the target user, providing a basis for distance calculation.

[0098] In this embodiment, the distance between the trajectory points in the moving trajectory and the center position of the vehicle is calculated to obtain the relative distance between the target user and the vehicle. Among them, the center position of the vehicle is determined by calculating the position of the vehicle center in the image according to the vehicle body size and camera parameters. The relative distance is determined by calculating the Euclidean distance between each trajectory point and the center position of the vehicle. Through distance trend analysis, the system can judge the moving direction of the target user and improve the accuracy of off-vehicle recognition.

[0099] In this embodiment, according to the relative distance between the target user and the vehicle, the off-vehicle confidence level C of the target user is calculated _out , and the specific calculation formula is as follows:

[0100]

[0101] Among them, D represents the relative distance, and the off-vehicle confidence level C _out ranges from [0, 1], and the closer the value is to 1, the higher the confidence level that the target user is getting off the vehicle.

[0102] In the embodiment of the present invention, through the target tracking algorithm based on physical characteristics, the target user is accurately matched to ensure the continuity and stability of tracking. By using the relative distance and confidence level calculation between the vehicle and the person, it is accurately determined whether the target user is moving away from the vehicle.

[0103] Further, referring to Figure 4 , shows Figure 1 the flowchart of step 103 in a parking control method provided. This method is basically the same as the parking control method provided in the first embodiment of the present invention. Step 103 may include:

[0104] Step 1031, detect the driver's seat of the vehicle to determine whether the driver's seat is occupied, and obtain the seat occupancy value of the vehicle;

[0105] Step 1032: Predetermine the weight coefficients of the seat occupancy value, the confidence of getting off the seat, and the confidence of getting out of the vehicle.

[0106] Step 1033: Weightedly fuse the seat occupancy value, the confidence of getting off the seat, and the confidence of getting out of the vehicle according to the weight coefficients to obtain the parking feasibility.

[0107] In the embodiment of the present invention, the driver's seat of the vehicle is detected to determine whether the driver's seat is occupied, and the seat occupancy value of the vehicle is obtained. Among them, when detecting the driver's seat of the vehicle, a seat occupancy sensor (such as a pressure sensor) can be used to detect whether the driver's seat is occupied. By detecting the pressure distribution on the seat, it is judged whether someone is sitting on the seat, or the capacitance change on the seat surface is detected to judge whether someone touches the seat, and the seat occupancy value S is output. The seat occupancy value takes binary values, 1 means the seat is occupied, and 0 means the seat is not occupied. According to the real vehicle test data set, the weight coefficients α, β, and γ of the seat occupancy value S, the confidence of getting off the seat C_in, and the confidence of getting out of the vehicle C_out are determined. The sum of the weight coefficients α, β, and γ is 1, and the weight values of α, β, and γ can be adjusted according to actual needs, and can be learned by using a linear regression model based on the real vehicle test data set, which is not specifically limited here.

[0108] It should be noted that a large amount of real vehicle test data is collected in advance, including the true labels of the seat occupancy value, the confidence of getting off the seat, the confidence of getting out of the vehicle, and the parking feasibility. A linear regression model is used to train the data set to learn the weight coefficients, scientifically allocate the weight coefficients, and improve the accuracy of calculating the parking feasibility. Among them, the weight coefficients can be dynamically adjusted according to different scenarios and user behaviors to improve the adaptability of the system.

[0109] Specifically, the seat occupancy value, the confidence of getting off the seat, and the confidence of getting out of the vehicle are weightedly fused according to the weight coefficients to obtain the parking feasibility. Specifically, the seat occupancy value S, the confidence of getting off the seat C_in, and the confidence of getting out of the vehicle C_out are weightedly fused to obtain the parking feasibility Decision. The parking feasibility Decision is calculated by the following formula:

[0110] Decision = α * (1 - S) + β * C_in + γ * C_out

[0111] Among them, 1 - S represents the confidence that the seat is not occupied. α, β, and γ are the weight coefficients of the seat occupancy value S, the confidence of getting off the seat C_in, and the confidence of getting out of the vehicle C_out. The value range of Decision is [0, 1]. The closer the value is to 1, the higher the parking feasibility.

[0112] In the embodiments of the present invention, by comprehensively considering the seat occupancy, the off - seat state, and the off - vehicle state, the parking feasibility is accurately calculated. By using multi - sensor data fusion and a weighted calculation mechanism, the accuracy of calculating the parking feasibility is improved.

[0113] Further, referring to Figure 5 shows the flowchart of step 104 in a parking control method provided by Figure 1 This method is basically the same as the parking control method provided by the first embodiment of the present invention. Step 104 may include:

[0114] Step 1041: Compare the parking feasibility with a pre - determined preset threshold to obtain a comparison result;

[0115] Step 1042: When the comparison result is that the parking feasibility is greater than or equal to the preset threshold, control the vehicle to execute the parking function.

[0116] In the embodiments of the present invention, the parking feasibility is compared with a pre - determined preset threshold to obtain a comparison result. The comparison result is divided into two cases: if the parking feasibility is greater than or equal to the preset threshold, it means that the parking function can be executed; otherwise, it means that the parking function cannot be executed. The preset threshold is optimized through a real - vehicle test data set. The selection of the threshold needs to balance safety and user experience. A higher threshold can improve safety but may reduce user experience, while a lower threshold can improve user experience but may increase safety risks. In this embodiment, the preset threshold is set according to actual needs and is not specifically limited here.

[0117] In this embodiment, when the comparison result is that the parking feasibility is greater than or equal to the preset threshold, control the vehicle to execute the parking function. The system automatically activates the parking function. The parking function includes the following steps: using sensors to identify available parking spaces, calculating the optimal parking path according to the parking space position and the vehicle state, and the system automatically controls the steering, acceleration, and braking of the vehicle to complete the parking operation.

[0118] In the embodiments of the present invention, comparing the parking feasibility with the preset threshold can accurately determine whether to execute the parking function. By actively identifying the driver's off - vehicle behavior, after accurately identifying that the driver has moved away from the vehicle, the vehicle parking function is automatically activated, realizing safe and reliable automatic parking of the vehicle, reducing driver operations, and further improving the user experience.

[0119] Specifically, in some embodiments, after step 1041 compares the parking feasibility with a pre - determined preset threshold to obtain a comparison result, it may further include:

[0120] When the comparison result is that the parking feasibility is less than the preset threshold, send a prompt message to the target user. The prompt message is used to prompt the target user that the parking function of the vehicle cannot be executed.

[0121] In this embodiment, when the comparison result shows that the parking feasibility is less than the preset threshold, a prompt message is sent to the target user to indicate that the parking function of the vehicle cannot be executed. The content of the prompt message includes information such as the non-execution result and recommended operations, such as "The driver has not left the vehicle. Please confirm the leaving status", etc. The prompt message can be sent to the target user through the in-vehicle display screen, mobile phone APP or voice prompt, etc., to ensure that the target user can receive the prompt message in a timely manner. The target user can feedback problems or suggestions through the in-vehicle system or mobile phone APP.

[0122] Through the multi-modal prompt message in the embodiment of the present invention, the system can timely remind the target user that the parking function cannot be executed, and timely remind when the parking function cannot be executed, so as to avoid potential safety hazards caused by accidental triggering.

[0123] Refer to Figure 7 , which shows a schematic structural diagram of a parking control device provided by an embodiment of the present invention. The device includes:

[0124] A target recognition module 201, configured to recognize the off-seat feature information of the target user in response to the activation of the vehicle parking function, and calculate the off-seat confidence of the target user by using the off-seat feature information;

[0125] A target tracking module 202, configured to perform target tracking on the target user to obtain the moving trajectory of the target user outside the vehicle, and calculate the off-vehicle confidence of the target user according to the moving trajectory;

[0126] A feasibility calculation module 203, configured to detect the seat occupancy value of the vehicle, fuse the seat occupancy value, the off-seat confidence, and the off-vehicle confidence to obtain the parking feasibility;

[0127] A parking control module 204, configured to control the vehicle to execute the parking function when the parking feasibility is greater than or equal to the preset threshold.

[0128] Further, the target recognition module 201 includes:

[0129] A detection sub-module, configured to detect a preset area in response to the activation of the vehicle parking function to determine the target user to be recognized;

[0130] An identification sub-module, configured to perform behavior identification on the target user to obtain the off-seat feature information of the target user; wherein, the off-seat feature information includes face detection results, head pose detection results, and off-seat behavior detection results;

[0131] The first calculation sub-module is used to calculate the off-seat confidence level of the target user by using the face detection result, the head pose detection result, and the off-seat behavior detection result.

[0132] Further, the recognition sub-module includes:

[0133] The first detection unit is used to perform face detection on the target user, determine the face size and the first confidence level that appear in the preset camera field of view, and obtain the face detection result;

[0134] The second detection unit is used to perform head pose recognition on the target user, determine the head pose deviation angle and the second confidence level, and obtain the head pose detection result;

[0135] The third detection unit is used to perform off-seat behavior detection on the target user, determine whether the target user is off-seat, and obtain the off-seat behavior detection result.

[0136] Further, the target tracking module 202 includes:

[0137] The tracking sub-module is used to obtain the body characteristics of the target user and perform target tracking on the target user according to the body characteristics;

[0138] The generation sub-module is used to collect the trajectory points of the target user and generate the moving trajectory of the target user outside the vehicle according to the trajectory points;

[0139] The second calculation sub-module is used to calculate the distance between the trajectory points in the moving trajectory and the center position of the vehicle to obtain the relative distance between the target user and the vehicle;

[0140] The third calculation sub-module is used to calculate the off-vehicle confidence level of the target user according to the relative distance between the target user and the vehicle.

[0141] Further, the calculation feasibility module 203 includes:

[0142] The first determination sub-module is used to detect the driver's seat of the vehicle, determine whether the driver's seat is occupied, and obtain the seat occupancy value of the vehicle;

[0143] The second determination sub-module is used to pre-determine the weight coefficients of the seat occupancy value, the off-seat confidence level, and the off-vehicle confidence level;

[0144] The fourth calculation sub-module is used to perform weighted fusion of the seat occupancy value, the off-seat confidence level, and the off-vehicle confidence level according to the weight coefficients to obtain the parking feasibility.

[0145] Further, the parking control module 204 includes:

[0146] A comparison sub-module, configured to compare the parking feasibility with a preset threshold determined in advance to obtain a comparison result;

[0147] A control sub-module, configured to control the vehicle to execute the parking function when the comparison result is that the parking feasibility is greater than or equal to the preset threshold.

[0148] Further, the parking control module 204 further includes:

[0149] A prompt sub-module, configured to send a prompt message to the target user when the comparison result is that the parking feasibility is less than the preset threshold, where the prompt message is used to prompt the target user that the parking function of the vehicle cannot be executed.

[0150] The parking control device provided by the embodiment of the present invention, by responding to the activation of the vehicle parking function, identifies the off-seat characteristic information of the target user, calculates the off-seat confidence of the target user using the off-seat characteristic information, performs target tracking on the target user to obtain the moving trajectory of the target user outside the vehicle, calculates the off-vehicle confidence of the target user according to the moving trajectory, detects the seat occupancy value of the vehicle, and fuses the seat occupancy value, the off-seat confidence, and the off-vehicle confidence to obtain the parking feasibility. When the parking feasibility is greater than or equal to the preset threshold, the vehicle is controlled to execute the parking function. The embodiment of the present invention accurately identifies the off-vehicle behavior of the driver and passengers during vehicle automatic parking, uses multi-sensor data fusion and confidence calculation mechanisms to obtain accurate and effective parking feasibility, automatically activates the vehicle parking function after determining that the driver is far away from the vehicle and is safe, avoids the current off-vehicle parking function from being accidentally triggered when the driver has not really left the vehicle or is not in a safe area, eliminates the need for the driver to perform secondary operations, significantly reduces the false trigger rate, reduces the safety hazards of vehicle parking, improves the safety and reliability of parking, and further enhances the user experience.

[0151] Referring to Figure 8 , the embodiment of the present invention further provides an electronic device, as Figure 8 shown, including a processor 301, a communication interface 302, a memory 303, and a communication bus 304, where the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304,

[0152] The processor 301 is used for the memory 303 that stores instructions executable by the processor;

[0153] Wherein, the processor 301 is configured to execute the instructions to implement the parking control method as described above.

[0154] The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0155] The communication interface is used for communication between the above terminal and other devices.

[0156] The memory may include a Random Access Memory (RAM), or may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0157] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0158] In another embodiment provided by the present invention, there is also provided a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the parking control method described in any one of the above embodiments is implemented.

[0159] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0160] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0161] Each embodiment in this specification is described in a related manner. For the same and similar parts between the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference can be made to the corresponding part of the method embodiment for the relevant content.

[0162] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included within the protection scope of the present invention.

Claims

1. A parking control method, characterized in that, The method includes: In response to the activation of the vehicle parking function, identifying the off-seat feature information of the target user, and calculating the off-seat confidence level of the target user using the off-seat feature information; Performing target tracking on the target user to obtain the movement trajectory of the target user outside the vehicle, and calculating the off-vehicle confidence level of the target user based on the movement trajectory; Detecting the seat occupancy value of the vehicle, and fusing the seat occupancy value, the off-seat confidence level, and the off-vehicle confidence level to obtain the parking feasibility; When the parking feasibility is greater than or equal to a preset threshold, controlling the vehicle to execute the parking function.

2. The method according to claim 1, characterized in that The step of, in response to the activation of the vehicle parking function, identifying the off-seat feature information of the target user, and calculating the off-seat confidence level of the target user using the off-seat feature information, includes: In response to the activation of the vehicle parking function, detecting a preset area to determine the target user to be identified; Performing behavior recognition on the target user to obtain the off-seat feature information of the target user; wherein, the off-seat feature information includes face detection results, head pose detection results, and off-seat behavior detection results; Calculating the off-seat confidence level of the target user using the face detection results, the head pose detection results, and the off-seat behavior detection results.

3. The method according to claim 2, wherein The step of performing behavior recognition on the target user to obtain the off-seat feature information of the target user includes: Performing face detection on the target user to determine the face size and the first confidence level that appear in the field of view of the preset camera, to obtain the face detection results; Performing head pose recognition on the target user to determine the head pose offset angle and the second confidence level, to obtain the head pose detection results; Performing off-seat behavior detection on the target user to determine whether the target user has left the seat, to obtain the off-seat behavior detection results.

4. The method according to claim 1, characterized in that, The step of performing target tracking on the target user to obtain the movement trajectory of the target user outside the vehicle, and calculating the off-vehicle confidence level of the target user based on the movement trajectory, includes: Obtaining the body features of the target user, and performing target tracking on the target user according to the body features; Collecting the trajectory points of the target user, and generating the movement trajectory of the target user outside the vehicle according to the trajectory points; Calculating the distance between the trajectory points in the movement trajectory and the center position of the vehicle to obtain the relative distance between the target user and the vehicle; Calculating the off-vehicle confidence level of the target user based on the relative distance between the target user and the vehicle.

5. The method according to claim 1, characterized in that The step of detecting the seat occupancy value of the vehicle, and fusing the seat occupancy value, the off-seat confidence level, and the off-vehicle confidence level to obtain the parking feasibility, includes: Detecting the driver's seat of the vehicle to determine whether the driver's seat is occupied, to obtain the seat occupancy value of the vehicle; Pre-determining the weight coefficients of the seat occupancy value, the off-seat confidence level, and the off-vehicle confidence level; Weightedly fusing the seat occupancy value, the off-seat confidence level, and the off-vehicle confidence level according to the weight coefficients to obtain the parking feasibility.

6. The method according to claim 1, characterized in that, When the parking feasibility is greater than or equal to a preset threshold, controlling the vehicle to execute the parking function includes: Comparing the parking feasibility with a preset threshold determined in advance to obtain a comparison result; When the comparison result is that the parking feasibility is greater than or equal to the preset threshold, controlling the vehicle to execute the parking function.

7. The method according to claim 6, characterized in that, After comparing the parking feasibility with the preset threshold determined in advance to obtain a comparison result, it further includes: When the comparison result is that the parking feasibility is less than the preset threshold, sending a prompt message to the target user, where the prompt message is used to prompt the target user that the parking function of the vehicle cannot be executed.

8. A parking control device, characterized in that, The device includes: A target recognition module, configured to, in response to the activation of the vehicle parking function, recognize the off-seat feature information of the target user, and calculate the off-seat confidence of the target user using the off-seat feature information; A target tracking module, configured to perform target tracking on the target user to obtain the moving trajectory of the target user outside the vehicle, and calculate the off-vehicle confidence of the target user according to the moving trajectory; A feasibility calculation module, configured to detect the seat occupancy value of the vehicle, fuse the seat occupancy value, the off-seat confidence, and the off-vehicle confidence to obtain the parking feasibility; A parking control module, configured to, when the parking feasibility is greater than or equal to a preset threshold, control the vehicle to execute the parking function.

9. An electronic device, characterized in that, It includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the parking control method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium, and when the computer program is executed by the processor, it implements the parking control method according to any one of claims 1 to 7.

Images