Automatic parking state control method and system based on vehicle body state self-sensing

Abstract

The invention belongs to the technical field of automatic parking control, and particularly relates to an automatic parking state control method and system based on vehicle body state self-sensing, and the method comprises the steps: verifying the image splicing state of a vehicle all-round camera system in real time, and judging whether the all-round camera system is in a normal working position or not; outputting a camera splicing abnormal flag bit and a self-correction external parameter value; vehicle body signals including one or more of a rearview mirror, a main driving door, a co-driving door and a trunk door are monitored in real time, and whether the current vehicle body signals are in a normal working state or not is judged to generate vehicle body signals; comprehensively splicing the abnormal flag bit and the vehicle body signal, judging whether the vehicle body state of the current vehicle meets a preset automatic parking operation condition or not, and dynamically controlling the operation state of the automatic parking function; and feedback information related to the current running state or the required operation of the automatic parking function is provided for a user through the human-computer interaction interface. According to the invention, the vehicle body state can be accurately and reasonably evaluated, and the working states of the parking system and the surround-view sensing system are dynamically controlled.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic parking control, and particularly relates to an automatic parking state control method and system based on self-perception of vehicle body state. Background Art

[0002] With the continuous improvement of the intelligent level of automobiles, functions such as Automated Parking Assist (APA) and Remote Parking Assist (RPA) have gradually become standard or optional configurations for many modern vehicles. These functions are designed to reduce the parking burden on drivers and improve parking efficiency and convenience.

[0003] However, the existing automatic parking systems still face many challenges in practical applications:

[0004] 1. Limitations in environmental perception: Many systems rely on surround-view cameras (usually fisheye cameras) to perceive parking spaces and obstacles. As Figure 1 shown, the left and right surround-view cameras are usually installed on foldable rearview mirrors, and the rear surround-view camera is installed on the openable trunk door. The prior art often does not fully consider the real-time states of these vehicle body components (such as rearview mirrors, doors, and trunk doors). When the rearview mirror is folded, the door or the trunk door is opened, the relative position and attitude of the camera change, resulting in distortion of the stitched image of the surround-view system or abnormal input of the perception algorithm, and then generating incorrect recognition results, which may cause the parking attitude to be skewed, unable to accurately identify available parking spaces (reducing the parking space release rate), and even in severe cases, unable to identify obstacles, leading to a collision risk and property damage.

[0005] 2. Imperfect system interaction: The automatic parking system needs to cooperate closely with vehicle actuators (such as the Electric Power Steering system EPS and the Electronic Stability Control system ESC). Existing solutions may not fully consider the interaction logic between the parking function and the ESC when it exits abnormally due to a fault, which may cause the vehicle to roll backward under working conditions such as on a slope, posing a safety hazard.

[0006] 3. Poor human-machine interaction experience: The interaction (HMI) between the user and the automatic parking system is a key factor affecting user acceptance and trust. Existing systems may have problems such as untimely and unclear fault information prompts, or too many and too frequent prompt messages, which instead cause unnecessary anxiety for users and reduce the trust in intelligent driving functions.

[0007] 4. Lack of safety mechanisms: The automatic parking system involves the lateral and longitudinal control of the vehicle, posing high requirements for functional safety and information security. Existing solutions may not fully deploy functional safety mechanisms (such as end-to-end E2E verification) and information security mechanisms (such as encryption and authentication) during signal transmission, making the system vulnerable to signal errors, losses, or malicious attacks from the outside (such as instruction forgery and tampering), threatening the reliability and security of the system, especially in the remote control scenario.

[0008] Therefore, there is an urgent need for a technical solution that can accurately and reasonably evaluate the vehicle body state, dynamically control the working states of the parking system and the surround view perception system, interact and coordinate with the ESC, have a simple and friendly human-computer interaction, and possess functional safety mechanisms and information security mechanisms. Summary of the Invention

[0009] Object of the Invention: The object of the present invention is to provide an automatic parking state control method and system based on self-perception of the vehicle body state, aiming to solve the problems of insufficient accuracy, stability, and security of automatic parking in the prior art due to the failure to fully consider the vehicle body state.

[0010] Technical Solution: The automatic parking state control method based on vehicle body state perception according to the present invention includes the following steps:

[0011] S1: Real-time verify the image stitching state of the vehicle surround view camera system, determine whether the surround view camera system is in a normal working position, and output a camera stitching abnormal flag bit and a self-correcting external parameter value;

[0012] S2: Real-time monitor the vehicle body signals including one or more of the rearview mirror, driver's door, passenger door, and trunk door, and determine whether the current vehicle body signal is in a normal working state to generate a vehicle body signal;

[0013] S3: Synthesize the camera stitching abnormal flag bit and the vehicle body signal, determine whether the vehicle body state of the current vehicle meets the preset automatic parking operation conditions and generate a vehicle body state flag bit, and dynamically control the operation state of the automatic parking function based on the vehicle body state flag bit, where the operation state at least includes an activation state, an inhibition state, and a shutdown state; wherein:

[0014] When it is determined that the preset automatic parking operation conditions are met, allow or maintain the activation state of the automatic parking function;

[0015] When it is determined that the preset automatic parking operation conditions are not met, switch the automatic parking function to the inhibition state or the shutdown state, and request the ESC to execute preset safety measures;

[0016] And when the vehicle body state changes from not meeting to meeting the preset automatic parking operation conditions, request the ESC to resume or continue to execute vehicle motion control related to parking;

[0017] S4: Provide feedback information related to the current operating state or required operations of the automatic parking function to the user through the human-machine interface.

[0018] To further improve the above technical solution, S1 includes:

[0019] The surround camera system includes a front vehicle-mounted camera, a rear vehicle-mounted camera, and two side vehicle-mounted cameras;

[0020] Input the stitched image generated by the surround camera system into a pre-trained deep learning network model for judgment. If it is judged that the surround camera system is in an abnormal working position, set the stitching anomaly flag position to 1, and continue to judge whether the deviation degree of any side vehicle-mounted camera from the normal working position is within the preset range θ. If so, learn and output the self-correction external parameter value corresponding to the moment when the any side vehicle-mounted camera is in the abnormal working position;

[0021] Adaptive adjustment of the external parameter value of the corresponding vehicle-mounted camera for correction and alignment when stitching the images obtained by the vehicle-mounted camera;

[0022] Based on the corrected and aligned stitched image, detect the parking space corner points and construct the initial parking space;

[0023] When the trunk door signal is detected to be in the closed state, use the rear vehicle-mounted camera to perform closed-loop correction on the initial parking space;

[0024] Dilate the virtual vehicle model horizontally and vertically according to the preset rules.

[0025] Furthermore, the deep learning network model is constructed and operated in the following manner:

[0026] Dataset preparation: Collect the stitched images and the corresponding camera external parameter data of at least one vehicle-mounted camera at the preset normal working position as positive samples; and collect the abnormal stitched images when at least one vehicle body component is in a variety of preset non-fully closed or non-fully opened state combinations, and the camera external parameters recalibrated in these states as negative samples, where the misaligned or distorted deformation areas are manually marked in the negative sample images;

[0027] Network model architecture: Adopt a network model based on YOLOv5, including: Input end: Uniformly process the input image into a preset size; Backbone network: Introduce depthwise separable convolution to reduce the computational amount and model parameters, and introduce the CBAM module to enhance the attention to the region of interest and the feature expression ability, and output feature maps of multiple different scales; Multi-scale feature fusion module: Fuse the feature maps of multiple scales obtained from the backbone network, enhance semantic information through the top-down path, enhance detail information through the bottom-up path, and enrich the feature expression by fusing feature maps of the same scale but different dimensions; Output module, including at least one classification output module and a regression prediction module, and the classification output module and the regression prediction module each have at least one detection head, which is used to predict the probability of splicing area abnormality according to the fused features, and output information indicating whether the camera state is normal, including: splicing abnormality flag bit and self-corrected external parameter value.

[0028] Further, S2 includes: receiving, through the vehicle gateway, the original signal corresponding to at least one body signal of the rearview mirror, the driver's door, the co-driver's door, and the trunk door; performing end-to-end verification on the original signal to verify the integrity and correctness of the original signal, and if an abnormality of the original signal is detected during the end-to-end verification, record the fault code and set the corresponding body signal flag bit to the abnormal flag bit.

[0029] Further, comprehensively based on the camera state information and the body signal, determine whether the body state of the current vehicle meets the preset automatic parking operation conditions and generate a body state flag bit, including:

[0030] When (rearview mirror = folded or driver's door = open or co-driver's door = open or trunk door = open or splicing abnormality flag bit = 1) and the number of detected parking space corner points < 2, and the duration of this state is greater than the first preset time threshold, then output a body state flag bit indicating that the automatic parking operation conditions are not met;

[0031] When (rearview mirror = folded or driver's door = open or co-driver's door = open or trunk door = open or splicing abnormality flag bit = 1) and the number of detected parking space corner points ≥ 2, and the duration of this state is greater than the first preset time threshold, then output a body state flag bit indicating that the automatic parking operation conditions are met;

[0032] When (rearview mirror = open and driver's door = closed and co-driver's door = closed and trunk door = closed and splicing abnormality flag bit = 0), and the duration of this state is greater than the second preset time threshold, then output a body state flag bit indicating that the automatic parking operation conditions are met.

[0033] Further, when the automatic parking function is in the activated state, if the vehicle body state flag changes from 1 to 0, the operation state of dynamically controlling the automatic parking function includes: switching the state of the automatic parking function to the inhibited state; and requesting the ESC to perform a slow exit operation to decelerate the vehicle until it stops at a preset deceleration, and then requesting the electronic parking brake to perform a parking action.

[0034] Further, in the inhibited state, it further includes: monitoring whether the vehicle body state flag returns to 1 within a third preset threshold. If it does, switch the state of the automatic parking function back to the activated state and resume executing the automatic parking task; if it exceeds the third preset threshold and the vehicle body state flag remains 0, switch the operation state to the closed state and request the ESC to activate the electronic parking brake to complete parking.

[0035] Further, it further includes applying an information security mechanism to process communication signals. The information security mechanism includes at least one of the following: for instructions or status information related to automatic parking control transmitted on the controller area network bus, using a message authentication code for verification or using an encryption protocol for encryption; performing information security processing on communication instructions related to remotely controlling the automatic parking function, and the information security processing includes two-way authentication and / or end-to-end encryption.

[0036] Further, the feedback information provided to the user through the human-machine interface includes at least one of the following: an automatic parking system fault prompt, a prompt that the automatic parking function has exited, a prompt that the automatic parking function has been restored, or an operation guide requiring the user to check and restore the vehicle body components to the normal state.

[0037] A system for implementing the above automatic parking state control method based on vehicle body state perception includes:

[0038] An image stitching verification module, configured to verify the image stitching state of the vehicle's surround view camera system in real time to determine whether the surround view camera system is in the normal working position, and output a camera stitching abnormal flag bit and a self-correcting external reference value;

[0039] A signal processing module, configured to receive signals indicating the states of at least one vehicle body component, where the vehicle body components include at least one of the rearview mirror, driver's door, passenger door, and trunk door, perform end-to-end verification on the signals to verify integrity and correctness, and output vehicle body signals;

[0040] A vehicle body state judgment module, configured to determine whether the current combined vehicle body state meets the preset automatic parking operation conditions according to the camera state information and the at least preset type of vehicle body signals output by the signal processing module after being processed and verified, and output a vehicle body state flag bit;

[0041] An automatic parking state control module, configured to dynamically control the operating state of the automatic parking system according to the vehicle body state flag bit and the vehicle body signal anomaly flag bit, disable or inhibit the automatic parking function when the combined vehicle body state does not meet the automatic parking operating conditions or the vehicle body signal anomaly flag bit indicates signal anomaly, and execute preset safety measures by interacting with the ESC module, and restore the automatic parking function after the combined vehicle body state returns to meet the automatic parking operating conditions and the vehicle body signal anomaly flag bit indicates normal signal;

[0042] A human-machine interaction interface for outputting feedback information related to the current operating state of the automatic parking system.

[0043] Advantages: Compared with the prior art, the advantages of the present invention are as follows:

[0044] 1. Improve the accuracy of the automatic parking system: By real-time monitoring the vehicle body state and dynamically controlling the operation of the automatic parking system, it avoids generating incorrect recognition results when the vehicle body state is abnormal, reduces the poor parking effect or collision risk caused by abnormal vehicle body state, and improves the accuracy.

[0045] 2. Improve the stability of the automatic parking system: When the vehicle body conditions are not met, the parking system state machine is controlled by the automatic parking state control module to enter the inhibition state, and then resumes operation after the vehicle body conditions return to normal, ensuring the stability of the system.

[0046] 3. Improve the safety of the automatic parking system: Ensure the integrity, correctness and security of signal transmission through functional safety mechanisms and information security mechanisms, and avoid system failures caused by signal errors, losses, and hacker attacks; Through the interaction with the ESC, when the parking function is abnormal, request the ESC to complete the parking action to prevent personal and property losses caused by vehicle slipping.

[0047] 4. Improve the usability of automatic parking and enhance users' confidence in intelligent parking: Timely, concise and appropriate feedback on the current state of the automatic parking system through the human-machine interaction interface or the mobile phone remote vehicle moving interface, improving the user driving experience, system usability and user confidence. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a layout diagram of a fish-eye camera.

[0049] Figure 2 It is an example of splicing misalignment caused by the installation size of the fish-eye camera not meeting the requirements.

[0050] Figure 3 It is an architecture diagram of a parking system.

[0051] Figure 4 It is a splicing anomaly detection algorithm based on a neural network.

[0052] Figure 5 It is a flowchart of the parking system.

[0053] Figure 6 It is a flowchart of slow exit.

[0054] Figure 7 It is a flowchart of vehicle body signal verification.

[0055] Figure 8 It is a flowchart of vehicle body state verification. Specific implementation mode

[0056] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the described embodiments.

[0057] Embodiment 1: As Figure 1 shown, for the installation positions of the surround view camera system under the current mainstream parking hardware architecture, including: a front fish-eye camera installed on the front bumper, which is relatively fixed to the vehicle body; left and right fish-eye cameras respectively installed on the left and right rearview mirrors, which move relative to the vehicle body as the rearview mirrors fold or the doors open and close; a rear fish-eye camera installed on the trunk, which moves relative to the vehicle body as the trunk door opens and closes. Therefore, when the rearview mirrors, the driver's door, the passenger door, and the trunk door are in the non-folded and closed states, the relative positions of the fish-eye cameras to the vehicle body change, resulting in abnormal parking perception output.

[0058] As Figure 2 shown, for the image stitching misalignment and deformation caused by a slight position deviation of the fish-eye camera relative to the vehicle body, which directly affects the parking space recognition accuracy and results in skewed parking or inability to park into the expected parking space.

[0059] As Figure 3 shown, the automatic parking state control method for vehicle body state perception includes: an image stitching verification module, a signal processing module, an information security module, a vehicle body state judgment module, an automatic parking state control module, and an HMI module.

[0060] Among them, the image stitching verification module: (1) is used to receive the images transmitted by the fish-eye cameras in real time and stitch them into a top view; (2) based on the trained lightweight deep learning network model, perform recognition verification on the input stitched image. When the position of the fish-eye camera is recognized as abnormal, the stitching abnormal flag bit is set to 1 and the self-corrected external parameter value is set, otherwise the stitching abnormal flag bit is set to 0; (3) is used to communicate with the vehicle body state judgment module in real time to inform the stitching abnormal flag bit information.

[0061] Signal Processing Module: (1) It is used to monitor the status signals of the rearview mirror, driver's door, passenger door, and trunk door in real time and perform E2E verification; (2) It is used for communication fault storage. When a signal anomaly is detected, it records and stores the fault code and notifies the downstream module; (3) It is used to communicate with the HMI module in real time to inform the user of the real-time operating status of the automatic parking function; (4) It is used to communicate with the ESC module in real time. When the automatic parking function degrades and resumes, it requests the ESC to enter different working modes.

[0062] Information Security Module: (1) CAN Bus Security: Introduce message authentication code and encryption protocol to prevent hackers from forging instructions (such as throttle, brake, steering) to steal vehicle control; (2) Mobile APP Remote Control Security: Use two-way authentication and end-to-end encryption to prevent man-in-the-middle attacks or instruction tampering.

[0063] Body Status Judgment Module: It is used to receive the body signals sent by the signal processing module and the stitching anomaly flag signals sent by the image stitching verification module to judge whether the body status meets the parking conditions.

[0064] Automatic Parking Status Control Module: It is used to receive the signals sent by the signal processing module and the body status judgment module and control the operation of the system state machine.

[0065] HMI Module: Interacts with the user in real time to inform the current parking system status.

[0066] To realize the real-time and accurate verification of the image stitching status of the vehicle surround camera system, a deep learning network model is constructed and applied in the embodiments of the present invention. The purpose of this model is to automatically detect the stitching image anomalies (such as misalignment, distortion) caused by the change of the physical position of the camera (for example, the rearview mirror is not fully unfolded, the door or trunk is not tightly closed), so as to judge whether the surround system is in a normal working state and give the self-corrected external reference value.

[0067] Dataset Preparation: Collect the stitched images and their corresponding camera external parameter data at the moment when the rearview mirror is opened and the driver and passenger doors and the trunk door are closed as positive samples; Considering that the fish-eye camera is in a non-working position mostly due to the change in the position of the camera relative to the vehicle body caused by the doors and trunk door not being fully closed or the rearview mirror not being fully opened, so here we make permutations and combinations of 5 different states (2%, 4%, 6%, 8%, 10%) of the rearview mirror folded, driver and passenger doors opened, and trunk door opened, and collect the abnormal stitched images under each combination and the camera external parameters re-calibrated in this state as negative samples to form a basic dataset, and manually label areas such as stitched misalignment and distorted deformation areas.

[0068] As Figure 4The shown deep learning network model is based on the network model architecture of YOLOv5, and combines depthwise separable convolution and convolutional block attention module to build a network model for anomaly recognition, including an input end, a backbone network, a multi-scale feature fusion module, and a classification output module.

[0069] Input end: The pictures are uniformly processed into pictures of 640*640*3.

[0070] Backbone network: To reduce the computational complexity and model parameters and maintain good performance, depthwise separable convolution is introduced into the backbone network; in order to make the model pay more attention to the region of interest and improve the model's feature expression ability, the CBAM module is introduced to help the model better understand the model content and identify various complex and changeable abnormal splicings, and finally output feature maps of three different scales: 160*160; 80*80; 40*40.

[0071] Multi-scale feature fusion module: This module enhances the semantic information of the three-scale feature maps obtained from the backbone network through a top-down path, enhances the detailed information of the feature maps through a bottom-up path, and enriches the feature expression through the fusion of feature maps of the same scale but different dimensions.

[0072] Output module: It includes a classification output module and a regression prediction module.

[0073] Classification output module: This module has 1 detection head, flattens and reduces the dimension of the fused 128-dimensional vector into a 1-dimensional vector and inputs it into the Sigmoid function to predict the probability of abnormal splicing in the splicing area. The DropOut layer is synchronously introduced to prevent overfitting, and the dropout coefficient is set to 0.3 here.

[0074] Regression prediction module: This module has 4 detection heads. By separately learning the relationship between the abnormal splicing effects of the fisheye cameras at the four positions of front + left front, front + right front, rear + left rear, and rear + right rear and the recalibrated external camera parameters, when the splicing anomaly flag bit = 1 during the actual operation of the model, it can respectively output the self-corrected external parameter values [R i ,t i ′ Independently store the self-corrected external parameter values and the splicing anomaly flag bit;

[0075] When the splicing anomaly flag bit = 1, the camera external parameters call the self-corrected external parameter values [R i ,t i ′ , when the splicing anomaly flag bit = 0, the camera external parameters call the original external parameter values [R i ,t i ​​. The model automatically calls different external parameter values according to the values of different stitching anomaly flag bits to correct the position of the fisheye camera in the world coordinate system.

[0076] When searching for a parking space or parking under the condition that the stitching anomaly flag bit = 1, the model needs to detect two corner points of the parking space to construct a valid parking space. For the initially constructed parking space, during the parking process, if the trunk door = closed state, the parking process needs to use the rear fisheye camera to perform closed-loop correction on the initial parking space.

[0077] When parking into the garage under the condition that the stitching anomaly flag bit = 1, in order to avoid the decrease in image recognition accuracy caused by the camera at an abnormal working position, it is necessary to expand the original vehicle model horizontally and vertically by [x1, y1] to further reduce the collision risk.

[0078] Embodiment 2: An automatic parking state control method based on vehicle body state self-perception is implemented using the system provided in Embodiment 1, including the following steps:

[0079] S1: Real-time verify the image stitching state of the vehicle's surround-view camera system, and determine whether the surround-view camera system is in a normal working position to generate a stitching state flag bit;

[0080] S2: Real-time monitor the vehicle body signals including one or several of the rearview mirror, driver's door, passenger door, and trunk door, and determine whether the current vehicle body signal is in a normal working state to generate a vehicle body signal;

[0081] S3: Combine the stitching state flag bit and the vehicle body signal to determine whether the vehicle body state of the current vehicle meets the preset automatic parking operation conditions and generate a vehicle body state flag bit, and dynamically control the operation state of the automatic parking function based on the vehicle body state flag bit. The operation state at least includes an active state, an inhibit state, and an OFF state; where:

[0082] When it is determined that the preset automatic parking operation conditions are met, allow or maintain the active state of the automatic parking function;

[0083] When it is determined that the preset automatic parking operation conditions are not met, switch the automatic parking function to the inhibit state or the OFF state, and request the ESC to perform a safety operation. The safety operation includes performing corresponding deceleration, activating the automatic parking function, or activating the electronic parking brake;

[0084] And when the vehicle body state changes from not meeting to meeting the preset automatic parking operation conditions, request the ESC to resume or continue to perform vehicle motion control related to parking;

[0085] S4: Provide feedback information related to the current operating state or required operations of the automatic parking function to the user through the human-machine interface.

[0086] For example Figure 5 , when the automatic parking / remote vehicle moving function is in the off state, the signal processing module records and stores a fault code when detecting an abnormal vehicle body signal, and does not output an HMI prompt when the parking function is not activated by the user; when the user attempts to activate the automatic parking / remote vehicle moving function in this state, the driver is reminded through the HMI or the remote parking interface: The automatic parking system has a fault, please repair it;

[0087] When the automatic parking / remote vehicle moving function is in the off state, the signal processing module detects that the vehicle body signal is normal, and the vehicle body state judgment module detects that the vehicle body state does not meet the requirements. When the user activates the parking function / remote vehicle moving function, the activation of the function is prohibited and the driver is reminded through the HMI or the remote parking interface: Please open the rearview mirror / close the door / close the trunk door;

[0088] When the automatic parking / remote vehicle moving function is in the activated state, and the signal processing module detects that the vehicle body signal is normal while the vehicle body state judgment module detects that the vehicle body state does not meet the requirements, the automatic parking state control module enters inhibit, records the fault code, reminds the driver through the HMI or the remote parking interface: Please open the rearview mirror / close the door / close the trunk door, and enters the slow exit mode.

[0089] For example Figure 6 As shown, the parking controller requests the ESC to enter the slow exit mode, and the ESC decelerates at the specified deceleration value. After the vehicle stops stably, the AutoHold function is started:

[0090] (1) When the waiting time > threshold Xs, if the vehicle body state still does not meet the requirements, the automatic parking / remote vehicle moving function is exited, the automatic parking state control module switches to OFF, the parking controller requests the ESC to pull up the EPB to complete parking, and the driver is reminded through the HMI or the remote parking interface: Automatic parking has exited, please park the vehicle in a safe place;

[0091] (2) When the waiting time is less than the threshold Xs and the vehicle body state is detected to meet the conditions, the automatic parking / remote vehicle moving function is restored, the automatic parking state control module switches to active, and the driver is reminded through the HMI or the remote parking interface: The parking function has been restored, and the automatic parking controller continues to request the ESC to complete the subsequent parking route at the specified speed and distance.

[0092] When the automatic parking / remote vehicle moving function is activated and the signal processing module detects an abnormal vehicle body signal, it records and stores a fault code; the parking controller requests the ESC to enter the slow exit mode, requests the ESC to decelerate according to the specified deceleration value, pulls up the EPB after the vehicle stops stably, and the automatic parking status control module switches to OFF, and reminds the driver through the HMI or the remote parking interface: The automatic parking system has a fault, please repair it.

[0093] Such as Figure 7 , the signal processing module performs E2E verification on the signals of the rearview mirror, front door, and trunk door transmitted in real time, and transmits the results: normal vehicle body signal / abnormal vehicle body signal to the downstream automatic parking status control module.

[0094] Such as Figure 8 , the vehicle body status judgment module receives the verified rearview mirror, front door, and trunk door status signals. When the rearview mirror = open and the front door = closed and the trunk door = closed and the splicing abnormal flag bit = 0 lasts for 5 Loops, it outputs the vehicle body status flag bit = 1: the vehicle body status meets the parking requirements; (when the rearview mirror = folded or the front door = open or the trunk door = open or the splicing abnormal flag bit = 1) and the number of detected parking space corner points ≥ 2 and lasts for 5 Loops, it outputs 1: the vehicle body status meets the parking requirements. (when the rearview mirror = folded or the front door = open or the trunk door = open or the splicing abnormal flag bit = 1) and the number of detected parking space corner points < 2 and lasts for 5 Loops, it outputs 0: the vehicle body status does not meet the parking requirements.

[0095] Interact with the user concisely through the following 4 prompt messages, inform the current system status, avoid unnecessary panic, improve the usability and credibility of the automatic parking system, and improve the user's experience of the automatic parking system.

[0096] Prompt message 1: Automatic parking has exited, please park the vehicle in a safe place;

[0097] Prompt message 2: The automatic parking system has a fault, please repair it;

[0098] Prompt message 3: Please open the rearview mirror / close the door / close the trunk door;

[0099] Prompt message 4: The parking function has been restored.

[0100] Ensure the security of the can bus by introducing a message authentication code and an encryption protocol to prevent hackers from forging instructions (such as accelerating, braking, steering) to control the vehicle. By introducing two-way authentication and end-to-end encryption, prevent users from being attacked by a man-in-the-middle to tamper with instructions when remotely moving the vehicle through the mobile phone APP.

[0101] When the vehicle body status flag bit = 1 and the vehicle body signal anomaly flag bit = 0 and a user request to activate automatic parking is received, control the parking state machine to jump from standby to the active state; request ESC and EPS to execute the angle, speed, distance, etc. requirements of the normal operating condition request; and request the HMI module to turn on the automatic parking working indicator light to inform the user of the current automatic parking state.

[0102] When the vehicle body status flag bit = 1 and the vehicle body signal anomaly flag bit = 1 and a user request to activate automatic parking is received, control the parking state machine to remain in the OFF state, prohibit the activation function, and record the fault code; and request the HMI module to inform the user of prompt message 2.

[0103] When the vehicle body status flag bit = 0 and the vehicle body signal anomaly flag bit = 0 and a user request to activate automatic parking is received, control the parking state machine to remain in the OFF state, prohibit the activation function; and request the HMI module to inform the user of prompt message 3.

[0104] In the activated state of the automatic parking / remote vehicle relocation function, when the vehicle body signal anomaly flag bit = 1, then control the automatic parking state machine to jump from active to inhabit, the parking system enters the safe state, execute the slow exit of the automatic parking / remote vehicle relocation function, request ESC to decelerate at the specified deceleration value of a m / s^2, pull up the EPB after the vehicle stops, and inform the user of prompt message 2 through the HMI module, and finally control the parking state machine to jump to OFF.

[0105] In the activated state of the automatic parking / remote vehicle relocation function, when the vehicle body status flag bit = 0 and the vehicle body signal anomaly flag bit = 0, the automatic parking state machine jumps from active to inhabit, records the fault code, and informs the user of prompt message 3 through the HMI module, requests ESC to enter the slow exit mode, and activates AutoHold when the vehicle speed = 0; when the vehicle body status flag bit = 1 within the threshold of X s, the automatic parking state machine jumps from inhabit to the active state, continues to complete the subsequent parking tasks, and informs the user of prompt message 4 through the HMI module; if the vehicle body status flag bit remains 0 within the threshold of X s, the automatic parking state machine jumps to OFF, and requests the signal processing module to pull up the EPB caliper and then inform the user of prompt message 1 through the HMI module.

[0106] The present invention establishes more stringent and reliable prerequisite conditions for automatic parking operation by perceiving in real time and comprehensively the physical states of the vehicle itself (doors, rearview mirrors, trunk) and the working states of key perception systems (surround view cameras) (judging by stitching quality). Combining with dynamic state machine control logic and in-depth safety cooperation with vehicle control systems such as ESC, it can significantly improve the safety, accuracy and stability during automatic parking while ensuring the functional availability. With clear and effective human-machine interaction, the user experience is improved and the user's trust in intelligent driving functions is enhanced. This technical solution is applicable to various scenarios such as automatic parking, remote parking and remote vehicle relocation.

[0107] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as a limitation of the present invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. An automatic parking state control method based on self - perception of vehicle body state, characterized in that, It includes the following steps: S1: Real-time check the image stitching status of the vehicle surround camera system, determine whether the surround camera system is in the normal working position, and output a stitching exception flag bit and a self-corrected external parameter value; S2: Real-time monitor the body signals including one or more of the rearview mirror, driver's door, passenger door, and trunk door, and determine whether the current body signal is in the normal working state to generate a body signal; S3: Synthesize the camera stitching exception flag bit and the body signal, determine whether the body state of the current vehicle meets the preset automatic parking operation conditions and generate a body state flag bit, and dynamically control the operation state of the automatic parking function based on the body state flag bit. The operation state at least includes an activation state, a suppression state, and a shutdown state; where: When it is determined that the preset automatic parking operation conditions are met, allow or maintain the activation state of the automatic parking function; When it is determined that the preset automatic parking operation conditions are not met, switch the automatic parking function to the suppression state or the shutdown state, and request the ESC to execute the preset safety measures; And when the body state changes from not meeting to meeting the preset automatic parking operation conditions, request the ESC to resume or continue to execute the vehicle motion control related to parking; S4: Provide feedback information related to the current operation state or required operation of the automatic parking function to the user through the human-machine interface.

2. The automatic parking state control method based on self - perception of vehicle body state according to claim 1, characterized in that, The S1 includes: The surround camera system includes a front vehicle-mounted camera, a rear vehicle-mounted camera, and two side vehicle-mounted cameras; Input the stitched image generated by the surround camera system into a pre-trained deep learning network model for judgment. If it is determined that the surround camera system is in an abnormal working position, set the stitching exception flag bit to 1, and continue to judge whether the deviation degree of any side vehicle-mounted camera from the normal working position is within the preset range θ. If so, learn and output the self-corrected external parameter value corresponding to the abnormal working position moment of any side vehicle-mounted camera; Adaptive adjustment of the external parameter value of the corresponding vehicle-mounted camera for correction and alignment during image stitching obtained by the vehicle-mounted camera; Based on the corrected and aligned stitched image, detect the parking space corner points and construct an initial parking space; When the trunk door signal is detected to be in the closed state, use the rear vehicle-mounted camera to perform closed-loop correction on the initial parking space; Perform inflation processing on the virtual vehicle model in the horizontal and vertical directions according to the preset rules.

3. The automatic parking state control method based on self-perception of vehicle body state according to claim 2, wherein The deep learning network model is constructed and operated in the following manner: Dataset preparation: Collect the stitched images and the corresponding camera external parameter data of at least one vehicle-mounted camera at the preset normal working position as positive samples; and collect abnormal stitched images when at least one body part is in various preset non-fully closed or non-fully open state combinations, and the camera external parameters re-calibrated in these states as negative samples. The misaligned or distorted deformation areas are manually marked in the negative sample images; Network model architecture: Adopt a network model based on YOLOv5, including: Input end: Uniformly process the input image into a preset size; Backbone network: Depthwise separable convolution is introduced to reduce the computational amount and model parameters, and the CBAM module is introduced to enhance the attention to the region of interest and the feature expression ability, and feature maps of multiple different scales are output; Multi-scale feature fusion module: Fuse the feature maps of multiple scales obtained from the backbone network, enhance the semantic information through the top-down path, enhance the detail information through the bottom-up path, and enrich the feature expression by fusing the feature maps of the same scale but different dimensions; Output module: At least includes one classification output module and one regression prediction module. The classification output module and the regression prediction module respectively have at least one detection head, which is used to predict the probability of splicing region abnormality according to the fused features, and output the information indicating whether the camera state is normal, including: splicing abnormality flag bit and self-corrected external parameter value.

4. The automatic parking state control method based on vehicle body state self-perception according to claim 1, characterized in that The S2 includes: Receive the original signal corresponding to at least one body signal of the rearview mirror, driver's door, co-driver's door, and trunk door through the vehicle gateway; Perform end-to-end verification on the original signal to verify the integrity and correctness of the original signal. If an abnormality in the original signal is detected during the end-to-end verification, record the fault code and set the corresponding body signal flag bit to the abnormality flag bit.

5. The automatic parking state control method based on vehicle body state perception according to claim 2, characterized in that, Based on the camera splicing abnormality flag bit and the body signal, judge whether the body state of the current vehicle meets the preset automatic parking operation conditions and generate a body state flag bit, including: When (rearview mirror = folded or driver's door = open or co-driver's door = open or trunk door = open or splicing abnormality flag bit = 1) and the number of detected parking space corner points < 2, and this state lasts for more than the first preset time threshold, then output a body state flag bit indicating that the automatic parking operation conditions are not met; When (rearview mirror = folded or driver's door = open or co-driver's door = open or trunk door = open or splicing abnormality flag bit = 1) and the number of detected parking space corner points ≥ 2, and this state lasts for more than the first preset time threshold, then output a body state flag bit indicating that the automatic parking operation conditions are met; When (rearview mirror = open and driver's door = closed and co-driver's door = closed and trunk door = closed and splicing abnormality flag bit = 0), and this state lasts for more than the second preset time threshold, then output a body state flag bit indicating that the automatic parking operation conditions are met.

6. The automatic parking state control method based on self - perception of vehicle body state according to claim 5, wherein, When the automatic parking function is in the activated state, if the body state flag bit changes from 1 to 0, the operation state of dynamically controlling the automatic parking function includes: Switch the state of the automatic parking function to the inhibited state; And request the ESC to perform a slow exit operation to decelerate the vehicle until it stops at a preset deceleration, and then request the electronic parking brake to perform a parking action.

7. The automatic parking state control method based on vehicle body state perception according to claim 5, characterized in that, In the inhibited state, it also includes: Monitor whether the body state flag bit returns to 1 within the third preset time threshold. If it does, switch the state of the automatic parking function back to the activated state and resume the execution of the automatic parking task; If the vehicle body status flag is still 0 after exceeding the third preset time threshold, switch the operating state to the closed state, and request the ESC to activate the electronic parking brake to complete parking.

8. The automatic parking state control method based on self - perception of vehicle body state according to claim 1, characterized in that, It also includes applying an information security mechanism to process communication signals, and the information security mechanism includes at least one of the following: For the instructions or status information related to automatic parking control transmitted on the controller area network bus, use a message authentication code for verification or use an encryption protocol for encryption; Perform information security processing on the communication instructions related to the remote control automatic parking function, and the information security processing includes mutual authentication and / or end-to-end encryption.

9. The automatic parking state control method based on self - perception of vehicle body state according to claim 1, wherein, The feedback information provided to the user through the human-machine interface includes at least one of the following: automatic parking system fault prompt, automatic parking function exited prompt, automatic parking function restored prompt, or an operation guide asking the user to check and restore the vehicle body components to the normal state.

10. A system for implementing the automatic parking state control method based on self - perception of vehicle body state according to claim 1, characterized in that, It includes: An image stitching verification module, configured to verify the image stitching status of the vehicle's surround camera system in real time to determine whether the surround camera system is in the normal working position, and output a camera stitching abnormality flag bit and a self-corrected external reference value; A signal processing module, configured to receive signals indicating the status of at least one vehicle body component, where the vehicle body components include at least one of the rearview mirror, driver's door, passenger door, and trunk door, perform end-to-end verification on the signals to verify integrity and correctness, and output vehicle body signals; A vehicle body status judgment module, configured to determine whether the current combined vehicle body status meets the preset automatic parking operation conditions according to the camera stitching abnormality flag bit and the at least preset type of vehicle body signals output by the signal processing module after processing and verification, and output a vehicle body status flag bit; An automatic parking status control module, configured to dynamically control the operating state of the automatic parking system according to the vehicle body status flag bit and the vehicle body signal abnormality flag bit, disable or inhibit the automatic parking function when the combined vehicle body status does not meet the automatic parking operation conditions or the vehicle body signal abnormality flag bit indicates signal abnormality, and execute preset safety measures by interacting with the ESC module, and restore the automatic parking function after the combined vehicle body status resumes to meet the automatic parking operation conditions and the vehicle body signal abnormality flag bit indicates normal signal; A human-machine interface, used to output feedback information related to the current operating state of the automatic parking system.

Images