An automatic parking state control method and system based on vehicle body state self-sensing

Abstract

The present application belongs to the automatic parking control technical field, and particularly relates to an automatic parking state control method and system based on self-sensing of a vehicle body state, which comprises: real-time checking of an image splicing state of a vehicle surround-view camera system, judgment of whether the surround-view camera system is in a normal working position, and output of a camera splicing abnormal flag bit and a self-corrected external parameter value; real-time monitoring of a vehicle body signal including one or several of a rearview mirror, a main driver door, a co-driver door and a trunk door, judgment of whether the current vehicle body signal is in a normal working state, and generation of the vehicle body signal; comprehensive splicing of the abnormal flag bit and the vehicle body signal, judgment of whether the current vehicle body state of the vehicle satisfies a preset automatic parking operation condition for dynamically controlling the running state of the automatic parking function; and provision of feedback information related to the current running state of the automatic parking function or required operation to a user through a man-machine interactive interface. The present application can accurately and reasonably evaluate the vehicle body state, and dynamically control the working state of the parking system and the surround-view sensing system.

Description

Technical Field

[0001] This invention belongs to the field of automatic parking control technology, specifically relating to an automatic parking state control method and system based on vehicle body state self-sensing. Background Technology

[0002] With the continuous improvement of automotive intelligence, Automated Parking Assist (APA) and Remote Parking Assist (RPA) have gradually become standard or optional features in many modern vehicles. These features aim to reduce the driver's parking burden and improve parking efficiency and convenience.

[0003] However, existing automated parking systems still face many challenges in practical applications:

[0004] 1. Limitations of Environmental Perception: Many systems rely on surround-view cameras (usually fisheye cameras) to perceive parking spaces and obstacles. For example... Figure 1 As shown, left and right surround-view cameras are typically mounted on foldable rearview mirrors, while the rear surround-view camera is mounted on the openable trunk lid. Existing technologies often do not adequately consider the real-time status of these vehicle components (such as rearview mirrors, doors, and the trunk lid). When the rearview mirrors are folded or the doors or trunk lid are opened, the relative position and orientation of the cameras change, causing distortion in the stitched image of the surround-view system or abnormal input to the perception algorithm. This leads to incorrect recognition results, potentially resulting in skewed parking posture, inaccurate identification of available parking spaces (reducing parking space availability), and even, in severe cases, failure to identify obstacles, posing a collision risk and causing property damage.

[0005] 2. Inadequate system interaction: The automatic parking system needs to work closely with the vehicle's actuators (such as the steering system EPS and braking system ESC). Existing solutions may not fully consider the interaction logic with ESC when the parking function is abnormally discontinued due to a malfunction, which may cause the vehicle to roll away under conditions such as slopes, posing a safety hazard.

[0006] 3. Poor Human-Machine Interface Experience: The interaction between the user and the automatic parking system (HMI) is a key factor affecting user acceptance and trust. Existing systems may have issues with untimely or unclear fault information prompts, or excessive or too frequent prompts, which can cause unnecessary anxiety for users and reduce their trust in intelligent driving functions.

[0007] 4. Lack of Safety Mechanisms: Automated parking systems involve lateral and longitudinal control of vehicles, placing high demands on functional safety and information security. Existing solutions may not fully deploy functional safety mechanisms (such as end-to-end E2E verification) and information security mechanisms (such as encryption and authentication) during signal transmission, making the system vulnerable to signal errors, loss, or malicious attacks from the outside (such as command forgery and tampering), threatening the system's reliability and security, especially in remote control scenarios.

[0008] Therefore, there is an urgent need for a technical solution that can accurately and reasonably assess the vehicle body status, dynamically control the parking system and the surround view perception system, coordinate with ESC, have a simple and user-friendly human-machine interface, and possess functional safety and information security mechanisms. Summary of the Invention

[0009] Purpose of the invention: The purpose of this invention is to provide an automatic parking state control method and system based on vehicle body state self-sensing, aiming to solve the problems of insufficient accuracy, stability and safety of automatic parking caused by insufficient consideration of vehicle body state in the prior art.

[0010] Technical solution: The automatic parking state control method based on vehicle body state perception described in this invention includes the following steps:

[0011] S1: Real-time verification of the image stitching status of the vehicle surround view camera system, determining whether the surround view camera system is in a normal working position, and outputting the camera stitching abnormality flag and self-correcting external parameter value;

[0012] S2: Real-time monitoring of one or more vehicle signals, including rearview mirror, driver's door, passenger door, and trunk door, to determine whether the current vehicle signal is in normal working condition, so as to generate a vehicle signal;

[0013] S3: Combining the abnormal camera stitching flag and the vehicle body signal, determine whether the current vehicle body state meets the preset automatic parking operating conditions and generate a vehicle body state flag. Based on the vehicle body state flag, dynamically control the operating state of the automatic parking function. The operating state includes at least an active state, a suppressed state, and a deactivated state; wherein:

[0014] When it is determined that the preset automatic parking operating conditions are met, the automatic parking function is allowed or maintained in an active state.

[0015] When it is determined that the preset automatic parking operating conditions are not met, the automatic parking function is switched to a suppressed state or a closed state, and the ESC is requested to execute preset safety measures.

[0016] And when the vehicle's status changes from not meeting the preset automatic parking operating conditions to meeting them, it requests ESC to resume or continue executing parking-related vehicle motion control;

[0017] S4: Provide users with feedback information related to the current operating status or required operation of the automatic parking function through the human-computer interaction interface.

[0018] To further improve the above technical solution, S1 includes:

[0019] The surround-view camera system includes a front vehicle-mounted camera, a rear vehicle-mounted camera, and side vehicle-mounted cameras.

[0020] The stitched image generated by the surround-view camera system is input into a pre-trained deep learning network model for judgment. If the surround-view camera system is determined to be in an abnormal working position, the stitching abnormality flag is set to 1, and the degree of deviation of the vehicle-mounted camera on either side from the normal working position is further judged to be within the preset range θ. If so, the self-correcting extrinsic parameter value corresponding to the vehicle-mounted camera on either side at the time of the abnormal working position is learned and output.

[0021] Adaptively adjust the extrinsic values ​​of the corresponding vehicle-mounted camera to correct and align the images acquired by the vehicle-mounted camera during stitching;

[0022] Parking space corner points are detected and initial parking spaces are constructed based on the corrected and aligned stitched images;

[0023] When the trunk door signal is detected as closed, the initial parking space is corrected using the rear vehicle camera in a closed loop.

[0024] The virtual car model is expanded horizontally and vertically according to preset rules.

[0025] Furthermore, the deep learning network model is constructed and run in the following manner:

[0026] Dataset preparation: Collect stitched images and corresponding camera extrinsic data of at least one vehicle-mounted camera in a preset normal working position as positive samples; and collect abnormal stitched images of at least one vehicle body component in a combination of multiple preset incompletely closed or incompletely open states, as well as the camera extrinsic data obtained by recalibration in these states as negative samples, wherein the stitching misalignment or distortion areas are manually marked in the negative sample images.

[0027] Network model architecture: A YOLOv5-based network model is adopted, including: Input end: Input images are uniformly processed to a preset size; Backbone network: Depthwise separable convolutions are introduced to reduce computation and model parameters, and a CBAM module is introduced to enhance the focus on regions of interest and feature representation capabilities, outputting feature maps of multiple different scales; Multi-scale feature fusion module: Fusing feature maps of multiple scales obtained from the backbone network, enhancing semantic information through a top-down path, enhancing detail information through a bottom-up path, and enriching feature representation by fusing feature maps of the same scale but different dimensions; Output module, including at least one classification output module and a regression prediction module. The classification output module and the regression prediction module each have at least one detection head, used to predict the probability of abnormality in the stitching region based on the fused features, and output information indicating whether the camera status is normal, including: stitching abnormality flag and self-correcting extrinsic values.

[0028] Further, S2 includes: receiving, through the vehicle gateway, an original signal corresponding to at least one of the following body signals: rearview mirror, driver's side door, passenger side door, and trunk door; performing end-to-end verification on the original signal to verify the integrity and correctness of the original signal; if an abnormality is detected in the original signal during the end-to-end verification, recording a fault code and setting the corresponding body signal flag bit to the abnormality flag bit.

[0029] Furthermore, by combining the camera status information and the vehicle body signal, it is determined whether the current vehicle body status meets the preset automatic parking operation conditions and a vehicle body status flag is generated, including:

[0030] When (rearview mirror = folding or driver's door = open or passenger's door = open or trunk door = open or splicing abnormal flag = 1) and the number of detected parking space corner points is less than 2, and the duration of this state is greater than the first preset time threshold, then output a vehicle status flag indicating that the automatic parking operation conditions are not met.

[0031] When (rearview mirror = folding or driver's door = open or passenger's door = open or trunk door = open or splicing abnormal flag = 1) and the number of detected parking space corner points is ≥ 2, and the duration of this state is greater than the first preset time threshold, then output a vehicle status flag indicating that the automatic parking operation conditions are met.

[0032] When (rearview mirror = open and driver's door = closed and passenger's door = closed and trunk door = closed and splicing abnormal flag = 0), and the duration of this state is greater than the second preset time threshold, then output a vehicle status flag indicating that the automatic parking operation conditions are met.

[0033] Furthermore, when the automatic parking function is activated, if the vehicle status flag changes from 1 to 0, the dynamic control of the automatic parking function's operating state includes: switching the automatic parking function's state to a suppressed state; requesting the ESC to perform a slow exit operation to decelerate the vehicle to a preset deceleration until it stops; and subsequently requesting the electronic parking brake to perform a parking action.

[0034] Furthermore, in the suppressed state, the method also includes: monitoring whether the vehicle status flag is restored to 1 within a third preset threshold; if it is restored, the automatic parking function is switched back to the active state and the automatic parking task is resumed; if the third preset threshold is exceeded and the vehicle status flag is still 0, the running state is switched to the closed state and the ESC is requested to activate the electronic parking brake to complete parking.

[0035] Furthermore, it also includes applying information security mechanisms to process communication signals, wherein the information security mechanism includes at least one of the following: verifying the instructions or status information related to automatic parking control transmitted on the controller local area network bus using a message authentication code or encrypting them using an encryption protocol; and performing information security processing on communication instructions involving remote control of automatic parking functions, wherein the information security processing includes two-way authentication and / or end-to-end encryption.

[0036] Furthermore, the feedback information provided to the user through the human-machine interface includes at least one of the following: automatic parking system fault prompt, automatic parking function deactivated prompt, automatic parking function restored prompt, or operation instructions requiring the user to check and restore the vehicle body components to normal condition.

[0037] A system for implementing the above-mentioned automatic parking state control method based on vehicle body state perception includes:

[0038] The image stitching verification module is configured to verify the image stitching status of the vehicle surround view camera system in real time, so as to determine whether the surround view camera system is in a normal working position, and output the camera stitching abnormality flag bit and self-correcting external parameter value.

[0039] A signal processing module is configured to receive a signal indicating the status of at least one body component, the body component including at least one of a rearview mirror, driver's side door, passenger side door, and trunk door, perform end-to-end verification of the signal to verify its integrity and correctness, and output a body signal.

[0040] The vehicle status determination module is configured to determine whether the current combined vehicle status meets the preset automatic parking operation conditions based on the camera status information and the processed and verified vehicle signals including at least preset types output by the signal processing module, and output the vehicle status flag bit.

[0041] The automatic parking status control module is configured to dynamically control the operating status of the automatic parking system based on the vehicle status flag and the vehicle signal abnormality flag. When the combined vehicle status does not meet the automatic parking operating conditions or the vehicle signal abnormality flag indicates an abnormal signal, the automatic parking function is disabled or suppressed. The module interacts with the ESC module to execute preset safety measures. The automatic parking function is restored after the combined vehicle status recovers to meet the automatic parking operating conditions and the vehicle signal abnormality flag indicates a normal signal.

[0042] The human-machine interface is used to output feedback information related to the current operating status of the automatic parking system.

[0043] Beneficial effects: Compared with the prior art, the advantages of the present invention are as follows:

[0044] 1. Improve the accuracy of the automatic parking system: By monitoring the vehicle's status in real time and dynamically controlling the operation of the automatic parking system, incorrect recognition results are avoided when the vehicle's status is abnormal, reducing poor parking performance or collision risks caused by abnormal vehicle status, and improving accuracy.

[0045] 2. High stability of the automatic parking system: When the vehicle conditions are not met, the automatic parking state control module controls the parking system state machine to enter a suppressed state. It will resume operation after the vehicle conditions return to normal, thus ensuring the stability of the system.

[0046] 3. Improve the safety of the automatic parking system: Ensure the integrity, correctness and security of signal transmission through functional safety mechanisms and information security mechanisms to avoid system failures caused by signal errors, loss or hacker attacks; through interaction with ESC, request ESC to complete the parking action when the parking function is abnormal to prevent the car from rolling and causing personal injury or property damage.

[0047] 4. Improve the ease of use of automatic parking and enhance user confidence in intelligent parking: Provide timely, concise, and appropriate feedback on the current status of the automatic parking system through the human-machine interface or the mobile remote parking interface to improve the user's driving experience, system usability, and user confidence. Attached Figure Description

[0048] Figure 1 This is a layout diagram of a fisheye camera.

[0049] Figure 2 This is an example of misalignment caused by the fisheye camera not meeting the installation dimensions.

[0050] Figure 3 This is a diagram of the parking system architecture.

[0051] Figure 4 It is a splicing anomaly detection algorithm based on neural networks.

[0052] Figure 5 This is a flowchart of the parking system.

[0053] Figure 6 This is a slow exit flowchart.

[0054] Figure 7 This is a flowchart of the vehicle body signal verification process.

[0055] Figure 8 This is a flowchart for vehicle body status verification. Detailed Implementation

[0056] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the embodiments described.

[0057] Example 1: As Figure 1 The diagram shows the installation locations of the surround-view camera system in the current mainstream parking hardware architecture, including: a front fisheye camera mounted on the front bumper, fixed relative to the vehicle body; left and right fisheye cameras mounted on the left and right rearview mirrors, which move relative to the vehicle body as the mirrors fold or the doors open and close; and a rear fisheye camera mounted on the trunk, which moves relative to the vehicle body as the trunk door opens and closes. Therefore, when the rearview mirrors, driver's door, passenger door, and trunk door are not folded or closed, the position of the fisheye cameras relative to the vehicle body changes, causing abnormal parking perception output.

[0058] like Figure 2 As shown, slight positional deviations of the fisheye camera relative to the vehicle's position cause image stitching misalignment and distortion, directly affecting the accuracy of parking space recognition and resulting in misaligned parking or failure to park in the expected parking space.

[0059] like Figure 3 The automatic parking status control method based on vehicle body status perception shown includes: an image stitching and verification module, a signal processing module, an information security module, a vehicle body status judgment module, an automatic parking status control module, and an HMI module.

[0060] Among them, the image stitching verification module: (1) is used to receive the image transmitted from the fisheye camera in real time and stitch it into a top view; (2) based on the trained lightweight deep learning network model, it performs recognition and verification on the input stitched image. When the fisheye camera position is found to be abnormal, the stitching abnormal flag position is 1 and the self-correcting external parameter value is set. Otherwise, the stitching abnormal flag position is 0; (3) is used to communicate with the vehicle body status judgment module in real time and inform it of the stitching abnormal flag position information.

[0061] Signal processing module: (1) Used to monitor the status signals of the rearview mirror, driver's door, passenger door and trunk door in real time and perform E2E verification; (2) Used to store communication faults, record and store fault codes when an abnormal signal is detected, and inform downstream modules; (3) Used to communicate with the HMI module in real time to inform the user of the real-time operating status of the automatic parking function; (4) Used to communicate with the ESC module in real time, and request the ESC to enter different working modes when the automatic parking function is downgraded and restored.

[0062] Information security module: (1) CAN bus security: Introducing message authentication codes and encryption protocols to prevent hackers from forging instructions (such as accelerator, brake, steering) to steal vehicle control; (2) Mobile APP remote control security: Using two-way authentication and end-to-end encryption to prevent man-in-the-middle attacks or instruction tampering.

[0063] Vehicle status judgment module: Used to receive vehicle signals sent by the signal processing module and stitching anomaly flag signals sent by the image stitching verification module, and to determine whether the vehicle status meets the parking conditions.

[0064] Automatic parking status control module: used to receive signals sent by the signal processing module and the vehicle status judgment module, and to control the operation of the system state machine.

[0065] HMI module: Interacts with the user in real time, informing them of the current parking system status.

[0066] To achieve real-time and accurate verification of the image stitching status of the vehicle surround view camera system, a deep learning network model is constructed and applied in this embodiment of the invention. The purpose of this model is to automatically detect image stitching anomalies (such as misalignment or distortion) caused by changes in the physical position of the camera (e.g., rearview mirror not fully extended, door or trunk not closed tightly), thereby determining whether the surround view system is in normal working condition and providing self-correcting extrinsic parameter values.

[0067] Dataset preparation: Stitched images and their corresponding camera extrinsic parameters at the times when the rearview mirror is open, the driver and passenger doors are closed, and the trunk door is closed were collected as positive samples. Considering that the fisheye camera is not in a working position due to the door and trunk door not being fully closed or the rearview mirror not being fully open, the camera position relative to the vehicle body changes. Therefore, five different states (2%, 4%, 6%, 8%, 10%) of the rearview mirror folded, driver and passenger doors open, and trunk door open were arranged and combined. Abnormal stitched images under each combination and the camera extrinsic parameters obtained by recalibration under that state were collected as negative samples to form the basic dataset. Areas such as stitching misalignment and distortion were manually labeled.

[0068] like Figure 4The deep learning network model shown is based on the YOLOv5 network model architecture. It combines deep separable convolution and convolutional block attention modules to build a network model for anomaly recognition, including an input end, backbone network, multi-scale feature fusion module, and classification output module.

[0069] Input end: Process all images to a size of 640*640*3;

[0070] Backbone Network: To reduce computation and model parameters while maintaining good performance, depthwise separable convolutions are introduced into the backbone network. To enable the model to focus more on regions of interest and improve its feature representation capabilities, a CBAM module is introduced to help the model better understand its content and identify various complex and varied abnormal splicing. The final output features at three different scales: 160*160, 80*80, and 40*40.

[0071] Multi-scale feature fusion module: This module enhances semantic information from feature maps of three different scales obtained from the backbone network through a top-down path, enhances detailed information of feature maps through a bottom-up path, and enriches feature expression by fusing feature maps of the same scale but different dimensions.

[0072] Output module: Includes a classification output module and a regression prediction module.

[0073] Classification output module: This module has one detection head, which flattens and reduces the 128-dimensional vector after fusion to a 1-dimensional vector and inputs it into the Sigmoid function to predict the probability of anomalies in the spliced ​​region. At the same time, a DropOut layer is introduced to prevent overfitting. Here, the dropout coefficient is set to 0.3.

[0074] Regression Prediction Module: This module has four detection heads. By learning the relationship between the abnormal stitching effects of the fisheye camera at four positions (front + left front, front + right front, rear + left rear, and rear + right rear) and the recalibrated camera extrinsic parameters, the model can output the self-correcting extrinsic parameter values ​​[R] corresponding to the abnormal working positions of the four fisheye cameras within a certain range θ when the stitching anomaly flag is 1 during actual operation. i ,t i ] ′ Store the self-correcting extrinsic values ​​and splicing exception flags independently;

[0075] When the stitching error flag is 1, the camera external parameters call the self-correcting external parameter value [R]. i ,t i ] ′ When the exception flag is 0, the camera extrinsic parameters call the original extrinsic parameter values ​​[R]. i ,t iThe model automatically calls different extrinsic parameter values ​​based on the values ​​of different splicing anomaly flags to correct the position of the fisheye camera in the world coordinate system.

[0076] When searching for or parking a car in a state where the stitching anomaly flag is 1, the model needs to detect two corner points of the parking space in order to construct a valid parking space. For the initially constructed parking space, if the trunk door is closed during the parking process, the parking process needs to use the rear fisheye camera to perform closed-loop correction on the initial parking space.

[0077] When parking in a vehicle with the splicing anomaly flag set to 1, to avoid decreased image recognition accuracy due to the camera being in an abnormal working position, the original vehicle model needs to be expanded horizontally and vertically by [x1, y1] to further reduce the risk of collision.

[0078] Example 2: An automatic parking state control method based on vehicle body state self-sensing using the system provided in Example 1, including the following steps:

[0079] S1: Real-time verification of the image stitching status of the vehicle surround view camera system, determining whether the surround view camera system is in a normal working position, and generating a stitching status flag bit;

[0080] S2: Real-time monitoring of one or more vehicle signals, including rearview mirror, driver's door, passenger door, and trunk door, to determine whether the current vehicle signal is in normal working condition, so as to generate a vehicle signal;

[0081] S3: Combining the splicing status flag and the vehicle body signal, determine whether the current vehicle body state meets the preset automatic parking operation conditions and generate a vehicle body status flag. Based on the vehicle body status flag, dynamically control the operation state of the automatic parking function. The operation state includes at least an active state, an inhabited state, and an OFF state; wherein:

[0082] When it is determined that the preset automatic parking operating conditions are met, the automatic parking function is allowed or maintained in an active state.

[0083] When it is determined that the preset automatic parking operating conditions are not met, the automatic parking function is switched to a suppressed state or a closed state, and the ESC is requested to perform a safety operation. The safety operation includes performing the corresponding deceleration, activating the automatic parking function, or activating the electronic parking brake.

[0084] And when the vehicle's status changes from not meeting the preset automatic parking operating conditions to meeting them, it requests ESC to resume or continue executing parking-related vehicle motion control;

[0085] S4: Provide users with feedback information related to the current operating status or required operation of the automatic parking function through the human-computer interaction interface.

[0086] like Figure 5 When the automatic parking / remote parking function is off, the signal processing module records and stores a fault code when it detects an abnormal vehicle signal. It does not output an HMI prompt if the user does not activate the parking function. When the user attempts to activate the automatic parking / remote parking function in this state, the driver is reminded through the HMI or remote parking interface: Automatic parking system malfunction, please check.

[0087] When the automatic parking / remote parking function is off, the signal processing module detects that the vehicle body signal is normal, and the vehicle body status judgment module detects that the vehicle body status does not meet the requirements. If the user activates the parking function / remote parking function, the activation of the function will be prohibited, and the driver will be reminded through the HMI or remote parking interface: Please open the rearview mirror / close the door / close the trunk door.

[0088] When the automatic parking / remote vehicle relocation function is activated, if the signal processing module detects that the vehicle body signal is normal, but the vehicle body status judgment module detects that the vehicle body status does not meet the requirements, the automatic parking status control module enters inhibit mode, records the fault code, and reminds the driver through the HMI or remote parking interface: Please open the rearview mirror / close the door / close the trunk door, and enter slow exit mode.

[0089] like Figure 6 As shown, the parking controller requests the ESC to enter slow exit mode. The ESC decelerates according to the specified deceleration value, and the AutoHold function is activated after the vehicle comes to a complete stop.

[0090] (1) When the waiting time > threshold Xs, if the vehicle status is still not satisfied, the automatic parking / remote parking function will be exited, the automatic parking status control module will be switched to OFF, the parking controller will request ESC to pull up EPB to complete parking, and the driver will be reminded through HMI or remote parking interface: Automatic parking has been exited, please park the vehicle in a safe place.

[0091] (2) When the waiting time is less than the threshold Xs, if the vehicle body status meets the conditions, the automatic parking / remote vehicle relocation function is restored, the automatic parking status control module is switched to active, and the driver is reminded through the HMI or remote parking interface that the parking function has been restored. The automatic parking controller continues to request ESC to complete the subsequent parking route at the specified speed and distance.

[0092] When the automatic parking / remote vehicle relocation function is activated, if the signal processing module detects an abnormal vehicle signal, it records and stores the fault code. The parking controller requests the ESC to enter slow exit mode and requests the ESC to decelerate according to the specified deceleration value. After the vehicle comes to a complete stop, the EPB is engaged, the automatic parking status control module switches to OFF, and the driver is notified through the HMI or remote parking interface: Automatic parking system malfunction, please have it checked.

[0093] like Figure 7 The signal processing module performs E2E verification on the transmitted signals from the rearview mirror, front door, and trunk door in real time, and transmits the result (normal vehicle signal / abnormal vehicle signal) to the downstream automatic parking status control module.

[0094] like Figure 8 The vehicle body status judgment module receives verified status signals from the rearview mirror, front door, and trunk door. When the rearview mirror is open, the front door is closed, the trunk door is closed, and the concatenation anomaly flag is 0 for 5 consecutive loops, the module outputs a vehicle body status flag of 1: the vehicle body status meets parking requirements. (When the rearview mirror is folded, the front door is open, the trunk door is open, or the concatenation anomaly flag is 1) and the number of detected parking space corner points is ≥2 for 5 consecutive loops, the module outputs 1: the vehicle body status meets parking requirements. (When the rearview mirror is folded, the front door is open, the trunk door is open, or the concatenation anomaly flag is 1) and the number of detected parking space corner points is <2 for 5 consecutive loops, the module outputs 0: the vehicle body status does not meet parking requirements.

[0095] The following four prompts provide a concise and clear interaction with the user, informing them of the current system status, avoiding unnecessary panic, improving the usability and reliability of the automatic parking system, and enhancing the user's experience with the automatic parking system.

[0096] Message 1: Automatic parking has been disengaged. Please park your vehicle in a safe location.

[0097] Message 2: Automatic parking system malfunction, please have it checked;

[0098] Message 3: Please open the rearview mirror / close the door / close the trunk.

[0099] Message 4: Parking function has been restored.

[0100] CAN bus security is ensured by introducing message verification codes and encryption protocols to prevent hackers from forging commands (such as acceleration, braking, and steering) to control the vehicle. Two-way authentication and end-to-end encryption prevent man-in-the-middle attacks from tampering with commands when users remotely move their vehicles via a mobile app.

[0101] When the vehicle status flag is 1 and the vehicle signal abnormality flag is 0, and a user request to activate automatic parking is received, the parking state machine is controlled to switch from standby to active state; the ESC and EPS are requested to execute the angle, speed, distance and other requirements of normal working conditions; and the HMI module is requested to light up the automatic parking working indicator light to inform the user of the current automatic parking status.

[0102] When the vehicle status flag is 1 and the vehicle signal abnormality flag is 1, and a user request is received to activate automatic parking, the parking state machine is kept in the OFF state to disable the activation function, the fault code is recorded, and the HMI module is requested to inform the user of prompt information 2.

[0103] When the vehicle status flag is 0 and the vehicle signal abnormality flag is 0, and a user request is received to activate automatic parking, the parking state machine is kept in the OFF state to prevent activation; and the HMI module is requested to provide the user with prompt information 3.

[0104] When the automatic parking / remote vehicle relocation function is activated, and the vehicle body signal abnormality flag is 1, the automatic parking state machine is controlled to switch from active to inhabit, the parking system enters a safe state, and the automatic parking / remote vehicle relocation function is slowly exited. The ESC is requested to decelerate according to the specified deceleration value am / s^2. After the vehicle comes to a complete stop, the EPB is engaged, and the user is informed of prompt message 2 through the HMI module. Finally, the parking state machine is controlled to switch to OFF.

[0105] When the automatic parking / remote vehicle relocation function is activated, and the vehicle status flag is 0 and the vehicle signal abnormality flag is 0, the automatic parking state machine switches from active to inhabit, records the fault code, and notifies the user of prompt message 3 through the HMI module, requests ESC to enter slow exit mode, and activates AutoHold when the vehicle speed is 0; when the vehicle status flag is 1 within the threshold X seconds, the automatic parking state machine switches from inhabit to active state, continues to complete the subsequent parking task, and notifies the user of prompt message 4 through the HMI module; if the vehicle status flag is still 0 within the threshold X seconds, the automatic parking state machine switches to OFF, requests the signal processing module to pull up the EPB caliper, and then notifies the user of prompt message 1 through the HMI module.

[0106] This invention establishes more stringent and reliable preconditions for automated parking by comprehensively sensing the vehicle's physical state (doors, rearview mirrors, trunk) and the operational status of key sensing systems (surround-view cameras) in real time (judged by stitching quality). Combined with dynamic state machine control logic and deep safety collaboration with vehicle control systems such as ESC, it significantly improves the safety, accuracy, and stability of the automated parking process while ensuring functional availability. Coupled with clear and effective human-machine interaction, it enhances the user experience and strengthens user trust in intelligent driving functions. This technical solution is applicable to various scenarios such as automated parking, remote parking, and remote vehicle relocation.

[0107] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. An automatic parking state control method based on vehicle body state self-sensing, characterized in that, Includes the following steps: S1: Real-time verification of the image stitching status of the vehicle surround view camera system, determining whether the surround view camera system is in a normal working position, and outputting stitching abnormality flag bit and self-correcting external parameter value; The surround-view camera system includes a front vehicle camera, a rear vehicle camera, and two side vehicle cameras. The stitched image generated by the surround-view camera system is input into a pre-trained deep learning network model for judgment. If the surround-view camera system is determined to be in an abnormal working position, the stitching abnormality flag is set to 1, and it is further judged whether the degree of deviation of the vehicle camera on either side from the normal working position is within a preset range θ. If so, the self-correcting extrinsic parameter value corresponding to the vehicle camera on either side at the time of the abnormal working position is learned and output. Adaptively adjust the extrinsic values ​​of the corresponding vehicle camera to correct and align the images acquired by the vehicle camera during stitching; Parking space corner points are detected and initial parking spaces are constructed based on the corrected and aligned stitched images; The deep learning network model is constructed and run in the following manner: Dataset preparation: Collect stitched images and corresponding camera extrinsic data of at least one vehicle-mounted camera in a preset normal working position as positive samples; and collect abnormal stitched images of at least one vehicle body component in a combination of multiple preset incompletely closed or incompletely open states, as well as the camera extrinsic data obtained by recalibration in these states as negative samples, wherein the stitching misalignment or distortion areas are manually marked in the negative sample images. Network model architecture: A YOLOv5-based network model is adopted, including: Input end: Process the input images to a preset size; Backbone network: Depthwise separable convolutions are introduced to reduce computation and model parameters, and the CBAM module is introduced to enhance the focus on regions of interest and feature representation capabilities, outputting feature maps of various scales; Multi-scale feature fusion module: fuses feature maps of multiple scales obtained from the backbone network, enhances semantic information through a top-down path, enhances detail information through a bottom-up path, and enriches feature expression by fusing feature maps of the same scale but different dimensions. Output module: It includes at least one classification output module and a regression prediction module. The classification output module and the regression prediction module each have at least one detection head, which is used to predict the probability of anomalies in the stitching region based on the fused features, and output information indicating whether the camera status is normal, including: stitching anomaly flag and self-correcting extrinsic value. S2: Real-time monitoring of one or more vehicle signals, including rearview mirror, driver's door, passenger door, and trunk door, to determine whether the current vehicle signal is in normal working condition, so as to generate a vehicle signal; S3: Combining the splicing anomaly flag and the vehicle body signal, determine whether the current vehicle body state meets the preset automatic parking operating conditions and generate a vehicle body state flag. Based on the vehicle body state flag, dynamically control the operating state of the automatic parking function. The operating state includes at least an active state, a suppressed state, and a deactivated state; wherein: When it is determined that the preset automatic parking operating conditions are met, the automatic parking function is allowed or maintained in an active state. When it is determined that the preset automatic parking operating conditions are not met, the automatic parking function is switched to a suppressed state or a closed state, and the ESC is requested to execute preset safety measures. And when the vehicle's status changes from not meeting the preset automatic parking operating conditions to meeting them, it requests ESC to resume or continue executing parking-related vehicle motion control; S4: Provide users with feedback information related to the current operating status or required operation of the automatic parking function through the human-computer interaction interface.

2. The automatic parking state control method based on vehicle body state self-sensing according to claim 1, characterized in that, When the trunk door signal is detected as closed, the initial parking space is corrected using the rear vehicle camera; the virtual vehicle model is expanded in the horizontal and vertical directions according to preset rules.

3. The automatic parking state control method based on vehicle body state self-sensing according to claim 1, characterized in that, S2 includes: The vehicle gateway receives the original signal corresponding to at least one of the following vehicle body signals: rearview mirror, driver's door, passenger door, and trunk door. An end-to-end verification is performed on the original signal to verify its integrity and correctness. If an abnormality is detected in the original signal during the end-to-end verification, a fault code is recorded and the corresponding vehicle body signal flag is set to the abnormal flag.

4. The automatic parking state control method based on vehicle body state self-sensing according to claim 1, characterized in that, Based on the combined information of the splicing anomaly flag and the vehicle body signal, the system determines whether the current vehicle body status meets the preset automatic parking operating conditions and generates a vehicle body status flag, including: When (rearview mirror = folded or driver's door = open or passenger's door = open or trunk door = open or splicing abnormal flag = 1) and the number of detected parking space corners is less than 2, and the duration of this state is greater than the first preset time threshold, the vehicle status flag indicating that the automatic parking operation conditions are not met will be output. When (rearview mirror = folded or driver's door = open or passenger's door = open or trunk door = open or splicing abnormal flag = 1) and the number of detected parking space corner points is ≥ 2, and the duration of this state is greater than the first preset time threshold, then output a vehicle status flag indicating that the automatic parking operation conditions are met. When (rearview mirror = open and driver's door = closed and passenger's door = closed and trunk door = closed and splicing abnormal flag = 0), and the duration of this state is greater than the second preset time threshold, then output a vehicle status flag indicating that the automatic parking operation conditions are met.

5. The automatic parking state control method based on vehicle body state self-sensing according to claim 4, characterized in that, When the automatic parking function is active, if the vehicle status flag changes from 1 to 0, the operating state of the dynamically controlled automatic parking function includes: Switch the automatic parking function to the disabled state; It then requests the ESC to perform a slow exit operation to decelerate the vehicle to a preset deceleration until it stops, and subsequently requests the electronic parking brake to perform a parking action.

6. The automatic parking state control method based on vehicle body state self-sensing according to claim 4, characterized in that, In the suppressed state, it also includes: Monitor whether the vehicle status flag is restored to 1 within a third preset time threshold. If it is restored, switch the automatic parking function back to the active state and resume the execution of the automatic parking task. If the vehicle status flag is still 0 after the third preset time threshold is exceeded, the operating state will be switched to the off state, and the ESC will be requested to activate the electronic parking brake to complete parking.

7. The automatic parking state control method based on vehicle body state self-sensing according to claim 1, characterized in that, It also includes applying information security mechanisms to process communication signals, wherein the information security mechanisms include at least one of the following: For commands or status information related to automatic parking control transmitted on the controller area network bus, a message authentication code is used for verification or an encryption protocol is used for encryption. Communication commands involving remote control of automatic parking functions are subject to information security processing, which includes two-way authentication and / or end-to-end encryption.

8. The automatic parking state control method based on vehicle body state self-sensing according to claim 1, characterized in that, The feedback information provided to the user through the human-machine interface includes at least one of the following: automatic parking system fault prompt, automatic parking function deactivated prompt, automatic parking function restored prompt, or operation instructions requiring the user to check and restore the vehicle body components to normal condition.

9. A system for implementing the automatic parking state control method based on vehicle body state self-sensing as described in claim 1, characterized in that, include: The image stitching verification module is configured to verify the image stitching status of the vehicle surround view camera system in real time, so as to determine whether the surround view camera system is in a normal working position, and output the camera stitching abnormality flag bit and self-correcting external parameter value. A signal processing module is configured to receive a signal indicating the status of at least one body component, the body component including at least one of a rearview mirror, driver's side door, passenger side door, and trunk door, perform end-to-end verification of the signal to verify its integrity and correctness, and output a body signal. The vehicle status judgment module is configured to determine whether the current combined vehicle status meets the preset automatic parking operation conditions based on the abnormal camera splicing flag and the processed and verified vehicle signals including at least preset types output by the signal processing module, and output the vehicle status flag. The automatic parking status control module is configured to dynamically control the operating status of the automatic parking system based on the vehicle status flag and the vehicle signal abnormality flag. When the combined vehicle status does not meet the automatic parking operating conditions or the vehicle signal abnormality flag indicates an abnormal signal, the automatic parking function is disabled or suppressed. The module interacts with the ESC module to execute preset safety measures. The automatic parking function is restored after the combined vehicle status recovers to meet the automatic parking operating conditions and the vehicle signal abnormality flag indicates a normal signal. The human-machine interface is used to output feedback information related to the current operating status of the automatic parking system.

Images