Anti-pinch detection method, device and equipment and computer readable storage medium

Abstract

The embodiment of the invention relates to the technical field of vehicles, and discloses an anti-pinch detection method, device and equipment and a computer readable storage medium. The method comprises the steps that sensor information collected by all sensors in the vehicle door closing process is obtained, the sensor information comprises three-dimensional point cloud information of the vehicle door, image information of a contact surface between the vehicle door and a door frame and door closing resistance information of the vehicle door; and based on the three-dimensional point cloud information, the contact surface image information and the door closing resistance information, whether the vehicle door clamps an object or not is determined. According to the technical scheme of the invention, multi-modal information fusion can be carried out by using the information collected by each sensor, anti-pinch detection is realized, and the anti-pinch detection accuracy is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of vehicles, in particular to a clamping detection method, device, equipment and computer readable storage medium. BACKGROUND

[0002] If an object is clamped during the closing process of a vehicle door, on the one hand, the vehicle door may not be able to be normally closed, and there may be a safety risk during driving, and on the other hand, if the force is large, the object may be damaged, such as finger clamping injury.

[0003] At present, it mainly depends on the driver to manually detect whether there is an object clamped in the closing process of the vehicle door.

[0004] However, this method is prone to missed detection, and the clamping detection accuracy is low. SUMMARY

[0005] In view of the above problems, embodiments of the present application provide a clamping detection method, device, equipment and computer readable storage medium, to solve the problem of low clamping detection accuracy of the vehicle door in the prior art.

[0006] According to an aspect of an embodiment of the present application, a clamping detection method is provided, the method comprising:

[0007] obtaining sensor information collected by each sensor during the closing process of the vehicle door, the sensor information comprising three-dimensional point cloud information of the vehicle door, contact surface image information of the vehicle door and door frame, and closing resistance information of the vehicle door;

[0008] determining whether the vehicle door clamps an object based on the three-dimensional point cloud information, the contact surface image information and the closing resistance information.

[0009] According to another aspect of an embodiment of the present application, a clamping detection device is provided, the device comprising:

[0010] an information obtaining module configured to obtain sensor information collected by each sensor during the closing process of the vehicle door, the sensor information comprising three-dimensional point cloud information of the vehicle door, contact surface image information of the vehicle door and door frame, and closing resistance information of the vehicle door;

[0011] a clamping detection module configured to determine whether the vehicle door clamps an object based on the three-dimensional point cloud information, the contact surface image information and the closing resistance information.

[0012] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus;

[0013] The memory is configured to store at least one executable instruction, and the executable instruction is configured to enable the processor to perform the operations of the method.

[0014] According to another aspect of the embodiments of the present application, a computer readable storage medium is provided, and the storage medium stores at least one executable instruction, and the executable instruction is configured to enable the electronic device or the anti-pinch detection device to perform the operations of the method.

[0015] The embodiments of the present application can collect three-dimensional point cloud information of the door, contact surface image information of the door and the door frame, and door closing resistance information of the door by different sensors, so as to perform multi-modal information fusion, realize anti-pinch detection, and improve the anti-pinch detection accuracy.

[0016] The above description is only a summary of the technical solutions of the embodiments of the present application, in order to more clearly understand the technical means of the embodiments of the present application, and to be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the embodiments of the present application more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings are included to provide a further understanding of the embodiments, and are incorporated herein and constitute a part of the specification. The drawings illustrate the embodiments, and together with the specification serve to explain the principles of the present application. In the drawings:

[0018] Figure 1 A scene schematic diagram of the anti-pinch detection method provided by the embodiments of the present application is provided.

[0019] Figure 2 A flowchart of the anti-pinch detection method provided by the embodiments of the present application is provided.

[0020] Figure 3 A tail gate closing schematic diagram provided by the embodiments of the present application is provided.

[0021] Figure 4 A dynamic anti-pinch detection flowchart provided by the embodiments of the present application is provided.

[0022] Figure 5 A static anti-pinch detection flowchart provided by the embodiments of the present application is provided.

[0023] Figure 6 A door anti-pinch control flowchart provided by the embodiments of the present application is provided.

[0024] Figure 7 A door anti-pinch system architecture schematic diagram provided by the embodiments of the present application is provided.

[0025] Figure 8 A structure schematic diagram of the anti-pinch detection device provided by the embodiments of the present application is provided.

[0026] Figure 9 A schematic diagram of the electronic device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0028] Door anti-pinch refers to the detection and prevention of doors from pinching passengers or other objects during the closing process. Currently, door anti-pinch detection mainly adopts the following two solutions: (1) Pressure sensor solution, which mainly judges resistance by the change of motor torque. It is easy to cause false triggering due to ambient temperature and mechanical wear, and cannot detect soft objects (such as clothing). (2) Infrared beam solution. It usually arranges infrared beams at the edge of the door and triggers and stops when blocked, but the installation is complicated and it is difficult to cover the full closing trajectory. Both solutions have a certain anti-pinch missed detection rate and low anti-pinch detection accuracy.

[0029] In response to the above problems, an anti-pinch detection method, device, equipment and computer-readable storage medium are provided in an embodiment of the present application. Various types of sensors are used to collect three-dimensional point cloud information of the car door during the door closing process, image information of the contact surface between the car door and the door frame, and closing resistance information of the car door, and then multimodal information fusion is performed to realize anti-pinch detection, which can improve detection accuracy.

[0030] Figure 1 A schematic diagram of a scenario of an anti-pinch detection method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the tailgate of the vehicle is in the open state. When the driver inputs the door closing command and controls the tailgate to close, if there is a foreign object between the tailgate and the trunk door frame, this method can be used to detect it. In addition, in some scenarios, the vehicle has a left door and a right door in addition to the tailgate (for example, Figure 1 The present invention can also realize anti-pinch detection for the left and right doors of the vehicle.

[0031] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0032] Figure 2 This is a flow chart of the anti-pinch detection method provided in an embodiment of the present application, which is executed by an electronic device carried on a vehicle. Figure 2 As shown, the method includes the following steps:

[0033] In step 210, sensor information collected by each sensor during the closing process of the vehicle door is acquired.

[0034] The sensor information includes three-dimensional point cloud information of the vehicle door, contact surface image information of the vehicle door and the door frame, and door closing resistance information of the vehicle door.

[0035] In step 220, whether the vehicle door clamps an object is determined based on the three-dimensional point cloud information, the contact surface image information, and the door closing resistance information.

[0036] In the embodiment of the present application, the three-dimensional point cloud information of the vehicle door, the contact surface image information of the vehicle door and the door frame, and the door closing resistance information of the vehicle door are collected by different sensors, so that multi-modal information fusion can be performed, anti-pinch detection can be realized, and the accuracy of anti-pinch detection can be improved.

[0037] In the embodiment, the vehicle can be a new energy electric vehicle. During the closing process of the vehicle door, the battery in the vehicle can be used to power each sensor so that each sensor can work normally.

[0038] For the above step 210, the closing process of the vehicle door can be divided into multiple stages, for example, a total of three stages, as follows:

[0039] Stage one: the vehicle receives a door closing instruction;

[0040] Stage two: the vehicle controls the vehicle door to start moving towards the door frame based on the door closing instruction;

[0041] Stage three: the vehicle door moves to the door frame, the locking mechanism is locked, and the closing is completed.

[0042] In the embodiment, when the vehicle receives the door closing instruction, each sensor can be triggered to start and collect information. For example, the sensors can include a binocular wide-angle camera, a macro camera array, and an auxiliary sensor,

[0043] For example, the rear door (i.e. Figure 1 The binocular wide-angle camera can be installed on the top inner side of the rear door, covering the full trajectory of the closing of the rear door (horizontal field of view of the binocular wide-angle camera ≥ 120 degrees, vertical field of view ≥ 80 degrees), the macro camera is arranged on the inner side of the door seal strip and is used for gap detection after the closing of the rear door (resolution ≥ 5 million pixels, accuracy 0.1 millimeter), and the auxiliary sensor can include a pressure sensor, which can be used for cross verification with the sensor information collected by the camera to realize anti-pinch detection.

[0044] In addition, the sensor can also include an inertial measurement unit (IMU, Inertial Measurement Unit), which can be used for monitoring the motion posture of the vehicle door.

[0045] In this embodiment, the binocular wide-angle camera is used to generate three-dimensional point cloud information collected during the closing process of the vehicle door. The pressure sensor is used to collect the closing resistance information of the vehicle door during the closing process. For example, when the vehicle door encounters an obstacle during the closing process, the closing resistance will increase dramatically. The macro camera uses structured light projection to detect the contact surface between the door seal and the door frame of the vehicle body, and obtains the contact surface image information of the vehicle door and the door frame.

[0046] For the above step 220, the three-dimensional point cloud information can be used to identify the foreign matter, the closing resistance can be used to further determine whether the vehicle door is pressed against the foreign matter during the closing process, and the contact surface image can be used to identify whether the vehicle door is tightly closed, so as to ensure that there is no foreign matter clamped between the vehicle door and the door frame after the vehicle door is closed.

[0047] For example, Figure 3 The tail door closing schematic diagram provided by the embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, when the driver triggers the tail door closing instruction, the tail door of the vehicle should change from the state shown in FIG. 1 to the state shown in FIG. 2. Figure 3 Figure 1 Figure 3

[0048] During the process that the tail door changes from the state shown in FIG. 1 to the state shown in FIG. 2, if the three-dimensional point cloud information collected by the binocular wide-angle camera detects that there is a static or dynamic object on the closing path of the tail door, it means that there is a foreign matter blocking the closing of the tail door. Further, if the pressure sensor also detects that the closing resistance increases dramatically, it means that the tail door begins to clamp the foreign matter. At this time, the anti-clamping operation can be performed. Figure 1 Figure 3 If the pressure sensor does not detect that the closing resistance increases dramatically (for example, the foreign matter is some relatively flexible object, which may not be detected by the pressure sensor even if it blocks the closing path of the tail door), the tail door may continue to perform the closing action and finally close. Further, after closing, the contact surface image between the tail door and the tail door frame is collected by the macro camera, so as to determine whether there is a foreign matter clamped (for example, clothes are clamped between the tail door and the tail door frame after the tail door is closed).

[0049] Considering that different objects have different characteristics (for example, clothes are relatively soft, and arms have relatively high hardness), in order to ensure the safety, reliability and scene applicability of the closing process of the vehicle door, in some embodiments, the anti-clamping detection during the closing process of the vehicle door can be divided into the following two stages:

[0050] Dynamic stage: anti-clamping detection during dynamic movement of the vehicle door;

[0051] Dynamic stage: anti-clamping detection during dynamic movement of the vehicle door;

[0052] ​​​​Static phase: Anti-pinch detection after the door is closed.

[0053] For the dynamic phase, whether the first object is pinched before the door is closed can be determined based on the three-dimensional point cloud information and the door closing resistance information; and for the static phase, whether the second object is pinched after the door is closed can be determined based on the contact surface image information.

[0054] The characteristics of the first object and the second object are different, for example, when the door is closing, the first object is an arm, which can block the movement of the door to a certain extent, and the second object can be clothes, which cannot effectively block the movement of the door to the closing direction. If the pinching situation cannot be identified, the pinching detection can be performed by using the macro lens camera after the door is closed.

[0055] In the embodiments of the present application, by setting the dynamic phase and the static phase during the closing of the door, if the dynamic phase detects that the first object (such as an arm) is pinched during the movement of the door, an anti-pinch operation can be immediately taken to prevent injury; and if the static phase detects that the second object is pinched, a remedial measure can be taken to prevent the person or object from being in a dangerous position, and it can also be determined whether the door is completely closed, thereby improving safety. At the same time, through the cooperation of the dynamic phase and the static phase, in the case that the anti-pinch detection of the dynamic phase is inaccurate, the static phase can be used for supplementary detection, which can improve the accuracy of detection.

[0056] Further, in some embodiments, Figure 4 A dynamic anti-pinch detection flowchart provided by the embodiments of the present application is shown in FIG. 4, which includes the following steps: Figure 4 As shown in FIG. 4, the method includes the following steps:

[0057] Step 410, based on the three-dimensional point cloud information, determining a movement trajectory of the door during the movement of the door to the closing direction, and whether the first object exists on the movement trajectory;

[0058] Step 420, if the first object exists, determining whether the door closing resistance is greater than or equal to a preset pressure threshold according to the door closing resistance information;

[0059] Step 430, if the door closing resistance is greater than or equal to the preset pressure threshold, determining that the first object is pinched before the door is closed to the closing direction.

[0060] In the embodiments of the present application, by using the door closing resistance information collected by the pressure sensor and the three-dimensional point cloud information collected by the binocular wide-angle camera, the accuracy of the anti-pinch detection of the dynamic phase can be improved, and injury can be prevented.

[0061] For the above step 410, the binocular wide-angle camera simulates the stereoscopic vision of human eyes through two images with different perspectives, so as to calculate the depth information and form a point cloud in a three-dimensional space. Based on the point cloud, a three-dimensional space data model can be reconstructed in real time to realize the perception of the surrounding environment of the vehicle door during the closing process.

[0062] In this embodiment, the information captured by the binocular camera after being processed into a point cloud can include relevant vehicle body structures and obstacles around the vehicle door, in addition to the movement trajectory of the vehicle door.

[0063] Among them, the obstacles can be divided into dynamic obstacles and static obstacles. During the closing process of the vehicle door, the vehicle body or the door frame can be used as a reference to determine which obstacles are dynamic obstacles and which obstacles are static obstacles. For example, the static obstacle can be a trunk, and the dynamic obstacle can be an arm, a pet, etc.

[0064] Among them, after generating three-dimensional point cloud information by using the binocular wide-angle camera, a three-dimensional space model of the vehicle door movement trajectory can be constructed, and by comparing the scale-invariant feature transform (SIFT) feature points in the continuous frame images, static or dynamic obstacles existing on the movement trajectory can be identified. In addition, the object size of the static or dynamic obstacle can also be determined based on the three-dimensional point cloud information generated by the binocular wide-angle camera.

[0065] Further, a size threshold, for example, 20 mm x 20 mm, can also be set. If the object size of the static or dynamic obstacle is greater than the size threshold, it can be directly determined that there is a pinching situation, and the anti-pinch operation can be directly triggered at this time. In addition, if the object size of the static or dynamic obstacle cannot be accurately estimated, the subsequent step 420 can also be executed to further confirm whether there is a pinching situation.

[0066] In addition, in some embodiments, if the vehicle door is driven to move in the closing direction by the motor, the existence of a pinching situation can be verified by the sudden change of the motor current (when the vehicle door encounters an obstacle, the motor load increases).

[0067] For the above step 420, the preset pressure threshold can refer to the door closing resistance collected by the pressure sensor during the movement of the vehicle door at the previous time. That is, after the first object is found on the movement trajectory by the binocular wide-angle camera, the size relationship between the door closing resistance collected by the pressure sensor at the current time and the door closing resistance collected by the pressure sensor at the previous time can be compared to determine whether the vehicle door has contacted the first object or has pinched the first object.

[0068] For the above step 430, the door closing can refer to that the door has been moved to a specific position and has completed the latching operation. Exemplarily, if the object is a first object with a larger size or a certain hardness, the object is clamped between the door and the door frame, causing a larger gap between the door and the door frame, so that the door cannot complete the closing. If the object is a second object with a smaller size or a certain softness, even if the object is clamped between the door and the door frame, a larger gap between the door and the door frame is not caused, and the door can still complete the closing.

[0069] Further, in some embodiments, Figure 5 A static anti-pinch detection flowchart provided for the embodiments of the present application is shown in FIG. 6, which includes the following steps: Figure 5

[0070] Step 510, detecting whether there is an abnormal object contour image in the contact surface image.

[0071] Step 520, if there is an abnormal object contour image, determining whether there is a second object between the door frame and the door based on the abnormal object contour image.

[0072] Step 530, if there is a second object, determining that the second object is clamped after the door is closed.

[0073] In the embodiments of the present application, after completing the anti-pinch detection in the dynamic stage, the static anti-pinch detection is further performed, so that the clothes or the sheet object that is not recognized by the dynamic stage can be avoided to be clamped, and the anti-pinch detection accuracy is further improved.

[0074] For the above step 510, a deep learning model can be used to segment the abnormal object contour image from the contact surface image. Exemplarily, the abnormal object can refer to an object other than the door, the door frame and other structures of the vehicle body.

[0075] Among them, a large number of contact surface sample images can be collected in advance by the macro lens camera, and the abnormal objects existing in the contact surface sample images are marked by a marking box, which is used as training data to train the deep learning model, so that the deep learning model can identify the abnormal object image in the contact surface image, and extract the contour image of the abnormal object.

[0076] In addition, the image segmentation algorithm can also be used to segment the abnormal object contour image. The contact surface sample image is used as an input image, each pixel of the input image is assigned to a specific category (such as an abnormal object, a non-abnormal object, etc.), and a segmentation mask consistent with the size of the original image is generated to segment the contour of the abnormal object.

[0077] ​The segmentation process can specifically include the following steps: 1. normalizing the image size; 2. extracting features layer by layer and down-sampling to low-dimensional semantic representation; 3. up-sampling and fusing the features of the skip connection to gradually restore the spatial resolution; and 4. using an activation function to generate a pixel-level classification in the final layer to obtain a segmentation mask consistent with the original image size.

[0078] For steps 520 and 530, the contour image of the second object can be pre-configured, and after obtaining the abnormal object contour image, the pre-configured contour image of the second object is directly compared with the abnormal object contour image. If the similarity is greater than the set similarity threshold, it is determined that the second object is clamped after the door is closed.

[0079] In addition, the abnormal object can also be identified in real time based on the abnormal object contour image. If the abnormal object is a clothing, a mobile phone case, or a sheet, it is determined to be the second object, and prompt information that the second object is clamped after the door is closed is output.

[0080] After detecting that the door clamps the first object or the second object, in order to avoid damage to the first object or the second object and improve safety, an anti-pinch operation needs to be performed. In some embodiments, in order to improve the anti-pinch effect, the door performs a first anti-pinch operation when clamping the first object, and performs a second anti-pinch operation when clamping the second object. That is, the anti-pinch operation performed by the door is different when clamping different objects. Thus, the anti-pinch effect can be improved by configuring different anti-pinch operations.

[0081] For example, Figure 6 The vehicle door anti-pinch control flowchart provided by the embodiments of the present application is shown in Figure 6 As shown, based on the above sensor information, taking the tail door anti-pinch of the vehicle as an example, the anti-pinch control can be divided into dynamic anti-pinch detection and static anti-pinch detection stages.

[0082] For the dynamic anti-pinch detection stage, three-dimensional point cloud data collected by the binocular camera is mainly used for real-time trajectory modeling, and a door movement trajectory is constructed. Then, based on this, foreign object detection is performed, and multi-modal verification is implemented in combination with the pressure information collected by the pressure sensor to determine whether there is a clamping object during the dynamic movement of the door.

[0083] For the static anti-pinch detection stage, the micro-lens camera is mainly used for high-precision gap scanning of the door and the door frame to determine whether there is a residual object (i.e., a foreign object) between the door and the door frame. If there is a foreign object, a corresponding response strategy is executed to implement the anti-pinch operation.

[0084] For example, in some embodiments, in the dynamic detection stage, if it is detected that the door clamps the first object, the door can be directly controlled to retreat in the opposite direction of the closing direction by a first preset distance.

[0085] When the first object is detected to be clamped during the movement of the vehicle door, the emergency stop of the vehicle door can be triggered first to reduce the injury to the clamped first object, and then the vehicle door is reversely driven to retreat in the opposite direction. For example, the first preset distance of the retreat can be 20 cm.

[0086] In this embodiment, taking the child's arm as the first object, the driver can open the tail gate closing instruction through remote control, at which time the vehicle automatically performs the tail gate closing operation, and the motor drives the tail gate to start moving in the closing direction. During the movement of the tail gate, double verification can be performed based on the three-dimensional point cloud data collected by the wide-angle binocular camera and the pressure data collected by the pressure sensor to determine whether the tail gate clamps the child's arm.

[0087] Among them, a confidence threshold can be set. After double verification based on the three-dimensional point cloud data and the pressure data collected by the pressure sensor, if the confidence of clamping the child's arm represented by the verification result is greater than the confidence threshold, the first anti-pinch operation is triggered.

[0088] Specifically, the power supply of the motor can be immediately cut off, and then the electric supporting rod and the like are controlled to start and support the tail gate in the opposite direction by 20 cm.

[0089] Among them, after the vehicle door clamps the first object, each component needs to respond quickly to cooperate to complete the first anti-pinch operation to reduce the clamping injury. The response time should be controlled within milliseconds, for example, less than or equal to 100 milliseconds.

[0090] Among them, in order to reduce the response time as much as possible, each component can be started in advance, so that they can all quickly enter the working state as soon as it is detected that the vehicle door clamps the first object.

[0091] In addition, in some embodiments, after the first anti-pinch operation is triggered, a prompt that the tail gate closing fails can also be fed back to the vehicle display screen or the remote control device, so that the driver can quickly know that the tail gate clamps the child's arm.

[0092] In addition, in some embodiments, if the second object is detected to be clamped in the static detection stage, different anti-pinch operations can be configured for the hardness level of the second object, and the specific operations are as follows:

[0093] (1) If the hardness level is less than a preset level threshold, the closing lock of the vehicle door is controlled to be opened, and the supporting rod assembly of the vehicle door is controlled to support the vehicle door until the relative distance between the vehicle door and the door frame is greater than or equal to a first distance threshold.

[0094] (2) If the hardness level is greater than or equal to the preset level threshold, the door lock is controlled to open, and the door support rod assembly is controlled to prop up the door until the relative distance between the door and the door frame is greater than or equal to a second distance threshold.

[0095] The second distance threshold is greater than the first distance threshold.

[0096] Here, hardness refers to an object's ability to resist localized pressure from a car door. For example, the hardness level can be divided into 10 levels. A higher hardness level indicates a greater resistance to localized plastic deformation, meaning it is less susceptible to scratches, pressure, or penetration. For example, the preset level threshold can be 2.

[0097] In this embodiment, after the vehicle door is closed, in order to improve safety and prevent the vehicle door from detaching from the door frame on bumpy roads, a suction locking mechanism is usually provided. When the vehicle door is closed, the suction locking mechanism will lock, thereby fixing the vehicle door more stably.

[0098] Among them, when the car door is closed, if it is found that a second object is clamped, it is necessary to first release the suction locking mechanism (i.e., open the suction locking mechanism) to prevent the suction force generated by the suction locking mechanism from causing continuous damage to the object.

[0099] In addition, in some embodiments, the volume of the second object may also be obtained. If the volume of the second object is greater than a preset volume threshold, the suction locking mechanism of the vehicle door may be controlled to open first.

[0100] In this embodiment, after the pull-in locking mechanism is opened, the door pop-up operation is performed based on the hardness level of the second object. The first distance threshold may be 30 mm, and the second distance threshold may be 50 mm.

[0101] For example, if the second object is a soft material (such as clothing) with a low hardness level (i.e., less than a preset hardness threshold), the door can be controlled to open 30 mm, and the vehicle's lights and speakers can emit an audible and visual warning to inform the driver of the pinching situation. If the object is hard or sharp (such as a door key), the door can be controlled to open 50 mm, and the vehicle's lights and speakers can emit an audible and visual warning to inform the driver of the pinching situation.

[0102] In addition, in some embodiments, the information can be pushed to the mobile device carried by the vehicle owner to inform the vehicle owner of the clamping situation, for example, by pushing the image of the second object to the mobile phone application carried by the vehicle owner.

[0103] In an embodiment of the present application, the distance at which the car door bounces up is controlled to be different based on the hardness level of the second object. The higher the hardness level of the object, the greater the distance at which the door bounces up. This allows the object to return to its shape before clamping, minimizing damage to the clamped second object.

[0104] The following is a detailed introduction to this solution based on actual scenarios.

[0105] For example, Figure 7 This is a schematic diagram of the door anti-pinch system architecture provided in the embodiment of the present application, as shown in FIG. Figure 7 As shown in the figure, it is divided into a perception layer and a control layer. The perception layer includes auxiliary sensors 71, a macro camera array 72, and a binocular wide-angle camera 73. The auxiliary sensors may specifically include pressure sensors and inertial measurement units. The control layer includes an embedded vision processor 74, a body domain controller (BDC) 75, and a cockpit domain controller (CDC) 76. The perception layer and the control layer communicate via a controller area network with flexible data rate (CANFD).

[0106] Among them, the embedded vision processor 74 can use an object recognition algorithm, such as YOLOv5+DeepSort to dynamically track the movement trajectory of objects and car doors. After receiving relevant control instructions, the BDC can control the motor and / or the suction and locking mechanism, and feedback the door operation status to the CDC. The CDC receives the operation status and determines whether the door closing is successful. If it fails, it can be displayed on the central control screen through a text reminder.

[0107] During the dynamic anti-pinch detection phase, a binocular camera generates a 3D point cloud, constructing a three-dimensional spatial model of the door's motion trajectory. SIFT feature points in consecutive frame images are compared. If a static or dynamic object (such as a hand, pet, or luggage) is detected on the path, and the object's size is ≥ 20 mm x 20 mm, an emergency stop is triggered (with a response time of ≤ 100 milliseconds). Simultaneously, the pressure sensor data is read, and if a sudden increase in pressure matches the visual recognition result, it is determined to be a real pinching risk.

[0108] For example, taking the driver's remote opening and closing command of the vehicle's tailgate as an example, the binocular camera captures images at a 30 frame rate to generate a real-time 3D spatial model. When the first object (such as a child's arm) is detected on the motion trajectory and the confidence level is ≥95%, the BDC immediately cuts off the motor power supply and reverse drives the door to open 20 cm. At the same time, it feeds back the reason for the tailgate closing failure to the CDC and controls the on-board display screen to prompt "Anti-pinch triggered, please remove the obstacle."

[0109] For the static anti-pinch detection phase, the macro lens camera is started after the door is closed, the contact surface of the weather strip and the vehicle body is detected by structure light projection, the foreign object profile is identified by an image segmentation algorithm (such as U-Net), and if the volume of the residual object is greater than or equal to a preset volume threshold (such as 5 cubic centimeters) or the hardness level is greater than or equal to 2, the door opening instruction is triggered. Among them, the opening instruction adopts a hierarchical response strategy, which is as follows:

[0110] If the object is a soft material (such as cloth), the tail door is opened by 30 mm and an audible and visual prompt is issued; if the object is a hard or sharp object (such as a key), the door is opened by 50 mm and a mobile phone application warning is pushed synchronously.

[0111] For example, continuing to take the door rear door suction locking as an example, the macro lens camera is started to scan and identify that the door weather strip gap is clamped into a mobile phone, and the hardness level is determined to be level 3; the BDC control suction locking mechanism is released, and the electric support rod opens the door by 50 mm; and the user is reminded through Bluetooth positioning that "the electronic device is clamped on the left side of the trunk".

[0112] Compared with the traditional anti-pinch detection scheme, only the anti-pinch during the closing process is concerned, and there is no remedial measure after the door is closed. If the object remains in the closing gap, it cannot be automatically processed and manual intervention is required. The embodiment of the present application realizes full-cycle anti-pinch protection through two-stage visual detection (dynamic + static) and multi-sensor fusion decision.

[0113] Figure 8 The structure diagram of the anti-pinch detection device provided in the embodiment of the present application is shown in FIG. 8. Figure 8 As shown in FIG. 8, the anti-pinch detection device 800 includes:

[0114] The information acquisition module 810 is configured to acquire sensor information collected by each sensor during the closing process of the door, and the sensor information includes three-dimensional point cloud information of the door, contact surface image information of the door and the door frame, and door closing resistance information of the door.

[0115] The pinch detection module 820 is configured to determine whether the door pinches an object based on the three-dimensional point cloud information, the contact surface image information, and the door closing resistance information.

[0116] In the embodiment of the present application, the three-dimensional point cloud information of the door, the contact surface image information of the door and the door frame, and the door closing resistance information of the door are collected by different sensors, so that multi-modal information fusion can be performed, anti-pinch detection can be realized, and anti-pinch detection accuracy can be improved.

[0117] In an optional manner, the pinch detection module can be specifically configured to:

[0118] determine, based on the three-dimensional point cloud information and the door closing resistance information, whether the first object is clamped by the door before the door is closed in the closing direction;

[0119] determine, based on the contact surface image information, whether the second object is clamped by the door after the door is closed.

[0120] In an optional manner, the clamping detection module can be specifically configured to:

[0121] determine, based on the three-dimensional point cloud information, a movement trajectory of the door in the closing direction, and whether the first object exists on the movement trajectory;

[0122] if the first object exists, determine, according to the door closing resistance information, whether the door closing resistance is greater than or equal to a preset pressure threshold;

[0123] if the door closing resistance is greater than or equal to the preset pressure threshold, determine that the first object is clamped by the door before the door is closed in the closing direction.

[0124] In an optional manner, the clamping detection module can be specifically configured to:

[0125] detect whether an abnormal object contour image exists in the contact surface image;

[0126] if the abnormal object contour image exists, determine, based on the abnormal object contour image, whether the second object exists between the door frame and the door;

[0127] if the second object exists, determine that the second object is clamped by the door after the door is closed.

[0128] In an optional manner, the method further includes a clamping prevention operation module configured to: if the first object is clamped by the door, perform a first clamping prevention operation; and if the second object is clamped by the door, perform a second clamping prevention operation.

[0129] In an optional manner, the clamping prevention operation module can be specifically configured to: control the door to retreat in a direction opposite to the closing direction by a first preset distance.

[0130] In an optional manner, the clamping prevention operation module can be specifically configured to:

[0131] obtain a hardness level of the second object;

[0132] if the hardness level is less than a preset level threshold, control a suction locking mechanism of the door to open, and control a support rod assembly of the door to support the door until a relative distance between the door and the door frame is greater than or equal to a first distance threshold;

[0133] If the hardness level is greater than or equal to the preset level threshold, the attraction locking mechanism of the vehicle door is controlled to open, and the support rod assembly of the vehicle door is controlled to support the vehicle door until the relative distance between the vehicle door and the door frame is greater than or equal to a second distance threshold, the second distance threshold being greater than the first distance threshold.

[0134] Figure 9 The electronic device structure schematic diagram provided for the embodiments of the present application does not limit the specific implementation of the electronic device. As shown in the figure, Figure 9 The electronic device can include one or more processors 901 and a communication interface 903. The one or more processors 901 are configured to execute the steps in the above method embodiments.

[0135] The electronic device can further include a memory 902 and a communication bus 904.

[0136] The processor 901, the communication interface 903, and the memory 902 can communicate with each other through the communication bus 904. The communication interface 903 is configured to communicate with network elements such as clients or other servers. The processor 901 is configured to execute the program 905, and can execute the related steps in the above method embodiments.

[0137] Specifically, the program 905 can include program code including computer executable instructions. The processor 901 can be a central processing unit CPU, or an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement embodiments of the present application. The one or more processors included in the electronic device can be the same type of processor, such as one or more CPUs; or different types of processors, such as one or more CPUs and one or more ASICs.

[0138] The memory 902 is configured to store the program 905. The memory 902 can include a high-speed RAM memory, and can also include a non-volatile memory, such as at least one disk memory.

[0139] The program 905 can be specifically invoked by the processor 901 to enable the electronic device to perform the following operations:

[0140] Obtain sensor information collected by various sensors during the closing process of the vehicle door, the sensor information including three-dimensional point cloud information of the vehicle door, contact surface image information of the vehicle door and the door frame, and closing resistance information of the vehicle door;

[0141] Determine whether the vehicle door clamps an object based on the three-dimensional point cloud information, the contact surface image information, and the closing resistance information.

[0142] The embodiments of the present application provide a computer readable storage medium, the storage medium stores at least one executable instruction, the executable instruction enables the electronic device or the encryption device to execute the method in any method embodiment described above when running on the electronic device or the encryption device. The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Moreover, embodiments of the present application are also not directed to any particular programming language.

[0143] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Moreover, embodiments of the present application are also not directed to any particular programming language.

[0144] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. Similarly, in order to simplify the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description of it. Among them, the claims following the specific embodiments are thus explicitly incorporated into the specific embodiments, wherein each claim itself is a separate embodiment of the present application.

[0145] Those skilled in the art can understand that the modules in the device in the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive.

[0146] It should be noted that the above embodiments illustrate the application rather than limit the application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not constitute a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, the several means can be embodied by one and the same item of hardware. The use of the words first, second and third, etc. does not imply any order. These words have been used to name the circumstances in which the embodiments have been described, but there can be several alternative. The sequence of steps in the embodiments described above need not necessarily be the sequence in which these steps are executed.

Claims

1. A method for detecting pinching, characterized in that: The method comprises: Acquiring sensor information collected by various sensors during the door closing process, the sensor information including three-dimensional point cloud information of the door, image information of the contact surface between the door and the door frame, and door closing resistance information of the door; Based on the three-dimensional point cloud information, the contact surface image information and the door closing resistance information, it is determined whether the vehicle door is holding an object.

2. The method according to claim 1, characterized in that The determining whether the vehicle door is holding an object based on the three-dimensional point cloud information, the contact surface image information, and the door closing resistance information includes: determining, based on the three-dimensional point cloud information and the door closing resistance information, whether the vehicle door is holding a first object when the vehicle door moves in a closing direction and before closing; Based on the contact surface image information, it is determined whether a second object is clamped by the vehicle door after closing.

3. The method according to claim 2, characterized in that The determining, based on the three-dimensional point cloud information and the door closing resistance information, whether the vehicle door holds a first object before moving in the closing direction and before closing, includes: determining, based on the three-dimensional point cloud information, a motion trajectory of the vehicle door during movement in a closing direction, and whether the first object exists on the motion trajectory; If the first object is present, determining whether the door closing resistance of the vehicle door is greater than or equal to a preset pressure threshold according to the door closing resistance information; If the door closing resistance is greater than or equal to the preset pressure threshold, it is determined that the vehicle door is clamping the first object before moving in the closing direction and not closing.

4. The method according to claim 2, characterized in that The determining, based on the contact surface image information, whether the vehicle door clamps a second object after closing, includes: detecting whether there is an abnormal object contour image in the contact surface image; If the abnormal object contour image exists, determining whether the second object exists between the door frame and the vehicle door based on the abnormal object contour image; If the second object exists, it is determined that the second object is clamped after the vehicle door is closed.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: If the vehicle door clamps a first object, performing a first anti-pinch operation; If the vehicle door clamps a second object, a second anti-pinch operation is performed.

6. The method according to claim 5, characterized in that The performing of the first anti-pinch operation includes: The vehicle door is controlled to retract a first preset distance in a direction opposite to the closing direction.

7. The method according to claim 5, characterized in that The performing of the second anti-pinch operation includes: obtaining a hardness level of the second object; If the hardness level is less than a preset level threshold, controlling the suction locking mechanism of the vehicle door to open, and controlling the strut assembly of the vehicle door to prop up the vehicle door until the relative distance between the vehicle door and the door frame is greater than or equal to a first distance threshold; If the hardness level is greater than or equal to the preset level threshold, the suction locking mechanism of the vehicle door is controlled to open, and the support rod assembly of the vehicle door is controlled to support the vehicle door until the relative distance between the vehicle door and the door frame is greater than or equal to a second distance threshold, and the second distance threshold is greater than the first distance threshold.

8. An anti-pinch detection device, characterized in that: The device comprises: an information acquisition module, configured to acquire sensor information collected by various sensors during the door closing process, the sensor information including three-dimensional point cloud information of the door, image information of the contact surface between the door and the door frame, and door closing resistance information of the door; A clamping detection module is used to determine whether the vehicle door clamps an object based on the three-dimensional point cloud information, the contact surface image information and the door closing resistance information.

9. An electronic device, characterized in that: include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one executable instruction. When the executable instruction is executed on the electronic device or the anti-pinch detection device, the electronic device or the anti-pinch detection device performs the operation of the method according to any one of claims 1 to 7.

Images