Method and device for determining opening degree of vehicle door and vehicle

Abstract

The invention provides a method and device for determining the opening degree of a vehicle door and a vehicle, and the method comprises the steps that an environment image of the outer side of the vehicle is collected, and the environment image comprises an obstacle; determining an obstacle area corresponding to the obstacle in the environment image; and the maximum opening degree of the vehicle door is determined according to the overlapping condition of a corresponding door opening area, in the environment image, of the vehicle door and the obstacle area, and the door opening area is an area scanned in the process from closing to opening of the vehicle door. According to the method, the maximum opening degree of the vehicle door is determined according to the overlapping condition between the door opening area and the obstacle area. Therefore, the maximum opening degree of the vehicle door can be obtained without establishing the mapping relation between the image coordinates and the actual physical distance through additional operations such as anti-distortion and camera calibration, so that the opening degree of the vehicle door is controlled, and the hardware cost and the deployment difficulty are reduced.

Description

Technical Field

[0001] This application relates to the field of vehicle control, and more particularly to a method, apparatus, and vehicle for determining the opening degree of a vehicle door. Background Technology

[0002] With the rapid development of vehicle intelligence and connectivity, smart doors have become a key area of ​​competition among automakers. One of the core trends is the refined control of door opening, which requires determining the maximum safe opening range of the door based on information about the surrounding environment. The key to achieving this goal is to accurately perceive the position and distance of obstacles around the door. Therefore, obstacle distance measurement and recognition have become the core technologies of smart door systems.

[0003] Current mainstream ranging solutions are divided into radar solutions and vision solutions. Radar solutions include ultrasonic radar and millimeter-wave radar, while vision solutions include binocular vision and monocular vision. These ranging solutions can meet the requirements of door opening control to a certain extent, but they have core problems such as difficult deployment, high cost, and weak scene adaptability. Summary of the Invention

[0004] This application provides a method, device, and vehicle for determining the opening degree of a vehicle door, so as to control the opening degree of the vehicle door and reduce hardware costs and deployment difficulty.

[0005] In a first aspect, a method for determining the opening degree of a vehicle door is provided, the method comprising: acquiring an environmental image of the exterior of a vehicle, the environmental image including obstacles; determining an obstacle region corresponding to the obstacle in the environmental image; and determining the maximum opening degree of the vehicle door based on the overlap between the door opening region corresponding to the vehicle door in the environmental image and the obstacle region, wherein the door opening region is the area swept by the vehicle door during the process of closing to opening.

[0006] In some implementations, the overlap condition includes: whether the intersection-union ratio between the door opening area and the obstacle area is greater than 0; and / or whether the minimum distance between the outline of the door opening area and the outline of the obstacle area is greater than a preset threshold.

[0007] In some implementations, determining the obstacle region corresponding to the obstacle in the environmental image includes: identifying the obstacle in the environmental image; segmenting the identified obstacle; generating a mask contour corresponding to the obstacle based on the segmentation result, and using the mask contour as the obstacle region.

[0008] In some implementations, the obstacle region is generated by an image segmentation network module, the input of which includes the environment image, and the output of which includes the mask contour.

[0009] In some implementations, the environmental image is a distorted image, and the mask contour is a mask contour that meets a predetermined accuracy, which is achieved by the cross-entropy loss function used in the image segmentation network module.

[0010] In some implementations, the training set of the image segmentation network module includes multiple acquired environmental images, wherein one or more of the following contents differ in the multiple environmental images: the category to which the obstacle belongs, the location of the obstacle, and the ambient light intensity.

[0011] In some implementations, the vehicle door includes multiple opening positions, each corresponding to a multiple opening area in the environmental image. The step of determining the maximum opening of the vehicle door based on the overlap between the opening area corresponding to the vehicle door in the environmental image and the obstacle area includes: determining the maximum opening position of the vehicle door based on the overlap between the multiple opening areas and the obstacle area, wherein the opening area corresponding to the maximum opening position is the largest opening area among the multiple opening areas that does not overlap with the obstacle area.

[0012] In some implementations, the method further includes: measuring the area swept by the vehicle door from closed to different opening positions; mapping the area swept by the vehicle door from closed to different opening positions onto the environmental image to obtain the multiple door opening areas in the environmental image corresponding to the multiple opening positions.

[0013] In some implementations, the environmental image is captured by a fisheye camera, which can be a monocular or multi-view camera, and the fisheye camera is positioned on the vehicle door.

[0014] In some implementations, the method further includes: outputting information about the maximum opening degree to a door drive module, wherein the door drive module controls the opening degree of the door based on the information about the maximum opening degree.

[0015] Secondly, a device for determining the opening degree of a vehicle door is provided. The device includes: a data acquisition module for acquiring an environmental image of the exterior of a vehicle, the environmental image including obstacles; and a processing module for: determining an obstacle region corresponding to the obstacle in the environmental image; and determining the maximum opening degree of the vehicle door based on the overlap between the door opening region corresponding to the vehicle door in the environmental image and the obstacle region, wherein the door opening region is the area swept by the vehicle door during the process of closing to opening.

[0016] Thirdly, a vehicle is provided, comprising: a transceiver, a memory, and a processor, wherein the memory is used to store a program, and the processor is used to invoke the program in the memory and control the transceiver to receive or transmit signals to cause the vehicle to perform the method as described in any one of the first aspects.

[0017] Fourthly, a computer-readable storage medium is provided, wherein program code is stored therein, and the program code can be invoked by a processor to execute the method as described in any one of the first aspects.

[0018] The technical solution provided in this application acquires environmental images of the vehicle's exterior and identifies the obstacle region corresponding to the obstacle and the door opening region swept across during the door's opening process from closing to opening. Based on the overlap between the door opening region and the obstacle region, the maximum door opening is determined. This eliminates the need for additional distortion correction and camera calibration to establish a mapping between image coordinates and actual physical distances, thus enabling control of the door opening and reducing hardware costs and deployment complexity. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the method for determining the door opening degree provided in an embodiment of this application.

[0020] Figure 2 This is a top view of the actual opening area of ​​the side-opening vehicle door provided in an embodiment of this application.

[0021] Figure 3 for Figure 1 A schematic diagram of the specific process for step 120.

[0022] Figure 4 This is a functional diagram of the image segmentation network module provided in an embodiment of this application.

[0023] Figure 5 This is a comparative diagram of the door opening area and the obstacle area provided in an embodiment of this application.

[0024] Figure 6 This is a schematic diagram of the opening area of ​​the sliding door in the environmental image provided in the embodiments of this application.

[0025] Figure 7 This is a schematic diagram of the maximum opening position provided in the embodiments of this application.

[0026] Figure 8 This is a schematic diagram illustrating a specific implementation method for determining the door opening degree provided in an embodiment of this application.

[0027] Figure 9 for Figure 8A diagram illustrating a more specific implementation of the method shown.

[0028] Figure 10 This is a schematic diagram of the structure of the door opening determination device provided in the embodiments of this application.

[0029] Figure 11 This is a structural schematic diagram of the vehicle provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0031] With the rapid development of automotive intelligence and connectivity, smart car doors, as one of the most direct manifestations of vehicle intelligence, are gradually becoming an important technological direction for automakers to seize market share. They not only enhance user experience and improve vehicle safety, but are also a key element for automakers to achieve differentiated competition and drive brand upgrades. The core development trend of smart car doors lies in achieving proactive intelligent sensing and precise door opening control, ensuring that car doors can automatically open and close flexibly and safely in various complex environments.

[0032] Door opening control, as a key technology in smart car doors, essentially combines information about the vehicle's surrounding environment to intelligently determine the maximum safe opening range of the door, thereby preventing collisions with external obstacles. The crucial challenge in achieving refined door opening control lies in accurately sensing the position and distance of obstacles around the door. Therefore, obstacle ranging and recognition have become core technologies of smart car door systems.

[0033] Currently, mainstream obstacle ranging solutions are mainly divided into two categories: radar solutions and vision solutions (also known as visual ranging technology). Radar solutions include ultrasonic radar and millimeter-wave radar. Ultrasonic radar has advantages such as small size and low cost, and is often used in short-range obstacle detection scenarios; however, its resolution and ranging range have certain limitations. Millimeter-wave radar has strong penetration capabilities and a long ranging range, making it suitable for complex environments, but its price is relatively high and integration is difficult. These radar solutions often need to be integrated with vision systems to improve scene perception capabilities. Vision solutions include binocular vision and monocular vision. Binocular vision uses two cameras to acquire stereo images and uses triangulation principles to achieve obstacle ranging, offering high accuracy but requiring strict hardware installation and relatively high cost. Monocular vision uses a single camera combined with corresponding algorithms to complete the ranging function; its hardware cost is low and deployment is flexible, but it requires high algorithm accuracy and model generalization ability.

[0034] As shown above, existing obstacle ranging technologies for intelligent vehicle doors mainly rely on radar or vision-based solutions. Among vision-based solutions, fisheye cameras are widely used in obstacle ranging technology for intelligent vehicle doors due to their wide field of view and ease of deployment. However, images captured by fisheye cameras suffer from severe distortion, resulting in a significant distortion of the spatial positional characteristics of obstacles in the image. Furthermore, existing monocular vision ranging technologies, especially monocular fisheye cameras, cannot theoretically obtain the absolute distance to obstacles directly. Therefore, existing monocular vision ranging technologies using fisheye cameras typically require geometric correction of the fisheye image (e.g., anti-distortion processing) and rely on complex camera intrinsic and extrinsic parameter calibration to establish a mapping relationship between image coordinates and actual physical distance to ensure the absolute distance to obstacles is obtained. Specifically, traditional technologies can improve ranging accuracy by dividing the monocular fisheye image into multiple sub-regions, calculating the world coordinates and error values ​​of obstacles in each sub-region, and then compensating for them. In other words, regional error compensation optimizes the ranging results to adapt to scenarios with large fisheye camera distortion. For example, using a single fisheye lens to acquire environmental images, combined with multiple steps such as neural network target detection, feature point extraction, image correction, and coordinate transformation, target distance measurement can be achieved. This method has the advantages of low cost, wide detection range, high speed, and high accuracy, and can adapt to the obstacle distance measurement needs of scenarios such as smart car doors. Another example is obtaining camera intrinsic parameters and distortion parameters through checkerboard calibration, and using inverse perspective transformation to convert the pixel coordinates in the distorted image to the actual physical scale, thereby achieving target distance measurement. Yet another example is the "DepthAnything" algorithm based on deep learning, which, through end-to-end model training, can achieve scene depth estimation on monocular images without external calibration parameters, providing new possibilities for obstacle distance measurement in smart car doors. While this solution simplifies the deployment process in practical applications, its accuracy and generalization ability still need further improvement.

[0035] In summary, while monocular vision ranging technology, especially that using fisheye cameras, can meet the requirements for intelligent door opening control to some extent through algorithm optimization and error compensation, it still has significant limitations. Firstly, traditional technologies generally require anti-distortion processing of fisheye images and rely on complex camera extrinsic parameter calibration and error compensation processes, such as checkerboard calibration, or multi-sensor fusion, greatly increasing the difficulty of system debugging and deployment. Secondly, traditional technologies typically require establishing a mapping relationship between image coordinates and actual physical distance through anti-distortion and calibration to ensure the acquisition of the absolute distance to obstacles, resulting in high technical barriers and complex deployment. Thirdly, for intelligent car door applications, traditional technologies struggle to balance low cost, ease of deployment, and scenario adaptability (e.g., requiring additional radar or multi-view vision hardware), failing to meet the actual needs of intelligent obstacle perception and door opening suggestion.

[0036] Therefore, how to overcome the limitations of traditional technical solutions and reduce hardware costs and deployment difficulties has become a key requirement for intelligent vehicle door obstacle ranging technology.

[0037] The technical solution provided in this application acquires environmental images of the vehicle's exterior and identifies the obstacle region corresponding to the obstacle and the door opening region swept across during the door's opening process from closing to opening. Based on the overlap between the door opening region and the obstacle region, the maximum door opening is determined. This eliminates the need for additional distortion correction and camera calibration to establish a mapping between image coordinates and actual physical distances, thus enabling control of the door opening and reducing hardware costs and deployment complexity.

[0038] The following, combined with Figures 1 to 9 The technical solutions of the embodiments of this application will be described in detail below.

[0039] Figure 1 This is a flowchart illustrating a method for determining a vehicle door opening degree according to an embodiment of this application. The door opening degree is the angle or corresponding opening range formed between the door when it is open and when it is closed. Taking a traditional side-opening vehicle door as an example, the door opening degree can be expressed as the angle formed between the door and the vehicle body when the door is open. Figure 2 As shown, the door opening can be represented by an angle α or an angle β.

[0040] See Figure 1 In step 110, an environmental image of the vehicle's exterior is acquired, including obstacles. This environmental image can be acquired using a fisheye camera, which can be a monocular camera. Here, the fisheye camera has an extremely wide field of view, completely covering the door opening area and its surrounding environment, providing ample visual data support for subsequent obstacle area identification. Therefore, in this embodiment, no additional auxiliary hardware such as radar, binocular, or multi-view cameras is required; a single monocular fisheye camera can meet the perception needs of the area around the door, significantly reducing hardware costs and integration complexity. Furthermore, to improve the ability to identify obstacles around the entire vehicle, in some embodiments, the fisheye camera can also be a multi-view camera. Of course, the environmental image in this embodiment can also be acquired by other types of cameras.

[0041] The fisheye camera can be installed on a vehicle door, for example, at the bottom edge of the door along the vehicle's height, i.e., close to the ground. Alternatively, it can be installed anywhere on the side of the vehicle body. This application does not specifically limit the location of the fisheye camera; it can be set according to requirements.

[0042] See also Figure 1In step 120, the obstacle region corresponding to the obstacle in the environmental image is determined. That is, the region where the obstacle is located is determined in the acquired environmental image. For example, this can be achieved by marking the boundary outline and internal coverage area of ​​the obstacle in the environmental image.

[0043] As mentioned earlier, the environmental image is captured by a fisheye camera; therefore, in some embodiments, this environmental image is distorted. Distorted images often cause deviations in the shape and size of obstacles from their actual positions, and blur the boundaries of key areas. Therefore, in order to accurately determine the obstacle region corresponding to an obstacle in the environmental image, it is necessary to first segment the obstacles in the environmental image.

[0044] The following combination Figure 3 The process will be described in detail.

[0045] like Figure 3 As shown, in some embodiments, step 120 may include steps 121 to 123.

[0046] In step 121, obstacles are identified in the environmental image.

[0047] In step 122, the identified obstacles are segmented.

[0048] In step 123, based on the results of the segmentation process, a mask contour corresponding to the obstacle is generated, and the mask contour is used as the obstacle region.

[0049] In other words, firstly, various obstacles (e.g., concrete pillars, pedestrians, vehicles, etc.) included in the environmental image are accurately identified. Then, the identified obstacles are segmented, separating them from the background (e.g., road surface, sky, buildings, etc.). Finally, the shape and boundary range of the obstacle in the environmental image are obtained, and this shape and boundary range can be regarded as the mask contour corresponding to the obstacle. Figure 4 As shown, assuming the columnar object in input 410 is an obstacle in the environment image, the closed area enclosed by the dashed line in output 430 is the mask contour corresponding to the obstacle.

[0050] Therefore, as Figure 4 As shown, in some embodiments, obstacle regions can be generated by an image segmentation network module 420, whose input includes an environmental image and whose output includes a mask contour.

[0051] The image segmentation network module 420 is a visual segmentation model specifically designed for distorted images. This module can process the input environmental image, automatically identify various obstacles in the environmental image, segment the identified obstacles, and output their corresponding mask contours to clarify the regions of obstacles in the environmental image, providing corresponding data support for subsequent region overlap judgment.

[0052] Furthermore, to improve the accuracy of obstacle region recognition, in some embodiments, the mask contour is a mask contour that meets a predetermined accuracy. This predetermined accuracy can be achieved through the cross-entropy loss function used in the image segmentation network module. Using a mask contour that meets the predetermined accuracy as the obstacle region can accurately pinpoint the boundary range of the obstacle in the environmental image, effectively avoiding problems such as obstacle region recognition errors, and providing accurate and reliable data support for subsequent region overlap judgment.

[0053] For example, the image segmentation network module can employ a lightweight deep learning segmentation model, such as a network with 7.6M parameters. This module takes distorted images captured by a fisheye camera as input and outputs mask contours corresponding to obstacles through segmentation processing. Its model structure consists of an encoder, a decoder, and a category classification head. During training, the cross-entropy loss function is used to optimize segmentation accuracy, thereby improving the precision of the segmentation results. Simultaneously, the model employs an obstacle category numbering mechanism to meet the recognition needs of different obstacle categories. That is, different numbering methods are used to distinguish different obstacles for various obstacle categories; for example, a concrete pillar is numbered 1, an ice cream cone is numbered 2, a railing is numbered 3, and so on. Furthermore, the model can be fine-tuned for different obstacle categories to accurately adapt to various scene requirements, improving both the model's scalability and scene generalization ability. In other words, the model can add and recognize new obstacle categories on top of the existing obstacle categories, and it can also recognize existing obstacle categories in different scenarios. For example, obstacle categories can be expanded to include dynamic obstacles, such as pedestrians and bicycles, through model fine-tuning.

[0054] For example, the image segmentation network module can also use other types of lightweight models (e.g., EfficientNet, MobileNet, etc.) to output obstacle mask contours.

[0055] Secondly, in order to improve the accuracy of the image segmentation network model, in some embodiments, the training set of the image segmentation network module may include multiple acquired environmental images, and the multiple environmental images differ in one or more of the following: the category of the obstacle, the location of the obstacle, and the ambient light intensity.

[0056] The core objective of the image segmentation network module is to accurately identify the types of obstacles and the corresponding obstacle regions in environmental images. Therefore, the training set needs to cover a variety of environmental images of different categories, locations, and ambient light intensities. Through continuous training, the accuracy and generalization ability of the model can be improved.

[0057] As an example, this image segmentation network module can collect multiple environmental images of different types of obstacles at different locations and under different ambient light intensities, use these images as a dataset for training, and distinguish different obstacles by numbering. For example, a cement pillar is numbered 1, an ice cream cone is numbered 2, and a railing is numbered 3, etc. Preferably, 200 to 300 environmental images with different ambient light intensities and locations can be collected for each type of obstacle and segmented and labeled. Then, all environmental images are divided into training, validation, and test sets according to a certain ratio. The training set is directly used for model parameter learning, and has the largest sample size, covering environmental images of different obstacle types, locations, and light intensities, so that the model can learn the core rules of the segmentation task from a massive amount of samples. The validation set is used for hyperparameter tuning and model selection during training to avoid overfitting. The test set is used to evaluate the real performance of the final model. No parameters are adjusted during testing; the segmentation accuracy and stability of the model in actual use are judged only by the test set results. The sample size of the validation set can be greater than or equal to the sample size of the test set. Optionally, the ratio of the training set, validation set, and test set can be set to, for example, 8:1:1. Finally, the model is repeatedly trained and fine-tuned based on multiple collected environmental images and the set ratio to ensure accurate segmentation of the mask contours of various obstacles in the environmental images, i.e., images with distortion. According to the training results, the model can achieve a segmentation accuracy of over 99% for each category.

[0058] In summary, to address the challenges of distorted images and diverse obstacle categories, this embodiment trains the image segmentation network model directly on the distorted image. This ensures that the model can identify and segment mask contours of different obstacle categories even when the image is distorted, and uses these mask contours as obstacle regions, providing high-quality data support for subsequent region overlap determination. In other words, all segmentation steps in this embodiment are performed directly on the original distorted image, without any anti-distortion processing or camera calibration, and without relying on extrinsic parameter input or sensor fusion. Therefore, compared to traditional technologies, this embodiment not only reduces hardware costs and deployment complexity, and maintenance costs, but is also more suitable for large-scale, rapid deployment scenarios.

[0059] See also Figure 1In step 130, the maximum opening degree of the car door is determined based on the overlap between the door opening area and the obstacle area in the environmental image. That is, the maximum opening range of the car door is determined by judging whether the door opening area and the obstacle area overlap in the environmental image. The opening area refers to the area swept by the car door during its opening process. Specifically, the area swept by the car door during its opening process is mapped onto the environmental image to obtain the corresponding opening area of ​​the car door in the environmental image. It should be noted that the shape of the opening area in the environmental image will vary depending on the type, shape, size, position, and opening degree of the car door. For example, the opening area of ​​a traditional side-opening car door can be approximated as a fan-shaped area. For a sliding car door, the opening area can be approximated as a rectangular area. This application does not specifically limit the shape of the door opening area; its specific form is determined by the actual shape, size, position, and opening degree of the car door.

[0060] The overlap described in step 130 can refer, for example, to whether the intersection-over-union (IoU) ratio between the door area and the obstacle area is greater than 0. For instance, if the IoU ratio is 0, it is considered that there is no overlap between the door area and the obstacle area; if the IoU ratio is greater than 0, it is considered that there may be overlap between the door area and the obstacle area.

[0061] Alternatively, the overlap described in step 130 could refer to whether the minimum distance between the outline of the door area and the outline of the obstacle area is greater than a preset threshold. For example, if the minimum distance between the outline of the door area and the outline of the obstacle area is greater than the threshold, it is considered that there is no overlap between the door area and the obstacle area; if the minimum distance between the outline of the door area and the outline of the obstacle area is less than or equal to the threshold, it is considered that there may be overlap between the door area and the obstacle area.

[0062] like Figure 5 As shown, Figure 5The shape of the door opening area in the diagram is merely an example. When the door is opened to a certain extent, opening area A overlaps with the obstacle area corresponding to obstacle 1, indicating that obstacle 1 affects the door's opening range. If the maximum opening range of the door is not limited, a collision between the door and obstacle 1 is highly probable. However, regardless of the door's opening range, opening area A does not overlap with the obstacle area corresponding to obstacle 2, indicating that obstacle 2 does not affect the door's opening range. Based on this, the maximum door opening area that does not overlap with obstacle 1 is selected. The door opening corresponding to this area is the maximum door opening range, i.e., the maximum safe opening range of the door.

[0063] In summary, this application addresses the fundamental problem of monocular vision ranging technology's inability to directly measure distances by transforming the spatial relationship between obstacles and car doors into a "region overlap judgment" problem. That is, it does not pursue the absolute distance of obstacles, but rather directly judges whether the door's opening area overlaps with the obstacle area in the environmental image, thereby providing a safety suggestion for the door's opening degree. Compared with traditional technologies, this application does not require establishing a mapping relationship between image coordinates and actual physical distance through distortion correction and camera calibration. Instead, it directly determines the maximum door opening degree in the environmental image based on the overlap between the obstacle area and the door opening area. This reduces hardware costs and deployment difficulty while enhancing scenario applicability. For example, in this application embodiment, the maximum door opening degree can be obtained without adding additional radar or multi-view vision hardware. This fundamentally avoids the technical bottleneck of monocular vision ranging technology.

[0064] In some embodiments, the vehicle door includes multiple opening positions, wherein each opening position is a preset fixed opening angle (or fixed opening range) of the door. By setting the opening position, the door can be stopped at the corresponding angle (or range) during the opening process, thereby meeting the needs of different usage scenarios. Figure 2 As shown, a traditional side-opening car door is used as an example. Assume vehicle 200 includes a right front door 210, a right rear door 220, a left front door 230, and a left rear door 240. Taking the right front door 210 as an example, it is assumed that the right front door 210 has two preset opening positions, such as position 1 and position 2. Position 1 represents the maximum opening degree the door can open in its normal state, which can be represented by α; position 2 represents the opening degree corresponding to a specific position the door opens to, which can be represented by β. Figure 2 It can be seen that the actual door opening area corresponding to gear 1 can be represented as the entire fan-shaped area with the door hinge axis as the rotation axis, the maximum lateral dimension of the door as the radius, and α as the rotation angle, while the actual door opening area corresponding to gear 2 can be represented as the fan-shaped area 211 with the same radius and β as the rotation angle under the same rotation axis.

[0065] As mentioned earlier, multiple opening positions can correspond to multiple actual door opening areas. Therefore, in some embodiments, multiple opening positions can each correspond to multiple door opening areas in the environmental image. Thus, step 130 includes: determining the maximum door opening position based on the overlap between the multiple door opening areas and the obstacle area, wherein the door opening area corresponding to the maximum opening position is the largest door opening area among the multiple door opening areas that does not overlap with the obstacle area. In other words, the safe maximum opening position can be inferred by comparing the overlap between the multiple door opening areas and the obstacle area corresponding to the multiple opening positions in the environmental image.

[0066] like Figure 6 As shown, taking the right-side sliding door as an example, in which... Figure 6 The area on the left represents the opening area of ​​the right front sliding door. Figure 6 The right-hand area represents the opening area of ​​the right rear sliding door of the vehicle. Assume the door has N opening positions: position 1, position 2, position 3, ..., position N. These N opening positions correspond to N opening areas in the environmental image: area 1, area 2, area 3, ..., area N. Position 1 corresponds to area 1, position 2 to area 2, position 3 to area 3, ..., and position N to area N. Assume area 1 has the largest area, followed by area 2, and so on, with area N having the smallest area. Figure 6 and Figure 7 The example shown uses a rectangle representing the door opening area.

[0067] Figure 7 This is a schematic diagram illustrating the maximum opening position provided in an embodiment of this application. Taking the right front door of a sliding vehicle as an example, as follows... Figure 7 As shown, assume that the obstacle region in the environmental image overlaps with region 1 corresponding to gear 1 and region 2 corresponding to gear 2, but does not overlap with the door opening regions corresponding to gears 3 to N. Therefore, region 3 corresponding to gear 3 is the largest door opening region among the N regions that does not overlap with the obstacle region. Therefore, gear 3 is selected as the maximum door opening gear.

[0068] It should be noted that in this embodiment, the area of ​​region 1 can also be set to be the smallest, followed by region 2, and so on, with region N having the largest area. This embodiment does not impose specific limitations on this, and can be set according to requirements.

[0069] As an example, assume the car door has three opening positions: position 1, position 2, and position 3. Position 1 corresponds to area 1, position 2 to area 2, and position 3 to area 3. If an obstacle area overlaps with area 1, then area 2 is the largest area among the three areas that does not overlap with the obstacle area; therefore, position 2 is selected as the maximum door opening position. If the obstacle area overlaps with both area 1 and area 2, then area 3 is the largest area among the three areas that does not overlap with the obstacle area; therefore, position 3 is selected as the maximum door opening position. If the obstacle area does not overlap with any of the three areas, then area 3 is the largest area among the three areas that does not overlap with the obstacle area; therefore, position 3 is selected as the maximum door opening position.

[0070] In order to accurately locate the door opening area corresponding to the door in the environmental image, in some embodiments, firstly, it is necessary to measure the area swept by the door when it opens from closed to different opening positions; then, the area swept by the door when it opens from closed to different opening positions is mapped onto the environmental image respectively, so as to obtain multiple door opening areas in the environmental image corresponding to multiple opening positions.

[0071] In other words, based on the shape and position of the doors of different car models and the preset opening degree information, the system first determines all the areas swept by the door as it opens from closed to different opening degrees; then, it maps the corresponding areas of the door at different opening degrees onto the environmental image. For example, the vehicle can be parked on a level surface, and the doors can be opened step by step according to preset opening degrees. The boundaries of the areas swept by the door as it opens from closed to the current opening degree can be accurately marked on the ground by projection positioning or manual drawing. Subsequently, an environmental image of the marked area on the ground can be captured using a fisheye camera pre-configured on the door, thus obtaining the door opening area in the environmental image at different opening degrees.

[0072] Furthermore, in some embodiments, after determining the maximum opening degree of the door, the maximum opening degree information is output to the door drive module. The door drive module controls the door opening degree based on this information. Specifically, the maximum opening degree of the door is inferred through region overlap judgment and then sent to the door drive module. The door drive module controls the door to open to the corresponding degree based on this information. Simultaneously, to optimize the user experience, in some embodiments, after determining the maximum opening degree of the door, it can be presented to the user as an opening suggestion, for example, through voice prompts, in-vehicle screen display, or other interactive methods.

[0073] As can be seen, in this embodiment, based on the door opening areas corresponding to multiple door opening positions in the environmental image, and combined with the obstacle areas corresponding to obstacles in the environmental image, region overlap judgment is performed to deduce the safe maximum door opening area. Then, the maximum opening position corresponding to the maximum door opening area is selected from the preset opening positions, thereby realizing intelligent control of the door. The entire process requires no additional hardware assistance or visual extrinsic parameter calibration. Therefore, compared with traditional technologies, the solution provided in this embodiment is simple and flexible to deploy and can adapt to intelligent door application scenarios.

[0074] In summary, the technical solution of this application embodiment is highly adaptable to the application scenarios of smart car doors, especially for the scenario of car door opening safety control. It proposes a method for determining the car door opening degree, which achieves obstacle recognition and car door opening degree inference through an image segmentation network module and region overlap judgment. The method includes: firstly, using an image segmentation network module to segment the distorted environmental image to generate a mask contour of obstacles, using the mask contour of the obstacles as the obstacle region; then, determining the maximum opening degree of the car door by judging the overlap between the door opening region and the obstacle region corresponding to multiple opening degrees of the car door in the environmental image. Since all segmentation steps are completed directly on the original distorted image, and the region overlap judgment is also completed directly on the original distorted image, this method is highly effective. Therefore, in this embodiment, no complex image distortion correction and camera calibration processes are required throughout the entire process. It also does not rely on external parameters or sensor fusion. The spatial relationship between the car door and surrounding obstacles can be inferred solely by a single-eye fisheye camera as the door gradually opens from a closed state to any opening position. This provides technical support for intelligent suggestions and automatic control of the door opening, and more effectively avoids the risk of collision with obstacles when the door opens. Compared with traditional technologies, it reduces hardware costs and deployment difficulty, improves the safety and intelligence level of the intelligent door system, and brings users a safer and more convenient driving experience.

[0075] It should be noted that the technical solutions provided in this application can be widely applied to side door safety assistance systems for various intelligent vehicles, especially suitable for automatic opening and closing scenarios of electric vehicle doors. They can also be applied to fields such as autonomous vehicles, intelligent parking assistance, and urban public transportation safety.

[0076] The embodiments of this application are described in more detail below with specific examples. It should be noted that... Figure 8 and Figure 9 The examples provided are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of this application to the specific numerical values ​​or specific scenarios illustrated. Figure 8 and Figure 9The examples are obviously subject to various equivalent modifications or changes, and such modifications or changes also fall within the scope of the embodiments of this application.

[0077] Figure 8 This application provides a specific implementation method for determining the opening degree of a vehicle door. The method mainly includes a data acquisition module, a processing module, and a door drive module. The data acquisition module may be, for example, an onboard fisheye camera, and the processing module may include, for example, an image segmentation network module and a post-processing module. Furthermore, this implementation method also includes... The modules are connected via a data bus to form a closed-loop system for data acquisition, processing, and decision-making.

[0078] See Figure 8 In step 810, image acquisition and input occur. This involves acquiring real-time images of the side environment of the vehicle (or fisheye images) using an onboard fisheye camera, and inputting the acquired raw environmental images into the image segmentation network module. Image acquisition can, for example, be performed by the onboard fisheye camera at the vehicle's terminal side.

[0079] In step 820, obstacle recognition and segmentation are performed. The input environmental image is processed using an image segmentation network module to identify and segment obstacles in the image, generating obstacle mask contours, which are then input into a post-processing module. The post-processing module analyzes the spatial relationship between the obstacle mask contours and the door opening areas in the environmental image, specifically determining whether obstacle areas overlap with multiple door opening areas. The image segmentation network module can, for example, be executed locally by the vehicle controller.

[0080] In step 830, the door opening area corresponding to each opening level is obtained. This involves obtaining the door opening area in the environmental image corresponding to different opening levels and inputting it into the post-processing module. The position and shape of the door opening area in the environmental image can be preset for different opening levels.

[0081] In step 840, the door opening area is determined. Using a post-processing module, a spatial comparison is performed between the obstacle mask outline and the door opening area. If the obstacle mask overlaps with this area, it is determined that the obstacle will affect the door opening angle. This post-processing module can, for example, be executed locally by the vehicle controller.

[0082] In step 850, the door opening level is inferred and suggested. Using the post-processing module, based on the overlap between the obstacle mask outline and the door opening area, and combined with the corresponding area of ​​the preset opening level in the environmental image, a safe maximum opening level is inferred to achieve door safety control. The door opening level suggestion can be output from the controller to the door drive module to achieve automatic or assisted door opening control.

[0083] More intuitively, Figure 9It shows the basis Figure 8 A specific example of the method shown.

[0084] like Figure 9 As shown, in step 910, the vehicle-mounted fisheye camera acquires an image of the environment on the side of the vehicle body. This image is a distorted image, and the distorted image is input into the image segmentation network module in step 920.

[0085] In step 920, the image segmentation network module processes the distorted image and outputs an image segmentation result, which includes the obstacle region.

[0086] In step 930, the door opening area corresponding to the preset door opening position is obtained and input into the post-processing module.

[0087] In step 940, the obstacle area and the door opening area are compared by the post-processing module.

[0088] In step 950, based on the comparison results in step 940, a suggested door opening degree is output.

[0089] In summary, actual testing has verified that the embodiments of this application can achieve obstacle recognition and opening degree inference using only a monocular fisheye camera without radar assistance. The measured error of the maximum opening degree is less than 1 degree, and the accuracy rate is over 98%. This not only significantly reduces the hardware cost of the system, but also further improves the safety and intelligence level of the intelligent door system.

[0090] like Figure 10 As shown, this application also provides a device 1000 for determining the opening degree of a vehicle door, including a data acquisition module 1010 and a processing module 1020. The data acquisition module 1010 is used to acquire an environmental image of the exterior of the vehicle, wherein the environmental image includes obstacles; the processing module 1020 is used to determine the obstacle region corresponding to the obstacle in the environmental image; and, based on the overlap between the door opening region corresponding to the vehicle door in the environmental image and the obstacle region, determine the maximum opening degree of the vehicle door, wherein the opening region is the area swept by the vehicle door during the process of closing to opening.

[0091] In some implementations, the overlap condition includes: whether the intersection-union ratio between the door opening area and the obstacle area is greater than 0; and / or whether the minimum distance between the outline of the door opening area and the outline of the obstacle area is greater than a preset threshold.

[0092] In some implementations, determining the obstacle region corresponding to the obstacle in the environmental image includes: identifying the obstacle in the environmental image; segmenting the identified obstacle; generating a mask contour corresponding to the obstacle based on the segmentation result, and using the mask contour as the obstacle region.

[0093] In some implementations, the obstacle region is generated by an image segmentation network module, the input of which includes the environment image, and the output of which includes the mask contour.

[0094] In some implementations, the environmental image is a distorted image, and the mask contour is a mask contour that meets a predetermined accuracy, which is achieved by the cross-entropy loss function used in the image segmentation network module.

[0095] In some implementations, the training set of the image segmentation network module includes multiple acquired environmental images, wherein one or more of the following contents differ in the multiple environmental images: the category to which the obstacle belongs, the location of the obstacle, and the ambient light intensity.

[0096] In some implementations, the vehicle door includes multiple opening positions, each corresponding to a multiple opening area in the environmental image. The step of determining the maximum opening of the vehicle door based on the overlap between the opening area corresponding to the vehicle door in the environmental image and the obstacle area includes: determining the maximum opening position of the vehicle door based on the overlap between the multiple opening areas and the obstacle area, wherein the opening area corresponding to the maximum opening position is the largest opening area among the multiple opening areas that does not overlap with the obstacle area.

[0097] In some implementations, the method further includes: measuring the area swept by the vehicle door from closed to different opening positions; mapping the area swept by the vehicle door from closed to different opening positions onto the environmental image to obtain the multiple door opening areas in the environmental image corresponding to the multiple opening positions.

[0098] In some implementations, the environmental image is captured by a fisheye camera, which can be a monocular or multi-view camera, and the fisheye camera is positioned on the vehicle door.

[0099] In some implementations, the method further includes: outputting information about the maximum opening degree to a door drive module, wherein the door drive module controls the opening degree of the door based on the information about the maximum opening degree.

[0100] See Figure 11 This application also provides a vehicle 1100, including: a transceiver 1110, a memory 1120 and a processor 1130. The memory 1120 is used to store a program, and the processor 1130 is used to call the program in the memory 1120 and control the transceiver 1110 to receive or send signals so that the vehicle 1100 performs the methods including any of the embodiments described above.

[0101] This application also provides a computer-readable storage medium storing program code that can be invoked by a processor to execute methods including any of the embodiments described above.

[0102] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0103] It should be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0104] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining the opening degree of a vehicle door, characterized in that, The method includes: Acquire environmental images of the exterior of the vehicle, including obstacles; Determine the obstacle region corresponding to the obstacle in the environmental image; The maximum opening degree of the car door is determined based on the overlap between the door opening area and the obstacle area in the environmental image. The door opening area is the area swept by the car door during the process of closing and opening.

2. The method according to claim 1, characterized in that, The overlapping situations include: Whether the intersection-union ratio between the door opening area and the obstacle area is greater than 0; and / or, Whether the minimum distance between the outline of the door opening area and the outline of the obstacle area is greater than a preset threshold.

3. The method according to claim 1 or 2, characterized in that, Determining the obstacle region corresponding to the obstacle in the environmental image includes: Identify the obstacle in the environmental image; The identified obstacles are segmented. Based on the segmentation results, a mask contour corresponding to the obstacle is generated, and the mask contour is used as the obstacle region.

4. The method according to claim 3, characterized in that, The obstacle region is generated by an image segmentation network module, the input of which includes the environmental image, and the output of which includes the mask contour.

5. The method according to claim 4, characterized in that, The environmental image is a distorted image, and the mask contour is a mask contour that meets a predetermined accuracy, which is achieved through the cross-entropy loss function used in the image segmentation network module; and / or, The training set of the image segmentation network module includes multiple acquired environmental images, and the multiple environmental images differ in one or more of the following: the category to which the obstacle belongs, the location of the obstacle, and the ambient light intensity.

6. The method according to claim 1 or 2, characterized in that, The vehicle door includes multiple opening positions, each corresponding to a different door opening area in the environmental image. The step of determining the maximum opening degree of the vehicle door based on the overlap between the door opening area and the obstacle area in the environmental image includes: The maximum opening degree of the vehicle door is determined based on the overlap between the multiple door opening areas and the obstacle area. The door opening area corresponding to the maximum opening degree is the door opening area with the largest area that does not overlap with the obstacle area among the multiple door opening areas.

7. The method according to claim 6, characterized in that, The method further includes: Measure the area swept by the vehicle door as it opens from closed to different opening positions; The areas swept by the car door as it opens from closed to different opening positions are mapped onto the environmental image to obtain the multiple door opening areas in the environmental image corresponding to the multiple opening positions.

8. The method according to claim 1 or 2, characterized in that, The environmental images are captured by a fisheye camera, which can be a monocular or multi-view camera, and the fisheye camera is installed on the vehicle door.

9. The method according to claim 1 or 2, characterized in that, The method further includes: The maximum opening position information is output to the door drive module, which controls the opening of the door based on the maximum opening position information.

10. A device for determining the opening degree of a vehicle door, characterized in that, The device includes: The acquisition module is used to acquire environmental images of the outside of the vehicle, including obstacles; Processing module, used for: Determine the obstacle region corresponding to the obstacle in the environmental image; and, The maximum opening degree of the car door is determined based on the overlap between the door opening area and the obstacle area in the environmental image. The door opening area is the area swept by the car door during the process of closing and opening.

11. A vehicle, characterized in that, include: A transceiver, a memory, and a processor, wherein the memory is used to store a program, the processor is used to invoke the program in the memory, and to control the transceiver to receive or transmit signals to cause the vehicle to perform the method according to any one of claims 1 to 9.

Images