Method and system for assisting in opening or closing a door

Abstract

To provide a method.SOLUTION: There is provided a computer-implemented method for assisting opening or closing of a door. The computer-implemented method comprises: - obtaining at least one image showing at least one region of interest, the at least one region of interest comprising a door operating area or an area adjacent to the door operating area; - detecting, in the at least one image, whether at least one object is present in the region of interest; - and determining, on the basis of the detection, at least one interference indicator for opening or closing the door.SELECTED DRAWING: Figure 2

Description

Background Art

[0001] 1. Field of the Invention The present disclosure relates to the field of automation, and more particularly, to methods and systems for assisting in opening or closing a door. The methods and systems may find applications in all types of industries, including the automotive industry.

[0002] 2. Description of Related Art In recent years, the demand for automation has been increasing across all fields of technology. In particular, the field of autonomous vehicles has experienced significant development in partially or fully automating the driving tasks normally performed by a driver. However, in vehicles, more specifically in shared vehicles such as buses, shuttles, trains, etc., the driver (or in some cases, the conductor or operator) typically performs tasks other than pure driving tasks, such as managing the opening and closing of doors, determining whether the vehicle can depart, assisting disabled persons, handling emergencies, and the like.

[0003] Therefore, there is a need for further automation, particularly with regard to assisting in the opening and closing of doors.

[0004] The following reference discloses a method related to the field of background management in images. [Reference 1] Real-Time High-Resolution Background Matting by Shanchuan Lin et al., arXiv:2012.07810v1

Summary of the Invention

[0005] In this regard, the present disclosure is a computer-implemented method for assisting in opening or closing a door, comprising: - obtaining at least one image showing at least one region of interest, wherein the at least one region of interest comprises a door operation area or an area adjacent to the door operation area; and - detecting in the at least one image whether at least one object is present in the region of interest. - Based on the detection, determine at least one interference indicator for opening or closing the door, This relates to computer implementation methods, including

[0006] The steps of the method may be performed by one or more processors. The method will be referred to as the support method below.

[0007] Acquiring at least one image may include, for example, acquiring an image through an image acquisition module such as a camera, or retrieving an already acquired image from a database, such as a local server, a remote server, or the same. Hereafter, unless otherwise specified, “image” refers to at least one image. More broadly, when used herein, for the sake of brevity, unless the context specifically indicates otherwise, “a,” “an,” and “the” are intended to refer to “at least one” or “each,” and similarly include plural forms. Conversely, the general use of plural also includes singular forms.

[0008] In some embodiments, at least one image comprises multiple consecutive frames from a video clip. Alternatively, at least one image may comprise multiple non-contiguous frames from a video clip selected, for example, at a given sampling frequency (e.g., every three frames of the video clip). Further alternatively, at least one image may comprise one or more static images, such as photographs. Using frames from a video clip allows for the verification of detection information over time, thereby providing robustness to the method.

[0009] Each acquired image may show one or more regions of interest. A region of interest (ROI) is an area of ​​the door environment where elements of interest can be observed to determine interference indicators. A region of interest comprises the door operating area or an area adjacent to the door operating area. In this respect, the door operating area is the area that intersects with the door during its opening or closing. The door operating area does not necessarily completely cover the door's travel distance.

[0010] In this disclosure, "object" should be understood as a general term meaning any obstruction to a door, including a person, animal, plant, or article.

[0011] An interference indicator describes how much the door environment, particularly objects within the area of ​​interest, can interfere with the opening and / or closing of the door (hereinafter abbreviated as opening / closing).

[0012] Therefore, the above method makes it possible to determine, based solely on an image, the extent to which objects around the door interfere with possible door movements (e.g., opening / closing), and to output interference indicators based on what additional actions may occur, such as triggers for opening / closing the door. Thus, the proposed method provides reliable assistance for opening / closing operations based on simple inputs. Consequently, the task of opening / closing a door can be handled automatically.

[0013] Optionally, the method is performed in real time to assist in opening / closing the door based on the current conditions regarding the environment surrounding the door.

[0014] Optionally, the decision involves taking multiple detections as input and applying a logic-based algorithm. The logic-based algorithm is, for example, a rule-based or probability-based algorithm that applies the rules or probabilities to multiple inputs, i.e., multiple detections, to determine an output. Multiple detections may include the detection of each of multiple objects in at least one image, or detections across multiple images (e.g., one detection per image). By combining a detection method and a logic-based algorithm based on multiple detections, the method is not only more reliable because it is based on multiple detections, but also simpler because the detections are combined by a logic-based algorithm. This avoids the need for complex models to handle a wide variety of situations. In contrast, combining detections through a logic-based algorithm allows for the leverage of the advantages of simpler detection and the certainty of logic-based reasoning, which brings human insight to the method. This architecture also brings self-adaptability to the assistive method.

[0015] Optionally, at least one image includes a first image showing a first side of the door sill and a second image showing a second side of the door sill, which is opposite to the first side. The door sill is determined by the position of the door when it is closed. The sill may be embodied by a frame, a sill, or any other means, depending on the type of door. By providing these, both sides of the door sill are taken into consideration in determining the interference indicator.

[0016] Alternatively or additionally, a single image may show both sides of the door sill.

[0017] Optionally, an image can be an RGB (red / green / blue) image, or more broadly, a single-channel or multi-channel image. An RGB image of a scene comprises pixels having red, green, and blue values ​​to form colors (including black and white or grayscale) that represent the colors of the scene. A single-channel image is an image in which each pixel has a color or intensity represented by a single value, while a multi-channel image is an image in which each pixel has a color or intensity represented by multiple values. Grayscale is an example of a single-channel scale, while RGB is an example of a multi-channel scale.

[0018] Optionally, images are acquired using a fisheye camera. Fisheye cameras themselves are known in the art and enable the acquisition of a large field of view.

[0019] Optionally, at least one interference indicator comprises a first interference indicator determined based on a first image and a second interference indicator determined based on a second image, wherein the method further includes selecting one of the first and second interference indicators corresponding to the highest interference level. An interference level is represented as high when the detected object interferes with or obstructs the opening / closing of the door, while an interference level is represented as low when the detected object, if any, interacts little or no with the door during the opening / closing movement. For example, the first and second interference indicators are represented on the same scale, with a larger value for a higher interference level, and the method may select the highest value of the first and second interference indicators.

[0020] With the first and second interference indicators, the method may determine the interference indicator based on the highest detection risk of interference between the door and its environment, taking into account multiple object configurations.

[0021] In the embodiment, the final decision to enable or trigger the opening / closing of the door may be based on the selected interference indicator.

[0022] Optionally, the first image shows at least partially the second side of the door sill, or the second image shows at least partially the first side of the door sill. In this embodiment, at least one of the images shows both sides of the door sill. This improves the reliability of the method.

[0023] Optionally, detection may further include detecting whether a person is present in the domain of interest. That is, detecting whether a person is present in the domain of interest is performed separately from detecting whether an object is present in the domain of interest. As a result, if a person is present in the domain of interest, the person is expected to be detected by both object detection and person detection. Person detection makes it possible to identify a person as a person with certain possible implications or attention, while object detection is more general and does not require classifying objects. In addition, having both object detection and person detection introduces redundancy in the method, but this also leads to increased robustness.

[0024] Optionally, humans are detected by skeletal detection. Skeletal detection is a simple and efficient method for detecting humans.

[0025] Optionally, detection includes extracting foreground elements in at least one image relative to a reference background. This may be applied to object detection and / or person detection. The reference background comprises elements of the door environment (e.g., the rest of a vehicle, road, trees, buildings, etc.) that may or may not be static relative to the door or door sill, and serves as a reference for detecting foreground elements. Foreground elements are elements added on top of the background, e.g., occupants, other vehicles, items on the ground, etc., that may interfere with the opening / closing of the door.

[0026] By extracting foreground elements with respect to a reference background, the method can adapt to different backgrounds (e.g., different locations, different lighting, etc.) and consider only relevant elements of the image. Thus, image-based assistance is more reliable.

[0027] Optionally, the extraction is performed by a machine learning model. The machine learning model can comprise one or more mathematical representations configured to process an image to output information representing the foreground elements. The weights of the machine learning model can be iteratively updated by a training method. The machine learning model can comprise an artificial neural network.

[0028] Optionally, the extraction includes calculating a mask of the foreground elements in at least one image. The mask of the foreground elements indicates the foreground elements while masking other pixels of the image. The mask calculation itself is known in the field of image processing and is easy to use. As a result, the proposed method is efficient and robust.

[0029] Optionally, the method further includes updating the reference background based on at least one image if no person is detected in the at least one image. If no person is detected in the image, the image is assumed to represent the actual background excluding possible objects that are easier to detect and less likely to move by themselves. Thus, the absence of a person is utilized to update the background. By updating the background, any changes in the environment can be taken into account, and ultimately, the results of the method can be improved.

[0030] Optionally, the method includes initializing the reference background. The initialization can be performed if the reference background has not been previously set or if the reference background has not been set for a specific period. The initialization can be performed automatically.

[0031] The present disclosure further relates to a system for assisting in opening or closing a door, - An acquisition module that acquires at least one image showing at least one region of interest, wherein the at least one region of interest includes a door operating area or an area adjacent to the door operating area, - A detector that detects in at least one image whether at least one object is present in the region of interest, - A decision unit that determines at least one interference indicator to open or close the door based on detection, This applies to systems that include the following features.

[0032] The system referred to below as the support system may be configured to perform the support methods described above and may have some or all of the characteristics described above. The support system may have a computer hardware structure.

[0033] This disclosure further covers a vehicle comprising at least one door and the aforementioned system for assisting in opening or closing the door. The vehicle may be a passenger car, a bus, a shuttle, or other automobile, or a train, an airplane, or other vehicle.

[0034] Optionally, the vehicle is an autonomous vehicle. The support system can enable the non-driving functions of the autonomous vehicle.

[0035] This disclosure further relates to a set of computer programs which, when the programs are executed by at least one computer, include instructions that perform the steps of the assistance method described above. The programs of this set may take the form of source code, object code, or intermediate code between source code and object code, for example, in a partially compiled form or any other desirable form, using any programming language.

[0036] This disclosure further relates to a recording medium readable by at least one computer, on which at least one computer program is recorded, which includes instructions for performing the steps of the above-described support method. The recording medium may be any entity or device capable of storing a program. For example, the medium may include storage means such as ROM, e.g., CD-ROM or micro-electronic circuit ROM, or magnetic storage means, e.g., diskette (floppy disk) or hard disk.

[0037] Alternatively, the recording medium may be an integrated circuit with a program embedded in it, and the circuit is configured to perform or be used to perform the method. [Brief explanation of the drawing]

[0038] The features, advantages, and technical and industrial significance of preferred embodiments of the present invention are described below with reference to the accompanying drawings, in which similar reference numerals indicate similar elements. [Figure 1] Figure 1 is a flowchart showing the calibration portion of the support method according to the embodiment. [Figure 2] Figure 2 is a flowchart showing the decision-making portion of the support method according to the embodiment. [Figure 3] Figure 3 is a diagram of a vehicle according to this embodiment. [Modes for carrying out the invention]

[0039] A method for assisting the opening or closing of a door according to this embodiment is described with reference to Figures 1 and 2. In this embodiment, the door is a vehicle door. However, the following description may be applied to other types of doors.

[0040] As shown in Figure 1, the assistance method includes acquiring at least one image representing a region of interest, the at least one region of interest comprising a door operating area or an area adjacent to the door operating area. Specifically, Figure 1 shows two such images, namely an inner image 10 and an outer image 11. The inner image 10 (first image) may represent a first side (inside) of the door sill, for example, the vehicle cabin, and the outer image 11 (second image) may represent a second side (outside) of the door sill, which is opposite to the first side.

[0041] Specifically, the inner image 10 shows the area of ​​interest inside the vehicle, for example, the area of ​​the vehicle ground on which a person walks before exiting through the door. In another example, the inner image 10 shows the area where the door moves, for example, in the case of a bus, the door may open inwards towards the vehicle. Such areas are called the inner ROI (Region of Interest).

[0042] Conversely, the outer image 11 shows the region of interest outside the vehicle, for example, the area of ​​the outside ground that a person walks on before entering the vehicle through the door. In another example, the outer image 11 shows the area where the door moves, for example, in the case of a vehicle such as a passenger car, the door generally opens outwards from the vehicle. Such an area is called the outer ROI (Region of Interest).

[0043] The inner image 10 and / or outer image 11 (hereinafter referred to as "Images 10, 11") are described herein as single images, but each may comprise multiple images, e.g., video, rather than being a static image (e.g., a photograph). In addition, this disclosure is not limited to a single inner image and a single outer image, and more or fewer images may be envisioned. Furthermore, the support method may use only one camera rather than having images from two or more viewpoints (here, one camera inside the vehicle and one camera outside the vehicle). The camera preferably shows at least the inside of the vehicle, or both the inside and outside through a window or open door.

[0044] Images 10 and 11 can be acquired by a camera, for example, a fisheye camera. Although a fisheye camera may provide distorted images, images 10 and 11 may not require any modification before being processed by the assistive method.

[0045] As shown above, at least one of the inner and outer images may show both sides of the door sill. In this example, it is assumed that the inner image (first image) 10 shows at least partially the second side (outside) of the door sill. In such a situation, the region of interest may overlap with the door sill. Hereafter, the region of interest that is on the outside of the vehicle but visible from the inside will be referred to as the inner-outer ROI.

[0046] In step 12, the assistance method acquires the inner image 10. In some cases, part or all of the video may be acquired in step 12. In step 14, the assistance method determines whether the ROI configuration has already been calibrated. If the ROI configuration has already been calibrated (YES), the assistance method proceeds to step 32 to load the inner ROI based on the inner image 10 and step 34 to load the inner-outer ROI. Steps 32 and 34 may include cropping the inner image 10. The ROI configuration may be loaded from memory.

[0047] However, if the ROI configuration is not calibrated (NO in step 14), the assisting method enters the ROI calibration process. In step 16, the assisting method reads the inner image 10 for calibration. If, for example, the inner image 10 comprises multiple images (video or other sequence), step 16 may include reading the frames. The assisting method then proceeds to step 18.

[0048] Step 18 includes optional trimming to limit the total computational requirements. For example, step 18 includes trimming image 10 to an available region of interest. In this example, there are two regions of interest in the inner image 10, namely the inner ROI and the outer ROI, and therefore two trimmed versions of image 10. Trimming image 10 to preserve the regions of interest can be done manually or automatically, for example, based on the detection of feature points in the image. For example, the trimmed region of image 10 may have a simple shape, such as a polygon, circle, or ellipse.

[0049] In some cases, after trimming, a configuration for each ROI may be created. Step 20 includes creating an inner ROI configuration, while step 22 includes creating an inner-outer ROI configuration. The ROI configuration may have the position, size, and / or shape of each ROI in the (potentially trimmed) inner image 10. In contrast to the trimmed image, the shape of the ROI may be any possible shape, not just a simple shape. These steps 20 and 22 result in an inner ROI 24 and an inner-outer ROI 26, respectively.

[0050] The internal ROI 24 and internal-external ROI 26 can be saved in their respective saving steps 28 and 30. Once the ROI configuration is saved, the ROI calibration process proceeds to the loading steps 32 and 34 described earlier.

[0051] The outer image 11 may be processed in the same way, and as a result, the complete calibration will not be described again here. For example, steps 13, 15, 17, 19, 21, 29, and 33 for acquiring the outer ROI 25 may correspond to the aforementioned steps 12, 14, 16, 18, 20, 28, and 32 for acquiring the inner ROI 24. Of course, the steps corresponding to the inner-outer ROI may not be performed for the outer image 11, as the outer image 11 does not show the inside of the vehicle. However, more schematically, multiple regions of interest may be loaded in each acquired image and calibrated if necessary.

[0052] The door itself may be recognizable in at least one of the ROIs.

[0053] To prepare for the object detection described later, the assistance method may determine in step 36 whether the reference background has been calibrated, for example, based on the inner image 10. Step 36 may follow step 12 described above. If the reference background has already been calibrated (YES), the assistance method proceeds to step 48, which loads the reference background. The reference background may be loaded from memory.

[0054] However, if the baseline background is not calibrated (NO in step 36), the support method enters the baseline background initialization process. First, in step 38, the support method reads the inner image 10 for calibration. If, for example, the inner image 10 consists of multiple images (video or other sequence), step 38 may include reading the frames. At this stage, it may be checked whether the vehicle is empty, for example, whether a person is present inside the vehicle. In fact, if a person was present in the vehicle, the inner image 10 would not represent only background data.

[0055] In step 12, a video or multiple images may be loaded, depending on the circumstances. Therefore, if a person is detected in one of the frames, step 38 may read another frame from the same video so that the reference background can be properly initialized. If no person is detected in the current frame, the assistance method proceeds to step 40. At least one of the frame analysis techniques, such as motion estimation and object detection, may be employed to identify and extract the most representative empty cabin frame from the multiple available frames. Several frames may be taken into consideration to create a reference background model, as described below.

[0056] Step 40 includes cropping a portion of the inner image 10 that represents a fixed background. Portions that do not represent a fixed background may be excluded to reduce computational requirements. In this example, the reference background may include any of the following: the vehicle ground, the vehicle seats, the vehicle side walls, etc., and conversely, the view recognizable through the windows may change, so the cropping area may exclude the vehicle windows. Therefore, step 40 includes cropping the reference background.

[0057] After trimming, a reference background model can be created that corresponds to the trimmed reference background. Step 42 includes creating a reference background model. Any background modeling technique may be used, such as the background modeling technique in [Reference 1]. Step 42 results in a reference background model 44. For example, creating a reference background model from multiple frames may include at least one of the following to capture a more accurate representation of the cabin background: frame alignment, correction of any misalignment or parallax, and averaging of pixel values ​​to reduce noise.

[0058] The reference background model 44 can be saved in the save step 46. Once the reference background model 44 is saved, the reference background initialization process proceeds to the load step 48, which has already been described.

[0059] While specific examples are provided, the ROI calibration process and the baseline initialization process may be modified from these examples. For example, some steps may be omitted (e.g., cut out), other steps may be added, or the described functions may be performed in a different manner.

[0060] Figure 2 shows the steps of the support method following the steps described with reference to Figure 1. That is, as indicated by the dashed arrows in Figures 1 and 2, step 50 in Figure 2 follows steps 32, 34, and 48, while step 51 follows step 31.

[0061] Figure 2 shows an example of how the support method detects whether at least one object is present in the region of interest in the acquired image.

[0062] Specifically, in step 50, the inner image 10 is acquired. In the case of video, step 50 may include reading a specific frame of the video. If the assisting method is applied in real time, step 50 may read the most recent frame of the inner image 10. The method then proceeds to step 52.

[0063] In step 52, the content of the inner ROI in inner image 10 is extracted based on inner image 10 (or frame) acquired in step 50 and the inner ROI configuration loaded in step 32. The assisting method then proceeds to step 54.

[0064] Step 54 involves extracting foreground elements from at least one image (in this case, the current frame) against the reference background loaded in Step 48. This extraction may be performed by a machine learning model, for example, according to [Reference 1]. More broadly, the extraction may involve calculating a mask of foreground elements in the image.

[0065] Next, the support method proceeds to step 56, where it is determined whether an object is present in the inner ROI based on the extracted foreground elements. For example, step 56 may include applying the mask from step 54 to the inner image 10 (or a given frame thereof), taking into account the inner ROI configuration from step 32, in order to determine whether a foreground element (object) is present in the inner ROI.

[0066] Furthermore, the assistance method further includes detecting whether a person is present in the region of interest, in this case, the region of interest outside the vehicle. With respect to the internal image, person detection may be performed from the image acquired in step 50. Specifically, from step 50, the assistance method proceeds to step 58, possibly in parallel with step 52.

[0067] In step 58, the content of the internal-external ROI in the internal image 10 is extracted based on the internal image 10 (or frame) acquired in step 50 and the internal-external ROI configuration loaded in step 34. The assisting method then proceeds to step 60.

[0068] In step 60, person detection itself is performed to determine whether a person is present in the internal ROI and the internal-external ROI. For example, a person may be detected by skeletal detection, e.g., 2D skeletal detection, which itself is known in the field of computer vision.

[0069] The support method then proceeds to step 56. Taking the above into consideration, step 56 collects the results of general object detection and person detection to determine whether an object or person is present in the internal ROI.

[0070] From step 60 described above, the support method also proceeds to step 62, where it is determined whether a person is present in the internal-external ROI based on the person detection in step 60 and the internal-external ROI configuration in step 34. This determination may be expressed as a first interference indicator, for example, a numerical value representing the level of interference between the detected person and the internal-external ROI.

[0071] Optionally, step 60 also leads to step 64, where it is determined whether a person is present in the vehicle (e.g., in the entire inner image, or at least in the inner ROI). If the vehicle is indeed empty (YES), the method proceeds to step 66, which updates the baseline background based on the image. For example, as an update, the assisting method may use steps 40-46 described with reference to Figure 1, using an image obtained from step 50 rather than an image obtained from step 38. However, other methods are possible. For example, the update may utilize a change detection algorithm and / or scene analysis to identify and introduce a new empty cabin frame to the existing background model.

[0072] By updating the baseline background when it is determined that the vehicle is empty (e.g., no one inside), it becomes possible to account for changes in the vehicle's environment, such as variations in light (e.g., day / night, sunny / cloudy weather) or changes in the vehicle's accessories. This allows for the maintenance of a highly accurate method when environmental conditions change.

[0073] The outer image 11 may be processed in the same way as the inner image 10, and as a result, this processing will not be described again here. For example, steps 51, 59, 61, and 63 for detecting objects, in particular people, in the outer ROI may correspond to the aforementioned steps 50, 58, 60, and 62 for detecting people in the inner-outer ROI. In step 63, the determination of whether a person is present in the outer ROI may be represented by a second interference indicator.

[0074] The assistance method includes determining, based on detection, at least one interference indicator for opening or closing the door. That is, the decisions in steps 56, 62, and 63 are taken into consideration to derive whether the door can be opened or closed.

[0075] In this embodiment, multiple detections from steps 56, 62, and 63 for the inner ROI, inner-outer ROI, and outer ROI are provided as input to a logic-based algorithm. The logic-based algorithm may comprise one or more rules that combine the results from detection steps 56, 62, and 63.

[0076] In this embodiment, for example, in step 64 following step 56, it is determined whether an object is detected in the inner ROI. Furthermore, in step 66, it is determined whether a person is detected in the inner ROI. In this example, if either a person or an object is detected in the inner ROI in step 64 or 66 (YES), the assistance method further proceeds to step 68, which displays an interference indicator indicating that the inner ROI is occupied. Thus, it is considered that something may interfere with the movement of the door and therefore the opening or closing of the door may be prevented.

[0077] Conversely, if neither an object nor a person is detected in step 64, the assistance method proceeds to step 70 and displays an interference indicator indicating that the inner ROI is open. Thus, the door can be opened or closed without interfering with any person or object inside the vehicle.

[0078] With respect to the exterior of the vehicle, the first indicator in step 62 and the second indicator in step 63 are combined in step 72 to determine whether a person is detected in the interior-exterior ROI or exterior ROI. If a person is detected (YES), the assistance method further proceeds to step 74, where it emits an interference indicator indicating that the ROI outside the vehicle is occupied. Thus, it is considered that something may interfere with the movement of the door and therefore the opening or closing of the door may be prevented.

[0079] Conversely, if no person is detected in the internal-external ROI or external ROI (NO), the assistance method proceeds to step 76, emitting an interference indicator that the ROI outside the vehicle is open. Thus, the door can be opened or closed without interfering with any person or object outside the vehicle.

[0080] This illustrates a method in which the support method can select one of two interference indicators, a first interference indicator and a second interference indicator, corresponding to the highest interference level, for example, using a Boolean OR operation.

[0081] Conflicting decisions in detection can be handled in different ways. In one example, as mentioned above, the highest level of interference may be considered to avoid any risk regarding door movement. In another example, if the outer ROI does not indicate human detection, but the inner-outer ROI does, it may be decided to ignore the current frame and read another frame for confirmation.

[0082] The above assistance method describes that object detection is performed inside the vehicle, while human detection is performed both inside and outside the vehicle. While other embodiments are conceivable, this embodiment is advantageous to the extent that the environment inside the vehicle generally does not change much, thereby facilitating object detection, while the environment outside the vehicle generally changes significantly, potentially making it uncertain whether object detection is reliable. In contrast, human detection functions adequately whether inside or outside the vehicle.

[0083] The interference indicators in one step 68 or 70 and the other 74 or 76 may be combined again to determine whether the door is opened or closed from both the inside and outside viewpoints. Any combination of such interference indicators, such as those described above, is assumed.

[0084] After generating interference indicators in steps 68, 70, 74, and 76, the support method may loop back to steps 50 and 51 to read the next frames of the respective inner and outer videos. In doing so, the support method may be implemented in real time, for example.

[0085] Figure 3 shows a vehicle 80 comprising a door 82 and an assist system 84 that assists in opening or closing the door 82. The system 84 comprises an acquisition module 86 that acquires at least one image showing at least one region of interest, the at least one region of interest comprising a door operating area or an area adjacent to the door operating area; a detector 88 that detects in at least one image whether at least one object is present in the region of interest; and a determination unit 90 that determines at least one interference indicator for opening or closing the door 82 based on the detection.

[0086] The support system 84 may comprise electronic circuits, processors (shared, dedicated, or group), combinational logic circuits, memory for running one or more software programs, and / or other suitable components for providing the functions described. In other words, the support system 84 may be a computer device. The support system 84 may be connected to memory (a recording medium) which, when executed, may store data, such as computer programs, that perform the support method relating to this disclosure.

[0087] The support system 84, in particular the acquisition module 86, has an input for receiving a digital image or a stream of digital images. In particular, the support system 84, in particular the acquisition module 86, may be connected to an optical sensor, in particular one or more digital cameras 92, such as the fisheye camera described above. The digital camera 92 is configured to record a scene, in particular to output digital data to the support system 84, in particular the acquisition module 86.

[0088] If the method is performed in real time, at least one image may be acquired in real time by an image acquisition module, such as a video acquisition module, or a camera or similar device (video camera, still camera, etc.). Alternatively or additionally, for example, in the case of post-processing of a film process, at least one image may be acquired in advance and later acquired by the support system 84.

[0089] While this disclosure refers to certain preferred embodiments, modifications to such embodiments can be provided without departing from the overall scope of the invention as defined by the claims. In particular, while individual steps of the assistance method are described in a specific manner, individual steps or subgroups of steps can be modified to produce the same functionality. Furthermore, although described in the context of autonomous vehicles, the assistance method and system can be implemented in conventional vehicles, for example, to prevent vehicle occupants from opening doors from the inside when another vehicle (such as a passenger car or bicycle) is overtaking. More schematically, individual features of the various embodiments shown / described can be combined in additional embodiments. Therefore, the description and drawings should be considered illustrative, not restrictive.

Claims

1. A computer-implemented method for assisting in the opening or closing of a vehicle door, The door is configured to open outwards or inwards of the vehicle. - To acquire a first image showing a first region of interest and a second image showing a second region of interest, wherein the first image is an image showing the inside of the door, the second image is an image showing the outside of the door, and the first and second regions of interest include the door operating area of ​​the door or an area adjacent to the door operating area. - In the first image, detect whether a person is present in the first region of interest, and also detect whether an object other than the person is present in the first region of interest, - In the second image, to detect whether the person is present in the second region of interest, - Based on the detections in the first image and the detections in the second image, determine a plurality of interference indicators for opening or closing the door, Includes, The plurality of interference indicators are numerical values ​​representing the level of interference between the person and the opening or closing of the door, and the level of interference between the object other than the person and the opening or closing of the door. The decision includes taking a plurality of the aforementioned detections as input and applying a logic-based algorithm, The detection includes optionally using a machine learning model to extract foreground elements from at least one of the first and second images against a reference background. Further includes initializing the aforementioned reference background, The plurality of interference indicators comprises a first interference indicator determined based on the first image and a second interference indicator determined based on the second image. The method further includes selecting one of the first interference indicator and the second interference indicator that corresponds to the highest interference level. The level of interference is represented by a high number if the detected person or object obstructs the opening or closing of the door, and by a low number if the detected person or object does not interact with the door while it is opening or closing. The interference indicator represents the extent to which a person or object within the region of interest may interfere with the opening and / or closing of the door, and the first and second interference indicators are represented on the same scale, with larger values ​​indicating a higher level of interference. Computer implementation method.

2. The method according to claim 1, wherein the person is optionally detected by skeletal detection.

3. A computer program set comprising instructions that, when executed by at least one computer, perform steps of the method according to claim 1.

4. A recording medium readable by at least one computer, on which at least one computer program is recorded, which includes instructions for performing the steps of the method according to claim 1.

Images