Safety area monitoring method, safety system and display method thereof

Abstract

The invention relates to a method for monitoring a safe area, which is used for solving the problem that a method in the prior art is easy to misjudge that an obstacle invades the safe area. A processing module executes the following steps: generating a predetermined path on the periphery of an image of a driving target, and defining a continuous area not belonging to the obstacle from a predetermined range extending from the predetermined path so as to generate a correction area; wherein under the condition that the obstacle appears in the preset range, a part of a boundary of the correction area corresponds to a part of a contour of the obstacle; judging whether the safety area is completely contained in the correction area or not; and if the judgment result is no, representing that the obstacle has invaded into the safety area. According to the invention, the effects of improving the monitoring accuracy and the driving safety and the like can be realized.

Description

Technical Field

[0001] This invention relates to a monitoring method for vehicles (conveyance), and more particularly to a monitoring method, security system and display method for the security area of ​​vehicles. Background Technology

[0002] In existing driving safety monitoring technologies, taking road vehicles (such as vehicles) as an example, techniques such as object detection and / or semantic segmentation are used to identify identified objects(s) other than the driving target (i.e., the vehicle being driven). A region surrounding the driving target is defined as a safe zone, and it is then determined whether the boundary of any identified object intrudes into this safe zone to avoid collisions between the driving target and the identified object. Since the identified object is the object that the driving target wants to avoid colliding with, it can also be generally referred to as an obstacle(s).

[0003] However, in existing collision avoidance systems (monitoring methods with corresponding safety zones), the identified obstacle area / boundary generated by the obstacle image during image recognition, especially when using only object detection technology, may differ from the actual obstacle area / boundary. Furthermore, the identified obstacle area is always larger than its actual area. This makes it easy for the identified obstacle area to encroach on the safety zone during the assessment of the risk of a collision, causing the monitoring method / collision avoidance system to become overly sensitive and prone to misjudgments / false warnings. These misjudgments may trigger unnecessary subsequent safety mechanisms (such as issuing warning sounds, or controlling the driver to decelerate, accelerate, and / or deviate from the original driving direction), easily interfering with or interrupting the driving process, causing driver distress or even jeopardizing driving safety.

[0004] For example, refer to Figure 1 As shown, it displays images of a driving target T' (such as a driven vehicle) captured from various positions and angles on the body of the driving target T' during its operation. These images include the frontal view (VF'), side view (VL'), and rear view (VR') of the corresponding body. For ease of understanding, the side view (VL') is described using a rearward-facing perspective as an example, but it should still include perspectives from the front side and perspectives from the front side perspective that are angled towards other directions (such as a frontal or rearward-facing perspective).

[0005] Please continue reading. Figures 2-4 , Figure 2 show Figure 1 The input image I' is captured by the side view VL' of other obstacles O' (such as other vehicles). Figure 3 The prior art collision avoidance system generates a safety zone SA' based on the driving target T' in the input image I', and an obstacle zone OA' (corresponding to the identified obstacle zone mentioned above) based on the identified obstacle O'. The obstacle O' closer to the driving target T' is defined as the first obstacle O1' and has a corresponding obstacle zone OA1'; the obstacle O' farther from the driving target T' is defined as the second obstacle O2' and has a corresponding obstacle zone OA2'. Figure 4 This indicates whether the safety area SA' overlaps with the obstacle area OA'; where the symbol "OB'" is defined as the boundary / outline of an obstacle O'. Figure 4 For example, it is used to represent the image boundary of the first obstacle O1'.

[0006] Specifically, based on the existing technology system, the safe area SA' and the obstacle area OA' are generated. The existing technology system can also obtain the relative positions of the safe area SA' and the obstacle area OA' in the input image I' and the area they surround, thereby determining whether any obstacle area OA' has intruded into the safe area SA'. Figure 4 For example, in reality, the obstacle boundary OB' corresponding to the first obstacle O1' does not intrude into the safety area SA'; however, because a part of the obstacle area OA' corresponding to it intersects / overlaps with a part of the safety area SA' to form an intersection area IA', the corresponding existing system will determine that the first obstacle O1' has intruded into the safety area SA' of the driving target T', resulting in a misjudgment.

[0007] Therefore, existing methods for monitoring secure areas still need improvement. Summary of the Invention

[0008] To address the aforementioned problems, the purpose of this invention is to provide a method for monitoring safe areas, which can improve the accuracy and reliability of the monitoring method and enhance driving safety.

[0009] A secondary objective of this invention is to provide a safety system that can improve the accuracy and reliability of monitoring methods and enhance driving safety.

[0010] Another object of the present invention is to provide a display method that can improve the user experience.

[0011] The directional terms or their approximate terms used throughout this invention, such as "front," "rear," "left," "right," "top," "bottom," "inner," "outer," and "side," are primarily for reference to the directions in the accompanying drawings. These directional terms or their approximate terms are only used to assist in explaining and understanding the various embodiments of this invention and are not intended to limit the invention. The "images in each direction" can be images captured from a driving target in various directions towards its periphery. "Each direction" can be, for example, the front, side (left and right), and rear directions. The "side image" can be an image acquired from the direct side, the side front, or the side rear. Furthermore, it should be noted that although this invention uses the monitoring of safe areas of vehicle side (especially side rear) images as an example, it can include and be applied to the monitoring of safe areas of images in other directions, and can also include and be applied to situations where vehicle images in all directions are fitted into a complete panoramic image and corresponding safe areas for monitoring.

[0012] The use of the quantifiers “a” or “an” for the elements and components described throughout this invention is for convenience and to provide the general meaning of the scope of the invention; in this invention, it should be interpreted as including one or at least one, and the concept of a single element also includes the plural, unless it clearly means otherwise.

[0013] The term "coupled" as used throughout this invention includes direct or indirect electrical and / or signal ground connections, which can be selected by those skilled in the art according to their usage requirements.

[0014] The terms "system," "device," "apparatus," "module," and "unit" used throughout this invention may, in whole or in part, include at least one "processor." A processor refers to a data processing device with specific functions, implemented in hardware or a combination of hardware and software, for processing and analyzing information and / or generating corresponding control information. It may also include corresponding data receiving or transmitting units for receiving or transmitting required data; and may further include corresponding databases or storage units (especially non-transient memory units) for reading and storing corresponding data. In particular, unless otherwise specifically excluded or contradicted, the processor may be a collection of multiple processors based on a distributed system architecture, used to include or represent the process, mechanism, and results of information streaming processing between multiple processors.

[0015] It should be noted that the image recognition technologies (including object detection technology and semantic segmentation technology, etc.) involved in the "object detection," "semantic segmentation," and "processing module" for image recognition described throughout this invention are existing technologies and are understood by those skilled in the art. They can be implemented using corresponding and diverse learning network architectures combined with corresponding models and algorithms, and using images containing labeled objects or classified features as training data for corresponding training and verification. Furthermore, the concepts of "safe area," "obstacle area," and "determining whether the above two areas intersect" described throughout this invention are existing technologies and are understood by those skilled in the art.

[0016] The method for monitoring a safe area according to the present invention comprises a processing module having a processor and a pre-established image recognition model performing the following steps: acquiring an input image, the input image being generated from a predefined viewpoint of a driving target, and the input image containing an image corresponding to an obstacle; generating a safe area around the image of the driving target in the input image; generating a predetermined path around the image of the driving target, and defining a continuous area not belonging to the obstacle by extending a predetermined range from the predetermined path to generate a correction area; wherein, when the obstacle appears in the predetermined range, a portion of the boundary of the correction area corresponds to a portion of the outline of the obstacle; and determining whether the safe area is completely contained within the correction area; if the determination result is that the safe area cannot be completely contained within the correction area, it means that the obstacle has intruded into the safe area.

[0017] The safety system of the present invention is set on a driving target and includes: an image capturing unit set on the driving target to acquire an input image of the driving target's surrounding environment; and a processing module coupled to the image capturing unit; wherein the processing module has a processor and a pre-established image recognition model to receive the input image and execute the safety area monitoring method of the present invention.

[0018] Therefore, the safety area monitoring method and safety system of the present invention, by utilizing the characteristic that the generated correction area is close to the boundary of the obstacle, can further determine more accurately whether the obstacle in the input image has intruded into the safety area by the relationship between the safety area and the correction area, thereby reducing the possibility of misjudgment and thus improving the accuracy and reliability of the monitoring method and enhancing driving safety.

[0019] Before determining whether the safe area is completely contained within the modified area, the image recognition model identifies the obstacle from the input image and generates an obstacle region corresponding to the obstacle. It then first determines whether the safe area and the obstacle region intersect. If they intersect, it then determines whether the safe area is completely contained within the modified area. Therefore, by using a two-stage method to determine whether an obstacle intrudes into the safe area, the system can more accurately determine whether an obstacle in the input image intrudes into the safe area, thereby improving the accuracy and reliability of the monitoring method and enhancing driving safety.

[0020] In the process of generating the correction region, the continuous region is defined by all pixels extending from each path pixel on the predetermined path in a first direction to a boundary pixel; wherein each boundary pixel is defined as an extension pixel extending from each path pixel on the predetermined path in the first direction to an obstacle pixel corresponding to the obstacle; in the case where the extension pixel extending from a path pixel on the predetermined path in the first direction is determined to have an obstacle pixel corresponding to the obstacle, the boundary pixel is defined by the path pixel on the predetermined path extending towards the first direction to the boundary pixel. The extended pixel extending in one direction is initially determined to be an obstacle, and is defined by the adjacent extended pixel of the obstacle pixel. In cases where no obstacle pixel corresponding to the obstacle is determined to be an extended pixel extending in the first direction from a path pixel on the predetermined path, the boundary pixel is defined as a pixel on a boundary of the input image from the extended pixel extending in the first direction from the path pixel on the predetermined path. The continuous region is defined using the aforementioned adjacent path pixels, extended pixels, and boundary pixels to generate the corresponding correction region. Therefore, by using this continuous region generation method, a correction region that closely approximates the boundary of an actual obstacle can be obtained, allowing for more accurate determination of whether an obstacle in the input image intrudes into the safe zone. This improves the accuracy and reliability of the monitoring method and enhances driving safety.

[0021] In cases where an extended pixel of a path pixel on the predetermined path extending in the first direction is determined to correspond to an obstacle pixel, the boundary pixel is defined by the preceding extended pixel of the path pixel on the predetermined path extending in the first direction to be initially determined as an obstacle pixel. Therefore, a correction area that closely approximates the boundary of an actual obstacle can be obtained, allowing for more accurate determination of whether an obstacle in the input image intrudes into the safe zone. This improves the accuracy and reliability of the monitoring method and enhances driving safety.

[0022] The range corresponding to this continuous region is limited to the safe area. Therefore, the computational workload of generating this continuous region can be reduced, thereby reducing the corresponding hardware requirements and / or improving computational efficiency.

[0023] The image area of ​​the safe zone of the driving target, either entirely or partially, varies depending on a driving situation; wherein the driving situation includes at least one of a driving speed, a driving direction, a current driving scene, and a turning signal. Therefore, by appropriately varying the safe zone in various situations, the effect of improving driving safety can be achieved.

[0024] The safety system further includes at least one of a warning device, a steering system, a braking system, and a transmission system coupled to the processing module. Therefore, when combined with the corresponding devices and systems, the safety system can take appropriate countermeasures when an obstacle intrudes into the safety area, thereby enhancing driving safety.

[0025] The present invention discloses a method for displaying a monitored security area, which presents information from the monitoring method on a display. The information includes at least one of the following: the input image, the security area, an obstacle area of ​​the obstacle, the correction area, and a difference area; wherein the difference area is defined by a difference between the input image and the correction area.

[0026] Therefore, the method for displaying the monitored safety area of ​​the present invention can provide and display a variety of information for drivers to choose from, thereby improving the user experience.

[0027] In a case where the corresponding judgment result indicates that the safe area cannot be completely contained within the corrected area, the area where the difference region intersects with the safe area is continuously displayed or flashed with a predetermined color. Therefore, by continuously displaying or flashing the predetermined color, a striking visual reminder can be given to the driver, making them aware of obstacles intruding into the safe area and making it easier to judge whether there is a risk of collision, thereby improving user experience and driving safety. Attached Figure Description

[0028] Figure 1 This is a top-down schematic diagram illustrating how a driving target can acquire images from various perspectives via an image capturing device, according to an embodiment of the present invention.

[0029] Figure 2 for Figure 1 A schematic diagram of an image obtained from the side view of the driving target in the embodiment.

[0030] Figure 3 for Figure 2The image shown is used to generate a schematic diagram of the corresponding safe area and obstacle area through image recognition technology.

[0031] Figure 4 for Figure 3 A schematic diagram showing the relationship between the safe zone, obstacle zone, and obstacle boundary.

[0032] Figure 5 This is a flowchart illustrating a preferred embodiment of the security area monitoring method of the present invention.

[0033] Figure 6 This is a schematic diagram of an image taken from the side view of the driving target in one embodiment of the present invention, showing an obstacle that does not intrude into the safe area.

[0034] Figure 7 This is a schematic diagram showing an obstacle encroaching on the safety area in an image from the side view of the driving target according to an embodiment of the present invention.

[0035] Figure 8 for Figure 6 and Figure 7 A schematic diagram of the generation mechanism of the correction region in the illustrated embodiment.

[0036] Figure 9 for Figure 5 A schematic diagram of the judgment rules for the intrusion event judgment step in the embodiment.

[0037] Figure 10 This is a flowchart illustrating another preferred embodiment of the monitoring method for a secure area according to the present invention.

[0038] Figure 11 This is a schematic diagram of a preferred embodiment of the security system of the present invention.

[0039] Explanation of reference numerals in the attached figures

[0040] 1: Image Capture Unit

[0041] 2: Processing Module

[0042] 3: Warning device

[0043] 4: Steering System

[0044] 5: Braking System

[0045] 6: Transmission System

[0046] DM1: First judgment method

[0047] DM2: Second judgment method

[0048] DM3: Third Judgment Method

[0049] I: Input Image

[0050] MA: Correction area

[0051] MOB: Corrects obstacle boundaries

[0052] O: Obstacle

[0053] OA: Obstacle Area

[0054] OB: Obstacle Boundary

[0055] PB: Draw pixels at the boundary

[0056] PE: Extend pixel points

[0057] PIB: Draw pixels at the boundary of the input image.

[0058] POB: Draw pixels at the boundary of obstacles.

[0059] PR: Path drawing of pixels

[0060] R: Pre-defined path

[0061] S1: Image Input Steps

[0062] S2, S2A: Image region generation steps

[0063] S21: Steps for generating a safe zone

[0064] S22: Obstacle Recognition and Corresponding Region Generation Steps

[0065] S23: Correct the region generation steps

[0066] S3, S3A: Intrusion Event Judgment Steps

[0067] S30A: Single decision step

[0068] S31: First Judgment Step

[0069] S32: Second Judgment Step

[0070] SA: Safe Zone

[0071] T: Driving Target

[0072] To: Outline

[0073] x: First direction

[0074] y: Second direction

[0075] I': Input Image

[0076] IA': Intersection region

[0077] O': Obstacle

[0078] O1': First obstacle

[0079] O2': Second obstacle

[0080] OA', OA1', OA2: Obstacle areas

[0081] OB': Obstacle boundary

[0082] SA': Security Zone

[0083] T': Driving Target

[0084] VF': Frontal View

[0085] VL': Side view

[0086] VR': Rear View Detailed Implementation

[0087] To make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments of the present invention are described below in conjunction with the accompanying drawings; in addition, those marked with the same symbols in different drawings are considered to be the same and their descriptions will be omitted.

[0088] Please refer to Figure 5 The diagram shown is a flowchart of the secure area monitoring method of the present invention. This secure area monitoring method includes an image inputting step S1, an image zone generating step S2, and an intrusion event determining step S3. Specifically, the secure area monitoring method is implemented by a processing module 2 (such as...). Figure 11 The processing module 2 has a processor and a pre-established image recognition model.

[0089] like Figures 5-7 As shown, in the image input step S1, an input image I is acquired / input. This input image I is generated from a predefined viewpoint corresponding to a driving target T, and the input image contains an image corresponding to an obstacle O. The predefined viewpoint is as follows: Figure 1 The presented frontal view V F ', Side view V L 'and rear view V R Any of the elements in ', and it can be an image fitted from multiple perspectives (e.g., a panoramic image). Wherein, Figure 6 , 7The technical content of this case is illustrated by the fact that the input image I has an obstacle O located on the side of the driving target T; the symbols x and y represent a first direction and a second direction, respectively, and are particularly used in... Figure 8 The text explains the generation mechanism of a modified region MA. Specifically, Figure 6 Although an obstacle region OA of an obstacle O intrudes into a safe area SA, the obstacle O (which corresponds to a boundary close to the actual obstacle) does not actually intrude into the safe area SA. Figure 7 An obstacle region OA is displayed, showing an obstacle O that has indeed encroached upon a safety zone SA. It should be noted that the input image I obtained in the image input step S1 should only display the image corresponding to the driving target T and the obstacle O; in other words, in the image input step S1, the safety zone SA and the obstacle region OA have not yet been generated and will be generated in the image region generation step S2. Furthermore, it should be noted that the input image I may or may not contain the image corresponding to a driving target T; in particular, in one example, the image of the driving target T can be eliminated through corresponding image processing, so that the corresponding input image I only presents the image corresponding to the obstacle O; in another example, the image of the driving target T can also be optionally displayed so that the driver / user understands the relationship between their vehicle (corresponding to the driving target T in the input image I) and the environment.

[0090] like Figures 5-7 As shown, the image region generation step S2 includes a safe region generation step S21, an obstacle identification and corresponding region generation step S22, and a correction region generation step S23.

[0091] In the safe area generation step S21, a safe area SA is generated around the image of the driving target T in the input image I based on a predefined safe area generation rule. Optionally, the total or partial image area of ​​the safe area SA can vary (increase or decrease) according to a driving situation; the driving situation includes at least one of the following: a travel speed of the driving target T, a travel direction, a current driving scene, and a turning signal. When the driving situation is the travel speed of the driving target T, the total image area corresponding to the safe area SA is positively correlated with the travel speed of the driving target T, and optionally, the safe area SA has a larger image area in a local part (relative to non-travel directions) of the driving target T corresponding to the travel speed. For example, when the driving target T is stationary or moving at a first speed, the safety area SA has a first image area, and when the driving target T moves at a second speed, the safety area SA has a second image area; wherein the first speed and the second speed are a speed value or a speed range, and the second speed is greater than the first speed; the second image area is larger than the first image area, and in particular, the range of the second image area covers the range of the first image area. When the driving situation is a current driving scenario faced by the driving target T, the size of the image area of ​​the safety area SA can vary according to the current driving scenario; for example, during rush hour when there are many vehicles on the road and it is congested, the image area of ​​the safety area SA is smaller; on a wide and unobstructed highway, the image area of ​​the safety area SA is larger. When the driving situation is a turning signal of the driving target T, the safety area SA has a larger image area in the local area on the turning side (relative to the non-turning side) of the driving target T.

[0092] It should be noted that, corresponding Figure 6 , 7 The input image I is generated in a predefined manner, that is, it is an input image I captured at a specific angle from the side and rear view, mounted at or around the rearview mirror of the vehicle serving as the driving target T. Since the mounting position and angle of the input image I are known and specific, the predefined safe area generation rules can be pre-established in the corresponding system (e.g., in the processing module) to generate the safe area SA by extending a specific distance (which can be converted to the number of pixels in the input image I) from the outline To of the driving target T in a specific direction. Furthermore, due to the present invention… Figure 6 , 7 The input image I is generated by the corresponding image capturing unit 1 (e.g., ...). Figure 11The perspective view captured from the side rear angle (as shown) indicates that the closer to the image capturing unit 1, the longer the length of the safe area SA in the first direction x (the more pixels are drawn). Additionally, it should be noted that although the safe area SA presented in the accompanying drawings has the same or similar specific shape, the shape of the safe area SA is not limited thereto.

[0093] In the obstacle identification and corresponding region generation step S22, the pre-established image recognition model identifies the obstacle O from the input image I and generates an obstacle region OA corresponding to the obstacle O. It should be noted that, as mentioned above, the image recognition technology used by the image recognition model (such as object detection technology or semantic segmentation technology) is an application of existing technology, and therefore will not be described in detail.

[0094] Except for reference Figure 6 , 7 In addition, and also refer to Figure 8 In the correction region generation step S23, according to a predefined correction region generation rule, a predetermined path R is generated around the periphery of the image of the driving target T, and a continuous region that is not associated with or different from the obstacle O is defined within a preset range extending from the predetermined path R to generate a correction region MA; wherein, when the obstacle O appears within the preset range, a portion of the boundary of the correction region MA / continuous region corresponds to a portion of the outline of the obstacle. Therefore, in the absence of any obstacle O, or in the absence of any obstacle O encroaching on a safe area SA of the driving target T, the safe area SA is entirely located within the correction region MA.

[0095] Specifically, the predetermined path R can be defined by any of the following methods. It should be noted that the examples of these predetermined paths R are only a part of a variety of possible embodiments, used to more clearly illustrate the content of the invention, and the invention is not limited thereto.

[0096] (1) A predetermined path R defined by the outer periphery of the outline To of the driving target T in the input image I; preferably, the predetermined path R extends from the bottom boundary to the top boundary of the input image I. Therefore, a larger correction area MA can be obtained, which helps to provide more information about obstacle boundaries OB in other image recognition functions. In this invention... Figures 6-8 The correction region MA in the middle is also generated in a similar way.

[0097] (2) The predetermined path R is defined by a boundary of the safe area SA that is close to the driving target T. Therefore, information about the obstacle boundary OB that is highly related to the safe area SA can be obtained by using a smaller correction area MA, thereby reducing the corresponding amount of computation, relatively reducing the corresponding hardware requirements and / or improving computational efficiency.

[0098] Specifically, to more clearly explain the rules for generating the modified region, Figure 8 This section explains the principle of generating the continuous region / corrected region MA from the perspective of procedural image processing, using the predetermined path R combined with various defined pixels. In the correction region generation rule, the continuous region is represented by pixels P along each path on the predetermined path R. R Draw a pixel P extending in a first direction x to a boundary. B Defined by all pixel points; where the boundary pixel P B To determine each path on the predetermined path R, draw a pixel P. R an extended pixel point P extending toward a first direction x E Is a pixel defined for an obstacle corresponding to obstacle O? A pixel P is drawn on a path along the predetermined path R. R The point P is drawn by extending the first direction x. E In cases where an obstacle O is identified as having a corresponding obstacle point, a boundary point P is drawn. B Draw a pixel P on the predetermined path R. R The point P is drawn by extending the first direction x. E The first pixel that is initially identified as obstacle O is the extension of the pixel P that precedes it. E Defined; where, for ease of understanding the above technical content, the boundary is drawn with a pixel P. B Can be defined and displayed as follows Figure 8 Draw pixel P on the boundary of the obstacle in the middle. OB Additionally, a pixel P is drawn along a path on the predetermined path R. R The point P is drawn by extending the first direction x. E In the case where there is no corresponding obstacle point for obstacle O, the boundary is drawn as a pixel point P. B Draw a pixel P on the predetermined path R. R The point P is drawn by extending the first direction x. E It is determined that the pixel P is defined as a pixel on a boundary of the input image I; wherein, for ease of understanding the above technical content, this boundary pixel P is defined as... B Can be defined and displayed as follows Figure 8 P is the pixel point drawn on the boundary of the input image. IB Draw the vertex P along the path that forms the adjacency relationship described above.R , extend the pixel point P E And draw the boundary point P B Define this continuous region to correspondingly generate the corrected region MA. Note that the pixel point P is drawn in this extension. E In the case where the obstacle is initially identified as a pixel, this example uses the previous extension of the pixel to draw pixel P. E As a boundary point P B (This will exclude any pixel point / feature corresponding to obstacle O from the continuous region / corrected region MA). However, in other applications, to expand or shrink the continuous region according to different needs, a pixel point adjacent to the obstacle pixel point (a pixel point farther away from or closer to the driving target T) can be used as the boundary pixel point P. B Furthermore, these changes should still be considered within the scope of this invention. Figure 8 The thick dashed line segment shown is defined as a modified obstacle boundary MOB, and is drawn from multiple boundaries that are determined to have obstacles O, with pixel points P. B This constitutes a boundary OB that is closer to the actual obstacle O. In this example, the aforementioned preset range is defined by the predetermined path R toward the boundary of the input image I in a predetermined direction (e.g., the first direction x).

[0099] Alternatively, in another example (not shown), the pixel P is drawn based on the boundary described above. B The mechanism for determining and defining boundaries extends in the first direction x, and can also be used to draw the boundary point P. B The determination and definition are based on extending in a second direction, y. For example, an auxiliary determination line segment can be formed by extending a pixel at one end of the predetermined path R in the first direction x, and by extending each pixel on the auxiliary line segment in the second direction y, which is determined to be an extension pixel P of the initial obstacle pixel point O. E The neighboring or previous extended pixel P E Define a boundary and draw a prime point P. B Alternatively, a pixel on a boundary of the input image I can be defined as a boundary pixel P. B This is used to define a corresponding corrected obstacle boundary MOB. Preferably, the corrected regions MA defined above in the first direction x and the second direction y can be joined together to define an updated corrected region MA, so that the corrected obstacle boundary MOB of the updated corrected region MA can be closer to the actual obstacle boundary OB of the obstacle O.

[0100] Optionally, to reduce the computational load of generating a continuous region, the range corresponding to the continuous region can be limited to the safety region SA; that is, all the pixel points corresponding to the continuous region and the defined correction region MA are located within the safety region SA. Specifically, in the process of generating / determining the continuous region, in the first direction x, at least one boundary of the safety region SA that is closer to the outer periphery of the driving target T is taken as the predetermined path R, and pixel points P of the safety region SA that are farther away from the path are drawn. R A boundary is replaced by a boundary of the input image I to generate a modified region MA defined within the safe region SA. In this example, the aforementioned preset range is defined by the boundary of the safe region SA from the predetermined path R toward a predetermined direction (e.g., the first direction x).

[0101] It should be noted that the function of determining whether any pixel of obstacle O exists in the continuous region in step S23 can also be implemented by a pre-established image recognition model in the aforementioned processing module 2. This image recognition model is understandable to those skilled in the art and will not be described further. Furthermore, the generated corrected region MA, in addition to including a drivable region, may also include a non-drivable region in certain specific situations. The drivable region can be, for example, a road surface suitable for vehicle travel, such as asphalt, gravel, brick, dirt, or grass. The non-drivable region can be, for example, a sidewalk or arcade, a road surface not suitable for vehicle travel. The drivable region, non-drivable region, and their types can be implemented using corresponding image recognition technologies, and preferably, their corresponding types can be varied according to different situations.

[0102] It is particularly noteworthy that the method for generating the modified region MA proposed in this invention can be implemented, for example, using regression analysis in image recognition technology, and can significantly improve computational efficiency. For instance, in a practical application, using the same input image I with obstacle O and other conditions identical, the computational efficiency of regression analysis and semantic segmentation is compared. Taking a 640-pixel x 640-pixel image as an example, the floating-point computation of regression analysis is only 2.4G, while that of semantic segmentation is 34.2G. In other words, the method for generating the modified region MA of this invention reduces computation by 92.98% compared to existing semantic segmentation techniques, thus significantly improving computational performance. Specifically, the method for generating the modified region MA of this invention can identify the modified obstacle boundary MOB most likely to intrude into the safe region SA based solely on the contour of the obstacle O and the adjacent side of the safe region SA. Therefore, compared to semantic segmentation techniques that identify the entire obstacle boundary OB of obstacle O, the computational workload is significantly reduced. In particular, when using semantic segmentation technology to identify the boundary of an obstacle O, if the corresponding input image I is reduced in dimension or a smaller recognizable pixel is used to improve computational efficiency, the obtained identified obstacle boundary may still have errors. In this case, by using the technology of generating a corrected obstacle boundary MOB according to the present invention, and by using a smaller or smaller recognizable pixel, the obtained corrected obstacle boundary MOB will be closer to the actual obstacle boundary OB than the identified obstacle boundary (obtained by semantic segmentation technology).

[0103] Except for reference Figures 5-7 In addition, and also refer to Figure 8 In the intrusion event judgment step S3, based on the security area SA, the obstacle area OA and the correction area MA obtained in the corresponding area generation step S2, the intrusion event judgment step S3 includes a first judgment step S31 and a second judgment step S32 to more accurately determine whether the obstacle O has intruded into the security area SA.

[0104] In the first judgment step S31, it is determined whether the safe area SA and the obstacle area OA intersect. If the judgment result is no (i.e., no intersection), it means that no intrusion event has occurred. If the judgment result is yes (i.e., there is intersection), then the second judgment step S32 is executed.

[0105] In the second determination step S32, it is further determined whether the safety area SA is completely contained within the modified area MA. This is equivalent to determining whether all or part of the obstacle boundary OB corresponding to the obstacle O is located within the safety area SA. If the determination result is that the safety area SA is completely contained within the modified area MA (meaning the entire obstacle boundary OB is located outside the safety area SA), then... Figure 6 If the result indicates that the safe zone SA cannot be completely contained within the modified zone MA (this corresponds to the boundary OB of the obstacle having completely or partially invaded the safe zone SA), then no intrusion event has occurred. Figure 7 If an obstacle O (as shown in the image) is detected, it indicates that an intrusion event has occurred, confirming that the obstacle O has indeed intruded into the security area SA. Specifically, upon detecting an intrusion event, the corresponding system / processing module will generate a corresponding intrusion signal to further trigger subsequent related security mechanisms.

[0106] Specifically, in the second judgment step S32, the method used to determine whether an intrusion has actually occurred can employ a first judgment method DM1, a second judgment method DM2, or a third judgment method DM3. In the first judgment method DM1, it is determined whether the area of ​​the region where the modified region MA, the safe region SA, and the obstacle region OA intersect is equal to the area of ​​the region where the safe region SA intersects with the obstacle region OA (i.e., the same meaning as "determining whether the safe region SA is completely contained within the modified region MA"). If they are equal, it means that the safe region SA is completely contained within the modified region MA, and the entire boundary OB of the obstacle is located outside the safe region SA (e.g., ...). Figure 6 If the values ​​are not equal (as shown), it is determined that no intrusion event has occurred; if they are not equal, it means that the safe area SA cannot be completely contained within the modified area MA, and all or part of the obstacle boundary OB is located within the safe area SA (e.g., ...). Figure 7 As shown in the image, this indicates that an intrusion event has occurred.

[0107] In the second determination method DM2, it is determined whether the area of ​​the intersection of the modified region MA and the safe region SA is equal to the area of ​​the safe region SA; in other words, it is determined whether the modified region MA can completely cover the safe region SA (i.e., the same meaning as "determining whether the safe region SA is completely contained within the modified region MA"). If they are equal, it means that the safe region SA is completely contained within the modified region MA, and the entire boundary OB of the obstacle is located outside the safe region SA (e.g., ...). Figure 6 If the values ​​are not equal (as shown), it is determined that no intrusion event has occurred; if they are not equal, it means that the safe area SA cannot be completely contained within the modified area MA, and all or part of the obstacle boundary OB is located within the safe area SA (e.g., ...). Figure 7 As shown in the image, this indicates that an intrusion event has occurred.

[0108] In the third judgment method DM3, it is determined whether a difference set region between the modified region MA and the input image I intersects with the safe region SA (i.e., the same meaning as "determining whether the safe region SA is completely contained within the modified region MA"). If no intersection occurs, it means that the safe region SA is completely contained within the modified region MA, and the entire boundary OB of the obstacle is located outside the safe region SA (e.g., Figure 6 If an intersection occurs, it means that the safe zone SA cannot be completely contained within the modified zone MA, and all or part of the obstacle boundary OB is located within the safe zone SA (as shown). Figure 7 As shown in the image, this indicates that an intrusion event has occurred.

[0109] Alternatively, please refer to Figure 10 As shown, this is another embodiment of the monitoring method for a secure area according to the present invention, similar to... Figure 5 The process also includes an image input step S1, and additionally includes an image region generation step S2A and an intrusion event judgment step S3A. In the image region generation step S2A, the image region generation step S2A includes the safe area generation step S21 and the correction area generation step S23; in other words, the image region generation step S2A can eliminate the aforementioned obstacle region generation step S22. In the intrusion event judgment step S3A, one of the aforementioned second judgment method DM2 and the aforementioned third judgment method DM3 can be directly used in a single judgment step S30A (corresponding to the aforementioned second judgment step S32) to achieve a more accurate judgment of whether the obstacle O has intruded into the safe area SA; in other words, the intrusion event judgment step S3A can eliminate the aforementioned first judgment step S31. Therefore, corresponding to Figure 10 The method is more Figure 5 This method can further improve the overall image recognition and analysis and security area monitoring performance by eliminating the obstacle area generation step S22 and the first judgment step S31.

[0110] Please refer to Figure 11 As shown, in order to implement the aforementioned monitoring method for the secure area (such as...) Figure 5 and 10As shown, the present invention can be configured with a safety system on a driving target T, and includes an image acquisition unit 1, a processing module 2, an optional warning device 3, an optional steering system 4, an optional braking system 5, and an optional transmission system 6. The processing module 2 is coupled to one or more of the image acquisition unit 1, the warning device 3, the steering system 4, the braking system 5, and the transmission system 6 to perform functions such as transmission, reception, reading, storage, calculation, or control of corresponding data in the safety system.

[0111] The image capturing unit 1 is at least one in number and is disposed on the driving target T to acquire an input image I of the surrounding environment of the driving target T. The input image I can be... Figure 1 An image corresponding to various perspectives, such as an image corresponding to at least one of the front view VF', side view VL' and rear view VR', or an image fitted by the above multiple perspectives, especially a panoramic image.

[0112] The processing module 2 has a processor and is coupled to the image capturing unit 1 to receive the input image I and execute the aforementioned security area monitoring method (such as...). Figure 5 , 10 As shown, the system determines whether any obstacle O has intruded into the safety area SA of the driving target T. The processing module 2 has a pre-established image recognition model. When an obstacle O is present in the input image I, it can identify the obstacle O and generate the corresponding obstacle area OA, and execute the correction area generation step S23 to obtain the correction area MA. If the processing module 2 determines that the safety area SA cannot be completely contained within the correction area MA (i.e., the obstacle O has intruded into the safety area SA), the processing module 2 issues a warning message to at least one of the warning device 3, the steering system 4, the braking system 5, and the transmission system 6 to trigger the corresponding safety mechanism. It should be noted that, as described above, the function of identifying the obstacle O and generating the corresponding obstacle area OA can be achieved using existing image recognition technologies (object detection technology and / or semantic segmentation technology, especially object detection technology).

[0113] In addition, based on the aforementioned monitoring methods for secure areas (such as...) Figure 5 and 10As shown, this invention proposes a method for displaying a monitored safety area, which can be displayed on a monitor (not shown, but particularly installed inside the driving target T), and includes one or more of the following display methods: a first display method, a second display method, a third display method, a fourth display method, a fifth display method, and a sixth display method. Therefore, the driver / user can choose their preferred display method according to their needs or habits, thus enhancing the user experience.

[0114] In this first display mode (e.g.) Figure 2 As shown, only the input image I is displayed. Therefore, a clean image without additional auxiliary areas can be provided to the driver. The auxiliary areas refer to one or more of the safety area SA, obstacle area OA, correction area MA, and difference area; the difference area is defined by the difference between the input image I and a correction area MA corresponding to an obstacle OA.

[0115] In this second display method (e.g.) Figure 6 , 7 As shown, the input image I and the corresponding safe area SA are displayed. Therefore, compared to the first display method, the display of the safe area SA makes it easier for the driver to observe whether other obstacles O have intruded into the safe area SA.

[0116] In this third display method (e.g.) Figure 6 , 7 As shown, the system displays an input image I and a corresponding safe area SA, showing a scenario where an obstacle O is present in the input image I, along with an obstacle area OA for the obstacle O. This allows the driver to easily observe whether other obstacles O have intruded into the safe area SA. Therefore, compared to the second display method, additionally displaying the obstacle area OA helps the driver to be alerted to whether other obstacles O have intruded into the safe area SA.

[0117] In this fourth display method (e.g.) Figure 6 , 7 As shown, the system displays an input image I and a corresponding safe zone SA, showing a scenario where an obstacle O is present in the input image I. It also displays an obstacle region OA and a correction region MA for the obstacle O. Therefore, compared to the second display method, additionally displaying the obstacle region OA and the correction region MA helps provide the driver with more real-time information, allowing the driver to observe this information and assess whether other obstacles O have intruded into the safe zone SA.

[0118] In this fifth display method (such as) Figure 6 , 7As shown, the system displays an input image I and a corresponding safety zone SA, showing a scenario where an obstacle O is present in the input image I. Simultaneously, it displays a difference region corresponding to the obstacle O, defined by the difference between the input image I and a correction region MA corresponding to the obstacle O. Compared to the third or fourth display method, this difference region displays the obstacle boundary of the obstacle O near the safety zone SA. Therefore, replacing the display of the obstacle region OA and / or the correction region MA with this difference region better shows the relationship between the boundary of the obstacle O and the safety zone SA, allowing the driver to determine whether other obstacles O have intruded into the safety zone SA.

[0119] In this sixth display mode, in the case where no intrusion event occurs (i.e., the security area SA is completely contained within the correction area MA), the corresponding image (e.g., ...) is displayed in the first display mode. Figure 2 (as shown); In the event of an intrusion (i.e., the security area SA cannot be completely contained within the correction area MA), the corresponding image is displayed in one of the second, third, fourth, and fifth display modes (e.g., Figure 6 , 7 (As shown). Therefore, the corresponding auxiliary area is only displayed when an intrusion occurs, so that the corresponding input image I does not have too much unnecessary information, which can improve the driver's user experience.

[0120] Optionally, in this display method, in the event of an intrusion (i.e., the security area SA cannot be completely contained within the modified area MA), the area where the difference region intersects with the security area SA is continuously displayed or flashed in a predetermined color to prominently display the modified intrusion area where an obstacle O intrudes into the security area SA.

[0121] It should be noted that, for ease of understanding of the technical content of the present invention, the input image I in the present invention is an image with a specific perspective, which is a side view (especially the left side view in the direction of travel) from the driving target T. However, the input image I is not limited to the single and specific perspective shown in the figure. It is understood that all the technical content and features of the present invention can be applied to images with other perspectives (including images fitted by multiple perspectives, such as panoramic images).

[0122] In addition, it should be noted that, in order to facilitate understanding of the technical content of the present invention, although the driving target T in the present invention is a vehicle in a land scenario, the technical content of the present invention (including at least the safety area SA and the correction area MA) can be applied to driving targets T in different types of scenarios, such as vehicles in air, water or underwater scenarios.

[0123] In summary, the safety zone monitoring method and safety system of the present invention, by generating a correction area that closely approximates the boundary of an obstacle, can further determine more accurately whether an obstacle in the input image has encroached upon the safety zone through the relationship between the safety zone and the correction area, or through the relationship between the safety zone, the correction area, and the obstacle area. This reduces the likelihood of false judgments, thereby improving the accuracy and reliability of the monitoring method and enhancing driving safety. Furthermore, the display method of the present invention, based on the safety zone monitoring method of the present invention, can provide multiple display options for the driver, thus enhancing the user experience.

[0124] Although the present invention has been disclosed using the above preferred embodiments, it is not intended to limit the present invention. Any modifications and alterations made by those skilled in the art to the above embodiments without departing from the spirit and scope of the present invention shall still fall within the technical scope protected by the present invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A method for monitoring a secure area, comprising a processing module having a processor and a pre-established image recognition model performing the following steps, characterized in that... Include: Acquire an input image, which is generated from a predefined viewpoint corresponding to a driving target, and the input image contains an image corresponding to an obstacle; A safe zone is generated around the image corresponding to the driving target in the input image; A predetermined path is generated around the periphery of the image of the driving target. A predetermined range extending from the predetermined path defines a continuous region that is not an obstacle to generate a correction region. Where the obstacle is present within the predetermined range, a portion of the boundary of the correction region corresponds to a portion of the outline of the obstacle. Determine whether the safe area is completely contained within the modified area; if the result is that the safe area cannot be completely contained within the modified area, it means that the obstacle has intruded into the safe area.

2. The method for monitoring a secure area as described in claim 1, characterized in that, Before determining whether the safe area is completely contained within the modified area, the image recognition model identifies the obstacle from the input image and generates an obstacle region corresponding to the obstacle. It then determines whether the safe area and the obstacle region intersect. If the result is that they intersect, it then determines whether the safe area is completely contained within the modified area.

3. The method for monitoring a secure area as described in claim 1, characterized in that, In the process of generating the correction region, the continuous region is defined by all pixels extending from each path pixel on the predetermined path in a first direction to a boundary pixel; wherein, each boundary pixel is defined as an extension pixel extending from each path pixel on the predetermined path in the first direction to an obstacle pixel corresponding to the obstacle; in the case where the extension pixel extending from a path pixel on the predetermined path in the first direction is determined to have an obstacle pixel corresponding to the obstacle, the boundary pixel is defined by the path pixel on the predetermined path extending in the first direction to the first boundary pixel. The extended pixel extending in the direction is initially determined to be an obstacle pixel adjacent to the obstacle pixel. In the case where the extended pixel extending in the first direction of a path pixel on the predetermined path is determined to have no obstacle pixel corresponding to the obstacle, the boundary pixel is defined as a pixel on a boundary of the input image where the extended pixel extending in the first direction of the path pixel on the predetermined path is determined to be an obstacle pixel. The continuous region is defined by the aforementioned path pixels, extended pixels, and boundary pixels that form an adjacency relationship, so as to generate the correction region accordingly.

4. The method for monitoring a secure area as described in claim 3, characterized in that, In the case where an obstacle pixel is determined to be an obstacle pixel corresponding to a path pixel on the predetermined path extending in the first direction, the boundary pixel is defined by the preceding path pixel on the predetermined path that was first determined to be an obstacle pixel.

5. The method for monitoring a secure area as described in claim 1, characterized in that, The range corresponding to this continuous area is limited to this safe area.

6. The method for monitoring a secure area as described in any one of claims 1 to 5, characterized in that, The image area of ​​the safe zone of the driving target, whether in whole or in part, varies according to a driving situation, which includes at least one of a driving speed, a driving direction, a current driving scene, and a turning signal.

7. A safety system installed on a driving target, characterized in that... Include: An image capturing unit is disposed at the driving target to acquire an input image of the driving target's surrounding environment; and A processing module is coupled to the image capturing unit; wherein the processing module has a processor and a pre-established image recognition model to receive the input image and execute the monitoring method for the safe area as described in any one of claims 1 to 6.

8. The security system as described in claim 7, characterized in that, Additionally, at least one of a warning device, a steering system, a braking system, and a transmission system is coupled to the processing module.

9. A method for displaying a monitored security area, characterized in that: Information from the monitoring method for a safe area as described in any one of claims 1 to 6 is presented on a display, the information including at least one of the input image, the safe area, an obstacle area of ​​the obstacle, the correction area, and a difference area; wherein the difference area is defined by a difference between the input image and the correction area.

10. The method for displaying a monitored security area as described in claim 9, characterized in that, In cases where the corresponding judgment result is that the safe area cannot be completely contained in the correction area, the area where the difference area intersects with the safe area is continuously displayed or flashed with a predetermined color.

Images