Adaptive image monitoring method and region of interest updating method thereof

Abstract

The invention relates to an adaptive image monitoring method and a method for updating a region of interest thereof, which are used for solving the problem that important object information cannot be provided by a region of interest in a known monitoring method. The method comprises the following steps of: identifying at least one object and a category corresponding to the object from a monitoring image; calculating an evaluation score of each of the at least one object, and defining an object corresponding to the maximum evaluation score as a maximum score object; determining a region of interest from one of a plurality of sub-regions in the monitoring image, wherein the region of interest has the object with the maximum score; the sum of the evaluation scores of the objects in the region of interest is not less than the sum of the evaluation scores of the objects in other sub-regions containing the object with the maximum score. The driving safety can be improved.

Description

Technical Field

[0001] The invention relates to an image monitoring method, in particular to an adaptive image monitoring method and a method for updating a region of interest thereof. Background Art

[0002] like Figure 1 As shown, in the known vehicle image monitoring method, the corresponding monitoring image MI' can be captured and synthesized through the image capture unit, and then a range is expanded outward in the monitoring image MI' with the driving vehicle 1' as the center to generate a fixed viewing angle and a fixed range of the focus area IZ', and the generated image corresponding to the focus area IZ' is provided to the driver for reference, so as to allow the driver to observe whether there is a specific object O' (including parking spaces, vehicles, pedestrians, animals, other signs of concern, other markings of concern or other objects, etc.) in the focus area IZ'.

[0003] However, the focus area IZ' generated by the known monitoring method is limited to a fixed extension of a specific range from the center of the driving vehicle 1', and does not adaptively adjust the position of the focus area IZ' according to the importance level corresponding to various categories of the object O' in the monitoring image MI'. For example, as shown in Figure 1, although there is an object O' in the monitoring image MI', the focus area IZ' formed by the known vehicle image monitoring method will not present the object O', so the driver may be negligent and collide with the object O'.

[0004] In view of this, the conventional image monitoring method still needs to be improved. Summary of the invention

[0005] To solve the above problems, an object of the present invention is to provide an adaptive image monitoring method to improve driving safety.

[0006] Another object of the present invention is to provide a method for updating a region of interest, which can improve driving safety.

[0007] The elements and components described throughout the present invention use the quantifiers "a" or "an" for convenience only and to provide a general meaning of the scope of the present invention; they should be interpreted in the present invention as including one or at least one, and a single concept also includes plural cases, unless it is obvious that it means otherwise.

[0008] The term "coupling" as used throughout the present invention includes direct or indirect electrical and / or signal connection, which can be selected by a person skilled in the art according to usage requirements.

[0009] The object recognition technology involved in the "object detection", "semantic segmentation" and "processing module" for image recognition described in the full text of the present invention is already a known technology and can be understood by a person of ordinary knowledge; the implementation of the object recognition technology can include the use of a multi-faceted learning network architecture with corresponding models and algorithms, and in particular, can include images with labeled objects or classified features as training data, and perform corresponding training and verification to complete it.

[0010] An adaptive image monitoring method of the present invention executes the following steps through a computer, including: acquiring multiple captured images from a driving vehicle in multiple directions toward the periphery; making each of the captured images have the same coordinate system to synthesize a monitoring image; identifying at least one object and a category corresponding to it from the monitoring image; calculating an evaluation score for each of the objects, and an object corresponding to the largest of the evaluation scores is defined as a maximum score object; and determining a focus area from one of the multiple sub-areas in the monitoring image, and the focus area has the maximum score object; the sum of the evaluation scores of the objects in the focus area is not less than the sum of the evaluation scores of the objects in other sub-areas containing the maximum score object.

[0011] Accordingly, the adaptive image monitoring method of the present invention provides the driver with information that requires special attention in the current environment by making the focus area have the object with the maximum score, and in the case of having multiple objects, making the sum of the evaluation scores of the objects in the focus area not less than the sum of the evaluation scores of the objects in other sub-areas containing the same maximum score object, thereby achieving the effect of improving driving safety.

[0012] A reference point can be defined from a preset position corresponding to the driving vehicle in the monitoring image, so that the focus area includes the reference point. In this way, the driver can more clearly understand the information that needs special attention in the current environment through the positional relationship between the reference point on the driving vehicle and various objects, thereby achieving the effect of improving driving safety.

[0013] The reference point may be the center of the vehicle in the monitoring image. Thus, the driver can more clearly understand the information that needs special attention in the current environment through the positional relationship between the center of the vehicle and various objects, thereby achieving the effect of improving driving safety.

[0014] The focus area may include a complete outline of the driving vehicle in the monitoring image. Thus, the driver can more clearly understand the information that needs special attention in the current environment through the positional relationship between the complete outline of the driving vehicle and each object, thereby achieving the effect of improving driving safety.

[0015] When at least one object is identified, a category of each object is obtained, and the evaluation score of each object can be calculated by the corresponding category. In this way, the focus area can present images that should be paid attention to first according to the importance of the object category, thereby achieving the effect of improving driving safety.

[0016] When at least one object is identified, a category of each object is obtained, and a relative position of each object relative to the reference point is obtained; the evaluation score of each object can be calculated based on the relationship between the corresponding category and relative position. In this way, the focus area can consider the importance of the object category and relative position to present the image that should be paid attention to first, thereby achieving the effect of improving driving safety.

[0017] The evaluation score can be calculated by the following formula:

[0018]

[0019] Sf represents the final calculated value of the evaluation score; W1 represents a preset object category weight of the category corresponding to the object; d represents a distance between the object and the reference point; c is a positive constant. In this way, the importance of the object category considered by the focus area will be inversely proportional to the distance of the relative position, so that the information presented by the focus area has the effect of considering the importance of the object and the collision risk at the same time.

[0020] The evaluation score can be calculated by the following formula:

[0021] Sf = W1 × dR;

[0022] dR represents a distance grade score corresponding to the distance between the object and the reference point; wherein, multiple distance intervals are divided according to multiple distance ranges, and the smaller the maximum value of the distance interval corresponding to the distance, the larger the corresponding distance grade score. Thus, through the distance grade score corresponding to the distance interval, the assessment score of the area with higher collision risk can be effectively strengthened, so that the information presented in the focus area has the effect of considering the importance of the object and the objects with higher collision risk at the same time.

[0023] Wherein, the monitoring image is established by a two-dimensional coordinate system and has corresponding x-axis and y-axis, and the evaluation score can be calculated by the following formula:.

[0024] Sf=|cosθ|×W1× d c Or Sf = |cosθ| × W1 × dR;

[0025] θ represents the angle between the line segment of the object and the reference point and the y-axis. Thus, by using cosθ in the formula, the evaluation score can be given the characteristic of considering the risk of the driving vehicle colliding with the object in the straight-line motion direction, so that the information presented in the focus area has the function of considering the importance of the object and the object with which the driving vehicle is more likely to collide in the straight-line motion.

[0026] Wherein, the monitoring image is established by a two-dimensional coordinate system and has corresponding x-axis and y-axis, and the evaluation score can be calculated by the following formula:

[0027]

[0028] S1 represents an initial score; S2 represents an intermediate score; W2 represents a weight value of a direction of travel, and is greater than 1. Thus, by using cosθ in the formula, objects on the positive y-axis side can be positive numbers, and objects on the negative y-axis side can be negative numbers; the intermediate score (S2) is then used to consider the direction of travel of the driving vehicle (forward gear and reverse gear); and the evaluation score (Sf) is then used to multiply the intermediate score (S2) of the objects on the same side of the driving vehicle's direction of travel by the weight value of the direction of travel (W), so that the information presented in the focus area has the function of considering the importance of the objects, and at the same time strengthening the effect of considering the collision risk of the objects on the same side of the driving vehicle's direction of travel.

[0029] The value of the traveling direction weight direction may be 3. In this way, the evaluation scores of the objects on the same side as the traveling direction of the driving vehicle are appropriately amplified.

[0030] The monitoring image is established by a two-dimensional coordinate system and has corresponding x-axis and y-axis; after determining the focus area, a vehicle center of the driving vehicle is used as the reference point; a deviation direction is defined by connecting the vehicle center in the monitoring image and a center of the focus area; a deviation angle is defined between the deviation direction and the positive y-axis; and a conversion angle is defined by the clockwise rotation of the vehicle center by the deviation angle; the monitoring image is converted from an image of the two-dimensional coordinate system to an image of the three-dimensional coordinate system; the three-dimensional coordinate system has corresponding x-axis, y-axis and z-axis, and the three-dimensional coordinate system and the two-dimensional coordinate system have the same configuration on the x-axis and y-axis; the image of the three-dimensional coordinate system is generated by observing from a perspective of the conversion angle, from a height raised on the z-axis from the boundary of the image of the three-dimensional coordinate system, toward the center of the vehicle. In this way, the monitoring image can be converted from a two-dimensional image to a three-dimensional image to present the object of attention, provide the driver with more intuitive visual information, and have the effect of enhancing visual presence, reality and connection, thereby achieving the effect of improving driving safety.

[0031] The method for updating the region of interest of the present invention, after applying the above-mentioned adaptive image monitoring method to determine a region of interest, comprises: defining a preset area within the region of interest, wherein the range of the preset area is smaller than the range of the region of interest; and when the vehicle center of the driving vehicle moves beyond the preset area, executing the above-mentioned adaptive image monitoring method again to generate an updated region of interest

[0032] Accordingly, the method for updating the area of ​​interest of the present invention can avoid frequent updates of the area of ​​interest by updating the area of ​​interest when the vehicle center of the driving vehicle moves beyond the preset area, so as to appropriately update the area of ​​interest to obtain the latest information that should be paid attention to, thereby achieving the effect of improving driving safety.

[0033] The preset area may be formed by shrinking the edge of the focus area inward to form an area surrounded by a shrinkage ratio. In this way, the preset area can be appropriately defined, thereby achieving the effect of improving driving safety.

[0034] The inner shrinkage ratio may be 0.5. In this way, the preset area can be better defined, thereby achieving the effect of improving driving safety.

[0035] The preset area may be formed by an area surrounded by the body contour of the driving vehicle expanding outward by an expansion ratio. In this way, the preset area may be appropriately defined, thereby achieving the effect of improving driving safety.

[0036] The expansion ratio may be 2. In this way, the preset area may be better defined, thereby achieving the effect of improving driving safety.

[0037] Simple diagram description

[0038] Figure 1 A schematic diagram showing the missing area of ​​interest in a known surveillance image.

[0039] Figure 2 A system block diagram of the image monitoring system of the present invention is shown.

[0040] Figure 3 The steps of the adaptive image monitoring method of the present invention are shown.

[0041] Figure 4 A schematic diagram showing the distribution of objects in the surveillance image of the present invention is shown.

[0042] Figure 5 Show according to Figure 4 A surveillance image of a surveillance image, and a schematic diagram of a region of interest generated by the present invention.

[0043] Figure 6 Show according to Figure 4 Surveillance image of the area of ​​interest, schematic diagram of other sub-areas not in the area of ​​interest.

[0044] Figure 7 Show according to Figure 4 Surveillance image, schematic diagram of the area of ​​interest generated by known technology.

[0045] Figure 8 A schematic diagram showing a two-dimensional monitoring image before a three-dimensional image is generated.

[0046] Fig. 9 Show according to Figure 8 Schematic diagram of the rules of the generated 3D image.

[0047] Fig.10 Show according to Figure 8 and 9 Schematic diagram of the generated 3D image.

[0048] Fig.11 A schematic diagram showing an example of a predetermined area in a region of interest.

[0049] Fig.12 Show continued Fig.11 After that, the center of the vehicle of the driving vehicle exceeds the preset area.

[0050] Fig.13 Show continued Fig.12 After that, the schematic diagram of the monitoring image and the area of ​​interest is updated according to the current status.

[0051] Fig.14 A schematic diagram showing another example of a preset area in a region of interest.

[0052] Wherein: 1: driving vehicle, 1C: vehicle center, 2: image capture unit, 3: processing module, 31: object recognition model, 4: display, BD: deviation direction, h: height, IZ: focus area, IZC: regional center, MI: monitoring image, O, O1, O2, O3, O4, O5, O6: object, OD: observation angle, PA: preset area, R: reference point, S1: image capture step, S2: monitoring image synthesis step, S3: object recognition step, S4: object evaluation score calculation step, S5: focus area generation step, S51: stereo image generation step, S6: focus area update step, SA: sub-area, Sf: evaluation score, θ B :Offset angle, θ T :Convert Angle

[0053] Known technologies: 1': driving vehicle, IZ': area of ​​interest, MI': monitoring image, O': object. Implementation

[0055] In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the preferred embodiments of the present invention are specifically cited below and described in detail with reference to the accompanying drawings; in addition, those marked with the same symbols in different drawings are regarded as the same and their descriptions are omitted.

[0056] Please refer to FIG. 2, which shows a preferred embodiment of the image monitoring system of the present invention, comprising a driving vehicle 1, a plurality of image capture units 2, a processing module 3 and a display 4. The plurality of image capture units 2 are disposed at different positions of the driving vehicle 1 to obtain a plurality of captured images from the driving vehicle 1 in a plurality of directions toward the periphery. The processing module 3 is respectively coupled to each of the image capture units 2 and the display 4 to execute the adaptive image monitoring method and the region of interest updating method of the present invention.

[0057] Please refer to FIG. 3 , which shows the steps of the adaptive image monitoring method of the present invention, including an image capture step S1 , a monitoring image synthesis step S2 , an object recognition step S3 , an object evaluation score calculation step S4 , a region of interest generation step S5 , an optional stereo image generation step S51 and an optional region of interest update step S6 .

[0058] In particular, the processing module 3 includes a processor and a corresponding non-volatile memory, and the processing module 3 has a pre-established image synthesis rule, an object recognition model 31, an object scoring rule, a focus region generation rule, a stereo image generation rule, and an optional focus region update rule; the adaptive image monitoring method of the present invention can be implemented through the rules of the processing module 3 and the object recognition model 31. In other words, the processing module 3 can be regarded as a computer, and the adaptive image monitoring method of the present invention is to perform the following steps through the computer.

[0059] In the image capturing step S1 , a plurality of captured images are acquired from the driving vehicle 1 in a plurality of directions toward the periphery. In particular, the plurality of captured images are acquired through the plurality of image capturing units 2 and transmitted to the processing module 3 .

[0060] In the monitoring image synthesis step S2, each of the captured images is made to have the same coordinate system to synthesize a monitoring image MI (as shown in FIG. 4); in particular, the pixels defined as the same position in each of the captured images are overlapped to synthesize the monitoring image MI. In particular, the method of synthesizing the monitoring image MI, including unifying each of the captured images into the same coordinate system and overlapping each of the captured images, is implemented through the image synthesis rule of the processing module 3. Optionally, the monitoring image MI can be a panoramic image. Optionally, a contour / graphic corresponding to the driving vehicle 1 is presented in the monitoring image MI.

[0061] In the object recognition step S3, at least one object O is identified from the surveillance image MI. In particular, the category and / or position (corresponding to the coordinates in a preset coordinate system) corresponding to each of the objects O is identified. In one example, the position of the identified object O is defined by the center point of the object O, and the center point specifically refers to the centroid of the outline of the object O; however, the definition of the position is not limited to this, and for example, there may be a corresponding preset definition according to the category of the object O. It should be noted that the aforementioned method of identifying an object is implemented through the object recognition model 31 described in the processing module 3, and the object recognition model 31 may be implemented through technologies such as object detection and / or semantic segmentation, and is understandable to those with ordinary knowledge in the technical field of the present invention, so it is not repeated here.

[0062] In particular, the categories of objects O that can be identified in the example of the present invention include parking spaces, cones, dividers, curbs, telephone poles, walls, railings, cars, motorcycles, bicycles, pedestrians (which can be further classified into adults and children), etc., but are not limited to these and may include other categories of objects O of interest.

[0063] In the object evaluation score calculation step S4, an evaluation score of each of the at least one object O is calculated; in particular, a corresponding object category score is given according to the category of each of the objects O, and the evaluation score is preferably calculated according to a relationship between a relative position of each of the objects O and a reference point R and the object category score. Optionally, the reference point R may be the vehicle center 1C of the driving vehicle 1.

[0064] In detail, the evaluation score can be calculated according to the object scoring rule in the processing module 3, and the monitoring image MI is established by a two-dimensional coordinate system and has corresponding x-axis and y-axis; in particular, the y-axis is parallel to the center line of the front and rear of the driving vehicle 1, and the x-axis and the y-axis are orthogonal. In particular, the x-axis and the y-axis are based on the reference point R as the origin, and the direction of the positive y-axis is toward the front of the vehicle.

[0065] In one example, the evaluation score in the object scoring rule may be calculated by the following formula:

[0066] Sf=W1.

[0067] Wherein, Sf represents the final calculated value of the evaluation score; W1 represents a preset object category weight of the category corresponding to the object O.

[0068] In another example, the evaluation score in the object scoring rule may be calculated by the following formula:

[0069]

[0070] Wherein, d represents a distance between the object O and the reference point R; and c is a positive constant.

[0071] In another example, unlike the above evaluation score which is calculated by the inverse of the distance between the object O and the reference point R, the evaluation score in the object scoring rule may be calculated by the following formula:

[0072] Sf=W1×dR.

[0073] Among them, dR represents a distance grade score corresponding to a distance (d) between the object O and the reference point R; multiple distance intervals are divided according to multiple distance ranges, and when the maximum value of the distance interval corresponding to the distance is smaller, the corresponding distance grade score is larger.

[0074] For example, the distance rating score (dR) may be defined as shown in Table 1 below.

[0075] Table 1: An example of distance rating scores (dR).

[0076] Distance interval Rating score (dR) Distance (d) < 4m 2 4m≤Distance(d)<∞ 1

[0077] In another example, the evaluation score in the object scoring rule may be calculated by the following formula:

[0078]

[0079] Wherein, θ represents the angle between the line segment between the object O and the reference point R and the y-axis. In particular, the angle is the angle between the line segment between the object O and the reference point R and the positive y-axis.

[0080] In another example, the evaluation score in the object scoring rule may be calculated by the following formula:

[0081] Sf=|cosθ|×W1×dR.

[0082] It should be noted that the calculation method of cosθ used in the present invention, especially taking the vehicle center 1C of the driving vehicle 1 as the reference point R, and multiplying the distance or distance level by cosθ, assigns an evaluation score that has the characteristic of considering the risk of the driving vehicle 1 colliding with the corresponding object in the straight forward or straight backward direction.

[0083] In another example, the evaluation score in the object scoring rule may be calculated by the following formula:

[0084]

[0085] Wherein, S1 represents an initial score; S2 represents an intermediate score; W2 represents a weight value of a moving direction, and is greater than 1, preferably 3. It should be noted that the formula of this example has the characteristic of considering whether the driving vehicle 1 will approach the object O in the direction of travel; therefore, after obtaining the initial score (S1), it is considered whether the object O is located near the front side or the rear side of the driving vehicle 1, and the direction of travel of the driving vehicle 1 is considered to determine whether to multiply the initial score (S1) by negative 1 to calculate the intermediate score (S2); then, if the intermediate score (S2) is positive, it means that the reference point R of the driving vehicle 1 will approach the object O in the current direction of travel, and then multiply it by a travel direction weight value (W2) greater than 1 to increase the evaluation score (Sf) of the corresponding object O; if the intermediate score (S2) is negative, it means that the reference point R of the driving vehicle 1 will move away from the object O in the current direction of travel, and then multiply it by negative 1 to obtain the evaluation score (Sf) of the corresponding object O.

[0086] In another example, the evaluation score may also be calculated by considering the distance rating score (dR), and the evaluation score in the object scoring rule may be calculated by the following formula:

[0087] S1=cosθ×W1×dR;

[0088]

[0089]

[0090] In particular, the object category weights of various categories of the aforementioned object O can be shown in Table 2 below.

[0091] Table 2: Object category weights for various categories.

[0092] Type of object Object category weight (W1) Parking space 1 Triangular pyramid 2 Divider rod 2 Roadside 3 Telephone pole 5 Wall 8 railing 8 car 13 locomotive 21 bicycle 34 pedestrian 55

[0093] Different from the example in Table 2, in another example, the “pedestrian” category can be further divided into adults and children, and the corresponding object category weights are 34 and 55.

[0094] It should be noted that the contents of "category" and "object category weight" of object O described in the full text of the present invention are merely illustrative of the technology used to implement the present invention, and the present invention is not limited thereto; in other examples, different categories of objects O can be added according to actual needs, and the object category weights of each category can be defined based on considerations such as safety, risk, severity of injury, degree of property loss, etc.

[0095] Next, please refer to Figures 4 to 7 for explanation of the region of interest generation step S5 of the present invention; and in order to more clearly explain the technical content of the present invention, Figures 4 to 7 show that the monitoring image MI contains multiple objects O, and each object O is numbered as O1 to O6, and the evaluation score (Sf) of each object O is marked in the figure, and the object O corresponding to the largest of the evaluation scores is defined as a maximum score object. Among them, Figure 4 shows the state of only having the monitoring image MI (the region of interest IZ has not been generated yet). Figure 5 shows a region of interest IZ generated by the region of interest generation step S5 proposed by the present invention, the object with the maximum score in the region of interest IZ is object O6, and the sum of the evaluation scores of the objects O1, O2, O5, and O6 contained in the region of interest IZ is 110. FIG. 6 shows that although the object with the maximum score in a sub-region SA is object O6, the sum of the evaluation scores of the objects O1, O3, O4, and O6 contained in the sub-region SA is 95. FIG. 7 shows that the conventional focus region IZ' contains objects O1, O3, and O4, but does not contain object O6 with the maximum score, and the sum of the evaluation scores is 35.

[0096] It is understandable that the monitoring image MI has a preset monitoring image size, and each of the sub-areas SA and the focus area IZ has a specific area image size that is smaller than the preset monitoring image size.

[0097] In detail, in the region of interest generation step S5, the region of interest IZ can be determined according to the region of interest generation rule in the processing module 3. The region of interest generation rule is defined as follows: an area of ​​interest IZ is determined from one of the multiple sub-areas SA in the monitoring image MI, and the region of interest IZ has the maximum score object; the sum of the evaluation scores of the objects O in the region of interest IZ is not less than the sum of the evaluation scores of the objects O in the other sub-areas SA containing the maximum score object. Each of the multiple sub-areas SA and the region of interest IZ preferably includes the reference point R, and more preferably includes the complete outline of the driving vehicle 1.

[0098] After the processing module 3 generates the area of ​​interest IZ, the display 4 receives and presents information of the area of ​​interest IZ.

[0099] Optionally, after the focus area generating step S5 determines the focus area IZ, the processing module 3 can perform the stereoscopic image generating step S51 according to the stereoscopic image generating rule to convert the two-dimensional image of the focus area IZ into a stereoscopic image. In the stereoscopic image generating rule, as shown in Figures 8 to 10, a vehicle center 1C of the driving vehicle 1 is used as the reference point R; a deviation direction BD is defined by connecting the vehicle center 1C in the monitoring image MI and a region center IZC of the focus area IZ, and a deviation angle θ is defined between the deviation direction BD and the positive y axis. B ; and at the offset angle θ B The vehicle center 1C is rotated 180 degrees clockwise to define a conversion angle θ T ; converting the monitoring image MI from a two-dimensional coordinate system image to a three-dimensional coordinate system image; the three-dimensional coordinate system has corresponding x-axis, y-axis and z-axis, and the three-dimensional coordinate system and the two-dimensional coordinate system have the same configuration in the x-axis and y-axis; the image of the three-dimensional coordinate system is based on the conversion angle θ T A perspective (clockwise rotation based on the positive y axis) is generated by observing the vehicle center 1C from the edge of the image of the three-dimensional coordinate system at a height h raised on the z axis. In order to facilitate the presentation of the viewing angle, the conversion angle θ T The "eye" patterns drawn in FIGS. 8 and 9 show the transformed viewing angle OD when viewed from the direction of the vehicle.

[0100] Optionally, after generating the area of ​​interest IZ, the processing module 3 may perform the area of ​​interest updating step S6 according to the area of ​​interest updating rule. In the area of ​​interest updating rule, as shown in FIGS. 11 to 14, a preset area PA is defined in the area of ​​interest IZ, and the range of the preset area PA is smaller than the range of the area of ​​interest IZ; when the vehicle center 1C of the driving vehicle 1 moves beyond the preset area PA, the image capturing step S1, the monitoring image synthesis step S2, the object recognition step S3, the object evaluation score calculation step S4 and the area of ​​interest generating step S5 are repeated to generate an updated area of ​​interest IZ.

[0101] In one example, as shown in FIGS. 11 to 13, the preset area PA is formed by shrinking the edge of the focus area IZ inwardly to a region surrounded by a shrinkage ratio, and the shrinkage ratio can be 0.3 to 0.8, preferably 0.5. It should be noted that the preset area PA in FIGS. 11 to 13 of the present invention is generated by the shrinkage ratio of 0.5, but the present invention is not limited thereto.

[0102] In another example, as shown in FIG. 14, the preset area PA may be formed by an area surrounded by an expansion ratio of the body contour of the driving vehicle 1, and the expansion ratio may be 1.5 to 3, preferably 2. It should be noted that the preset area PA in FIG. 14 of the present invention is generated by the expansion ratio of 2, but the present invention is not limited thereto. In addition, in this example, when the body contour of the driving vehicle 1 exceeds the range of the area of ​​interest IZ, the area of ​​interest update step S6 is executed; in other words, the preset area PA may be formed by an area surrounded by an expansion ratio of the body contour of the driving vehicle 1 in the area of ​​interest IZ.

[0103] It should be noted that the adaptive image monitoring method and the region of interest updating method thereof of the present invention are particularly suitable for the driving vehicle 1 in a low-speed driving state, providing the most valuable intelligence in the environment (objects with the highest evaluation scores and multiple objects with the highest sum of evaluation scores), and can particularly avoid harming objects corresponding to living individuals (a higher object category weight can be set for the category of living objects), or avoiding the highest risk of property loss (a higher object category weight can be set for the category of high-value objects).

[0104] In summary, the adaptive image monitoring method of the present invention provides information that the driver needs to pay special attention to in the current environment by making the focus area have the maximum score object, and in the case of having multiple objects, the sum of the evaluation scores of each object in the focus area is not less than the sum of the evaluation scores of each object in other sub-areas containing the same maximum score object, thereby improving driving safety. In addition, through the observation angle formed by the conversion angle defined by the vehicle center and the area center of the focus area, the monitoring image can be converted from a two-dimensional image to a three-dimensional image to present the object that should be paid attention to, and provide the driver with more intuitive visual information to enhance the visual presence, sense of reality and sense of connection, thereby improving driving safety. In addition, the focus area update method of the present invention defines a preset area in the focus area, and updates the focus area when the vehicle center of the driving vehicle moves beyond the preset area, so as to avoid frequent updates of the focus area and update the focus area in a timely manner to obtain the latest information that should be paid attention to.

[0105] Although the present invention has been disclosed using the above preferred embodiments, they are not intended to limit the present invention. Any person skilled in the art can make various changes and modifications to the above embodiments without departing from the spirit and scope of the present invention, and the technical scope protected by the present invention is still within the scope of the present invention. Therefore, the protection scope of the present invention should include all changes within the meaning and equivalent scope of the attached patent application. In addition, when the above several embodiments can be combined, the present invention includes any combination of implementations.

Claims

1. An adaptive image monitoring method, characterized in that: The steps of performing the steps on a computer include: Acquire a plurality of captured images from a driving vehicle in a plurality of directions of the periphery; Make each of the captured images have the same coordinate system to synthesize a monitoring image; Identify at least one object and its corresponding category from the surveillance image; Calculating an evaluation score for each of the at least one object, and defining an object corresponding to the largest evaluation score as a maximum score object; and A focus area is determined from one of the multiple sub-areas in the surveillance image, and the focus area has the object with the maximum score; the sum of the evaluation scores of each object in the focus area is not less than the sum of the evaluation scores of each object in the other sub-areas including the object with the maximum score.

2. The adaptive image monitoring method according to claim 1, characterized in that: A reference point is defined from a preset position corresponding to the driving vehicle in the monitoring image so that the focus area includes the reference point.

3. The adaptive image monitoring method according to claim 2, characterized in that: The reference point is a vehicle center of the driving vehicle presented in the monitoring image.

4. The adaptive image monitoring method according to claim 2, characterized in that: The focus area includes a complete outline of the driving vehicle in the monitoring image.

5. The adaptive image monitoring method according to claim 2, characterized in that: When at least one object is identified, a category of each of the objects is obtained, and the evaluation score of each of the objects is calculated according to the corresponding category.

6. The adaptive image monitoring method according to claim 2, characterized in that: When at least one object is identified, a category of each of the objects is obtained, and a relative position of each of the objects relative to the reference point is obtained; the evaluation score of each of the objects is calculated based on a relationship between the corresponding category and the relative position.

7. The adaptive image monitoring method according to claim 6, characterized in that: The evaluation score is calculated by the following formula: Wherein, Sf represents the final calculated value of the evaluation score; W1 represents a preset object category weight of the category corresponding to the object; d represents a distance between the object and the reference point; and c is a positive constant.

8. The adaptive image monitoring method according to claim 6, characterized in that: The evaluation score is calculated by the following formula: Sf = W1 × dR; Wherein, Sf represents the final calculated value of the evaluation score; W1 represents a preset object category weight of the category corresponding to the object; dR represents a distance level score corresponding to a distance between the object and the reference point; wherein, multiple distance intervals are divided according to multiple distance ranges, and the smaller the maximum value of the distance interval corresponding to the distance, the larger the corresponding distance level score.

9. The adaptive image monitoring method according to claim 6, characterized in that: The monitoring image is established by a two-dimensional coordinate system with corresponding x-axis and y-axis, and the evaluation score is calculated by the following formula: Among them, Sf represents the final calculated value of the evaluation score; θ represents the angle between the line segment of the object and the reference point and the y-axis; W1 represents a preset object category weight of the category corresponding to the object; d represents a distance between the object and the reference point; c is a positive constant.

10. The adaptive image monitoring method according to claim 6, characterized in that: The monitoring image is established by a two-dimensional coordinate system with corresponding x-axis and y-axis, and the evaluation score is calculated by the following formula: Among them, S1 represents an initial score; θ represents an angle between the line segment of the object and the reference point and the y-axis; W1 represents a preset object category weight of the category corresponding to the object; d represents a distance between the object and the reference point; c is a positive constant; S2 represents an intermediate score; Sf represents the evaluation score; W2 represents a weight value of a moving direction, and is greater than 1.

11. The adaptive image monitoring method according to claim 10, characterized in that: The traveling direction weight direction value is 3.

12. The adaptive image monitoring method according to claim 6, characterized in that: The monitoring image is established by a two-dimensional coordinate system with corresponding x-axis and y-axis, and the evaluation score is calculated by the following formula: Sf = |cosθ| × W1 × dR; Wherein, Sf represents the final calculated value of the evaluation score; θ represents an angle between the line segment of the object and the reference point and the y-axis; W1 represents a preset object category weight of the category corresponding to the object; dR represents a distance grade score corresponding to a distance between the object and the reference point; wherein, multiple distance intervals are divided according to multiple distance ranges, and the smaller the maximum value of the distance interval corresponding to the distance, the larger the corresponding distance grade score.

13. The adaptive image monitoring method according to claim 6, characterized in that: The monitoring image is established by a two-dimensional coordinate system with corresponding x-axis and y-axis, and the evaluation score is calculated by the following formula: S1=cosθ×W1×dR; Among them, S1 represents an initial score; θ represents an angle between the line segment of the object and the reference point and the y-axis; W1 represents a preset object category weight of the category corresponding to the object; dR represents a distance level score corresponding to a distance between the object and the reference point; wherein, multiple distance intervals are divided according to multiple distance ranges, and the smaller the maximum value of the distance interval corresponding to the distance is, the larger the corresponding distance level score is; S2 represents an intermediate score; Sf represents the evaluation score; W2 represents a weight value of a moving direction, and is greater than 1.

14. The adaptive image monitoring method according to claim 13, characterized in that: The weight value of the traveling direction is 3.

15. An adaptive image monitoring method according to any one of claims 1 to 14, characterized in that: The monitoring image is established by a two-dimensional coordinate system and has corresponding x-axis and y-axis; after determining the area of ​​interest, a vehicle center of the driving vehicle is used as the reference point; a deviation direction is defined by a line connecting the vehicle center in the monitoring image and an area center of the area of ​​interest; a deviation angle is defined between the deviation direction and the positive y-axis; and a conversion angle is defined by the clockwise rotation of the vehicle center by the deviation angle; the monitoring image is converted from an image in the two-dimensional coordinate system to an image in the three-dimensional coordinate system; the three-dimensional coordinate system has corresponding x-axis, y-axis and z-axis, and the three-dimensional coordinate system and the two-dimensional coordinate system have the same configuration on the x-axis and y-axis; the image of the three-dimensional coordinate system is generated by observing toward the center of the vehicle from a perspective of the conversion angle, from a height raised on the z-axis from the boundary of the image of the three-dimensional coordinate system.

16. A method for updating a region of interest, after applying an adaptive video monitoring method according to any one of claims 1 to 15 to determine a region of interest, characterized in that: Include: Define a preset area within the region of interest, and the range of the preset area is smaller than the range of the region of interest; and When the vehicle center of the driving vehicle moves beyond the preset area, the adaptive image monitoring method is executed again to generate an updated area of ​​interest.

17. A method for updating a region of interest as claimed in claim 16, characterized in that: The preset area is formed by shrinking the edge of the focus area inward to form an area surrounded by a shrinkage ratio.

18. A method for updating a region of interest as claimed in claim 17, characterized in that: The shrinkage ratio is 0.

5.

19. A method for updating a region of interest as claimed in claim 16, characterized in that: The preset area is formed by an area surrounded by the body contour of the driving vehicle expanding outward by an expansion ratio.

20. A method for updating a region of interest as claimed in claim 19, characterized in that: The expansion ratio is 2.

Images