A collision warning method, device, vehicle, equipment and storage medium

Abstract

The present invention discloses a collision warning method, device, vehicle, equipment and storage medium. At each detection time point, a disparity map is synthesized based on images collected by camera devices arranged on both sides of the vehicle; the disparity map is preprocessed; the detection result of a target object within a preset range of the vehicle is determined based on the disparity map; and a collision warning is issued based on the target object detection result. The technical solution of the embodiment of the present invention synthesizes the images collected by the camera device of the vehicle into a disparity map, preprocesses the disparity map, detects the target object within a preset range around the vehicle based on the disparity map, and issues a collision warning based on the target object detection result, which solves the problem that the driver needs to observe the situation on the side and rear of the vehicle and assess the dangerous situation by himself, and can assist the user to avoid safety hazards on both sides and rear of the vehicle.

Description

Technical Field

[0001] The present invention relates to the field of vehicle safety technology, and in particular to a collision warning method, device, vehicle, equipment and storage medium. Background Art

[0002] As the number of vehicles increases, the probability of traffic accidents is also increasing, especially in the blind spots on both sides and rear of the vehicle. It is difficult for the driver to judge the potential dangers on both sides and rear of the vehicle by observation, such as vehicles or pedestrians appearing from the side and rear.

[0003] In order to avoid possible dangerous situations on both sides of the vehicle as much as possible, electronic rear-view mirrors are installed on both sides of the vehicle. The electronic rear-view mirrors provide image feedback of the vehicle's surroundings to the driver. The driver can see the blind spots on the side or rear of the vehicle based on the images provided by the electronic rear-view mirrors.

[0004] However, this method still requires the driver to assess the dangerous situation by himself, and may still encounter dangerous accidents due to negligence in observation, inaccurate distance assessment due to lack of experience, etc. Therefore, a method is needed to better assist users in avoiding safety hazards on both sides and behind the vehicle. Summary of the invention

[0005] The present invention provides a collision warning method, device, vehicle, equipment and storage medium, which can assist users in avoiding safety hazards on both sides and rear of the vehicle.

[0006] According to one aspect of the present invention, a collision warning method is provided, comprising:

[0007] At each detection time point, a disparity map is synthesized based on images collected by cameras arranged on both sides of the vehicle;

[0008] Preprocessing the disparity map;

[0009] Determining a target object detection result within a preset range of the vehicle according to the disparity map;

[0010] A collision warning is performed according to the target object detection result.

[0011] Optionally, when the preprocessing is median filtering, the preprocessing of the disparity map includes:

[0012] Dividing the disparity map into a preset number of filtering sub-regions;

[0013] Determine the median pixel value of all pixels in each of the filtering sub-areas;

[0014] The pixel value of the pixel in each of the filtering sub-areas is replaced by the median pixel value.

[0015] Optionally, when the preprocessing is edge processing, the preprocessing of the disparity map includes:

[0016] Determining at least two grayscale value levels to be divided in the disparity map;

[0017] Determining a maximum inter-class variance value of the disparity map;

[0018] Determine the value range of each grayscale value level according to the maximum inter-class variance value;

[0019] The grayscale value levels are divided according to the value range, and edge processing is performed on each grayscale value level.

[0020] Optionally, determining the target object detection result within the preset range of the vehicle according to the disparity map includes:

[0021] Determining the target object included in the disparity map;

[0022] Determining the relative position and relative speed of the target object and the vehicle;

[0023] The relative position and the relative speed of the target object are used as the target object detection result.

[0024] Optionally, determining the relative position and relative speed of the target object and the vehicle includes:

[0025] According to the static feature points of the target object;

[0026] Determine the current coordinate position of the static feature point in the preset coordinate system of the disparity map;

[0027] Determining the relative position according to the fixed coordinate position of the vehicle in the coordinate system and the current coordinate position;

[0028] Obtain historical disparity maps at historical detection time points;

[0029] Determine the historical coordinate position of the static feature point of the target object in the historical disparity map;

[0030] The relative speed of the target object is determined according to the historical coordinate position and the current coordinate position.

[0031] Optionally, performing a collision warning according to the target object detection result includes:

[0032] According to the relative position and the relative speed of the target object, and according to the vehicle state of the vehicle, determining an estimated time for the target object to reach a preset warning range around the vehicle;

[0033] At the estimated time, a collision warning is performed according to the vehicle status.

[0034] Optionally, when the vehicle state is stopped, performing a collision warning according to the vehicle state at the estimated time includes:

[0035] determining a target door of the vehicle according to the relative position;

[0036] determining whether there is an operation on the target door at the estimated time point;

[0037] When there is no operation on the target door, generating warning information to warn the driver;

[0038] When the target door is operated, the warning information is generated to warn the driver and lock the opening function of the target door;

[0039] Optionally, when the vehicle state is driving, performing a collision warning according to the vehicle state at the estimated time includes:

[0040] determining a target direction according to the relative position;

[0041] determining whether there is a first operation action of operating the turn signal toward the target direction or a second operation action of turning the steering wheel toward the target direction at the estimated time point;

[0042] When the first operation action or the second operation action does not exist, generating warning information to warn the driver;

[0043] When the first operation action occurs, generating a voice prompt message to warn the driver;

[0044] When the second operation action occurs, the steering wheel is restricted from rotating in the target direction.

[0045] According to another aspect of the present invention, there is provided a collision warning device, comprising:

[0046] A disparity map synthesis unit, used to synthesize a disparity map based on images collected by camera devices arranged on both sides of the vehicle at each detection time point;

[0047] A disparity map preprocessing unit, used for preprocessing the disparity map;

[0048] A target object detection result determination unit, configured to determine a target object detection result within a preset range of the vehicle according to the disparity map;

[0049] The target object detection result processing unit is used to issue a collision warning according to the target object detection result.

[0050] According to another aspect of the present invention, a vehicle is provided, comprising: a collision warning device and a vehicle body, wherein the collision warning device is mounted on and communicatively connected to the vehicle body.

[0051] The collision warning device is used to execute the collision warning method described in any embodiment of the present invention.

[0052] According to another aspect of the present invention, there is provided a vehicle, comprising: a collision warning device and a vehicle body, wherein the collision warning device is communicatively connected with the vehicle body;

[0053] The collision warning device is used to execute the collision warning method described in any embodiment of the present invention.

[0054] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0055] at least one processor; and

[0056] a memory communicatively connected to the at least one processor; wherein,

[0057] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the collision warning method described in any embodiment of the present invention.

[0058] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the collision warning method described in any embodiment of the present invention when executed.

[0059] The technical solution of the embodiment of the present invention is to synthesize the images captured by the vehicle's camera device into a disparity map, pre-process the disparity map, detect targets within a preset range around the vehicle according to the disparity map, and issue a collision warning based on the target object detection result, thereby solving the problem that the driver needs to observe the situation on the side and rear of the vehicle and assess the dangerous situation by himself, and can help users avoid safety hazards on the sides and rear of the vehicle.

[0060] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 A flowchart of a collision warning method provided in Embodiment 1 of the present invention;

[0063] Figure 2 A flowchart of a disparity map preprocessing method provided in Embodiment 2 of the present invention;

[0064] Figure 3 A flowchart of a disparity map preprocessing method provided in Embodiment 2 of the present invention;

[0065] Figure 4 A flowchart of a method for determining a target object detection result provided in Embodiment 3 of the present invention;

[0066] Figure 5 A flowchart of a collision warning method provided in Embodiment 4 of the present invention;

[0067] Figure 6 A schematic diagram of the structure of a collision warning device provided in Embodiment 5 of the present invention;

[0068] Figure 7 A schematic diagram of the structure of a vehicle provided in Embodiment 6 of the present invention;

[0069] Figure 8 It is a structural schematic diagram of an electronic device for implementing the collision warning method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0071] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0072] Embodiment 1

[0073] Figure 1 This is a flowchart of a collision warning method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case where a target object behind a vehicle is identified and a warning is issued. The method can be executed by a collision warning device, which can be implemented in the form of hardware and / or software and can be configured in a vehicle. Figure 1 As shown, the method includes:

[0074] S110 . At each detection time point, synthesize a disparity map based on images collected by cameras disposed on both sides of the vehicle.

[0075] Among them, the camera device refers to a device installed on both sides of the vehicle for acquiring video or images of the vehicle's surroundings using imaging technologies such as CMOS. The camera device can be respectively set on the vehicle's left and right rearview mirrors, left and right door panels, and other locations.

[0076] The disparity map refers to an image formed from the images collected by the cameras on both sides of the vehicle, with any one of the images as the reference, and its size is the size of the reference image, and the element value is the disparity value. The disparity map contains the distance information of the scene. The disparity map can be calculated from the images collected by the cameras on both sides of the vehicle that constitute the binocular camera.

[0077] S120: Preprocess the disparity map.

[0078] Among them, preprocessing is used to remove relevant factors in the disparity map that will affect subsequent detection results, eliminate irrelevant information in the image, restore useful real information, enhance the detectability of relevant information and simplify data to the maximum extent, thereby improving the reliability of feature extraction, image segmentation, matching and recognition. For example, through median filtering, noise is suppressed based on sorting statistics theory, and each pixel value of an image area is extracted, and each pixel value is arranged according to size, and the median of the sequence is taken to replace the original pixel value. The median filter will exclude values ​​that are too large or too small, thereby eliminating isolated noise points.

[0079] S130 , determining a target object detection result within a preset range of the vehicle according to the disparity map.

[0080] The preset range refers to the range where there is a risk of collision between the target and the vehicle. The preset range can be set according to the size of the vehicle, the width of the vehicle including the door when the vehicle door is open, and other indicators. The target detection result includes multiple indicators of the target obtained through the disparity map. Since the disparity map can reflect the distance, the above includes but is not limited to the position of the target within the preset range, the relative speed of the target within a certain period of time, etc.

[0081] S140: Perform a collision warning according to the target object detection result.

[0082] Among them, if the target object detection result indicates that the target object has a risk of colliding with the vehicle, a collision warning will be automatically issued to prompt the driver to pay attention to the target object and deal with it in time, and if necessary, some functions of the vehicle will be restricted, such as opening the door and changing lanes of the vehicle.

[0083] The technical solution of the embodiment of the present invention is to synthesize the images captured by the vehicle's camera device into a disparity map, pre-process the disparity map, detect targets within a preset range around the vehicle according to the disparity map, and issue a collision warning based on the target object detection result, thereby solving the problem that the driver needs to observe the situation on the side and rear of the vehicle and assess the dangerous situation by himself, and can help users avoid safety hazards on the sides and rear of the vehicle.

[0084] Embodiment 2

[0085] Figure 2 Flow chart of a disparity map preprocessing method provided by Embodiment 2 of the present invention. This embodiment is optimized on the basis of the above embodiment. Figure 2 As shown, the method includes:

[0086] S210: Divide the disparity map into a preset number of filtering sub-regions.

[0087] Median filtering is an image processing technology based on sorting statistics theory to suppress noise. Its principle is to extract the pixel values ​​of an image area, arrange the pixel values ​​in order of size, and take the median of the sequence to replace the original pixel value. Median filtering will exclude values ​​that are too large or too small, thereby eliminating isolated noise points.

[0088] After obtaining the disparity map, the disparity map is divided into a plurality of filtering sub-regions, and each filtering sub-region is subjected to median filtering to eliminate noise points.

[0089] S220: Determine the median pixel value of all pixels in each of the filtering sub-regions.

[0090] S230: Replace the pixel value of the pixel in each of the filtering sub-areas with the median pixel value.

[0091] For each filter sub-region, the grayscale values ​​of n pixels in the filter sub-region are determined, assuming that they are X1, X2, X3, X4…X n , these n grayscale values ​​are arranged in descending order, and we get X'1, X'2, X'3, X'4…X' n , find the median Y of n gray values:

[0092] When n is an odd number:

[0093] When n is an even number:

[0094] The grayscale values ​​of the pixels in the filter sub-area are replaced with Y, and the median filter is completed. The median filter automatically removes values ​​that are very different from those in the neighborhood, and can effectively filter out speckle noise and salt and pepper noise in the image. At the same time, the edges will not be blurred during filtering.

[0095] Figure 3 Flow chart of a disparity map preprocessing method provided by Embodiment 2 of the present invention. This embodiment is optimized on the basis of the above embodiment. Figure 3 As shown, the method includes:

[0096] S310: Determine at least two grayscale value levels to be divided in the disparity map.

[0097] Among them, the preprocessing of the disparity map can be performed by the maximum inter-class variance method for edge processing. The maximum inter-class variance method is often used in image processing to automatically segment images with clustering characteristics. The algorithm assumes that two categories in the image contain two types of clustered pixel values, and calculates the most suitable threshold for separating the two peaks so that the inter-class variance of the two groups of data is maximized. Specifically, according to the Canny operator principle, it can be known that the selection of two thresholds divides the image gradient into three categories: one category is higher than the upper threshold, one category is higher than the lower threshold and less than the upper threshold, and one category is less than the lower threshold. The size of the upper and lower thresholds determines the number of each gradient, that is, the threshold selection of the Canny operator is converted into a gradient classification problem. The reasonable selection of the size of the number of each group of gradients affects the effect of edge processing. In an embodiment of the present invention, the gradient of the gray value of the image can be divided into three categories, and the dichotomy of the maximum inter-class variance method is replaced by the trichotomy, that is, finding two optimal thresholds to divide the overall gradient into three categories. The algorithm finally obtained is called the dual-threshold maximum inter-class variance method based on the gradient histogram.

[0098] S320: Determine a maximum inter-class variance value of the disparity map.

[0099] The grayscale value of the disparity map is divided into L levels {1, 2, 3...L}, where the number of pixels with grayscale level i is n i , 1≤i is less than or equal to L, that is:

[0100]

[0101] Wherein, N is the total number of pixels of the image in the disparity map.

[0102] The image grayscale is divided into three categories C1, C2, and C3 according to the upper and lower thresholds. Let the upper threshold be t h , the lower threshold is t l , C1 is the gray level from 1 to t l , C2 is the gray level t l +1 to t h , C3 is the gray level t h +1 to L. The total number of pixels of C1, C2, and C3 are:

[0103] In the disparity map, the probability P of each gray level i i All P i =n i / N.

[0104] The probabilities of C1, C2, and C3 are: Among them, w1+w2+w3=1.

[0105] The average gray levels of the three categories are:

[0106] Therefore, the average grayscale of the disparity map is:

[0107] The between-class variance is defined as:

[0108] It is deduced that:

[0109] S330. Determine a value range of each grayscale value level according to the maximum inter-class variance value.

[0110] S340: Divide the grayscale value levels according to the value range, and perform edge processing on each grayscale value level.

[0111] According to the calculation formula of between-class variance, since u T and t l ,t h The choice of is irrelevant, so the optimal upper and lower thresholds are obtained by calculating the maximum value of the formula in the brackets, set to

[0112]

[0113] Therefore, the first grayscale C1 is 1 to C3 is To L, C2 is between the above two.

[0114] Embodiment 3

[0115] Figure 4 This is a flow chart of a method for determining a target object detection result provided by Embodiment 3 of the present invention. The present invention further explains and illustrates the above embodiments. Figure 4 As shown, the method includes:

[0116] S410: Determine the target object included in the disparity map.

[0117] The target is usually a key movable object, such as a running pedestrian, a moving vehicle, etc. The target can be determined based on its outline and size in the disparity map. For example, at the same distance, the size of a vehicle is much larger than that of a pedestrian, and the size of the target can be used to determine whether it is a vehicle or a pedestrian.

[0118] S420: Determine the relative position and relative speed between the target object and the vehicle.

[0119] S430: taking the relative position and the relative speed of the target object as the target object detection result.

[0120] Among them, the relative position can be determined based on the distance in the disparity map and the position of the vehicle itself. For example, the center point of the vehicle is used as the origin, the coordinates of the target object are determined based on the distance in the disparity map, and the relative position of the target object is determined based on the relationship between the coordinates and the origin. The relative speed is used to estimate the time taken for the target object to enter the preset range from the relative position, and is used together with the relative position as the target object detection result for risk assessment. For example, when the relative position of the target object is far away from the vehicle, and the relative speed is low, and it is estimated that it will not reach the position of the vehicle in a short time, the danger level is low, and a low-risk collision warning is performed; when the target object will reach the preset range within a certain period of time, the danger level is medium, and a medium-risk collision warning is performed; when the target object will reach the preset range in a short time, the danger level is high, and a high-risk collision warning is performed.

[0121] In the third embodiment of the present invention, determining the relative position between the target object and the vehicle includes:

[0122] According to the static feature points of the target object;

[0123] Determine the current coordinate position of the static feature point in the preset coordinate system of the disparity map;

[0124] The relative position is determined according to the fixed coordinate position of the vehicle in the coordinate system and the current coordinate position.

[0125] After the target object is determined in the disparity map, the static feature points of the target object are determined, and the current coordinate position of the static feature points in the coordinate system of the disparity map is obtained. The relative direction between the target object and the vehicle is determined based on the current coordinate position of the static feature points and the fixed coordinate position of the vehicle in the coordinate system (since the coordinates of the vehicle in the coordinate system are unchanged, its coordinate position is also fixed), and the actual interval distance is determined based on the conversion relationship between the coordinate system and the distance, and the relative position is determined in combination with the relative direction.

[0126] In the third embodiment of the present invention, determining the relative speed between the target object and the vehicle includes:

[0127] Obtain historical disparity maps at historical detection time points;

[0128] Determine the historical coordinate position of the static feature point of the target object in the historical disparity map;

[0129] The relative speed of the target object is determined according to the historical coordinate position and the current coordinate position.

[0130] The historical detection time point can be determined according to the time interval or the number of frames. For example, the coordinate position of the static feature point in the inspection image of two adjacent frames is used to determine the change in its coordinates, and the actual moving distance of the target object is determined according to the corresponding relationship between the coordinate value and the actual distance; the time between two adjacent frames is determined according to the number of frames shot (for example, the interval between two adjacent frames is one thirtieth of a second when 30 frames per second are used, and the relative speed of the target object is determined in combination with the actual moving distance.

[0131] Embodiment 4

[0132] Figure 5 A flowchart of a collision warning method provided in Embodiment 4 of the present invention is shown in FIG. Figure 5 As shown, the method includes:

[0133] S510: Determine an estimated time for the target object to reach a preset warning range around the vehicle according to the relative position and the relative speed of the target object and the vehicle state of the vehicle.

[0134] The vehicle status refers to the current driving status of the vehicle, such as driving or parked. When the vehicle is in driving status, in addition to considering the position and speed of the target object, the vehicle's own driving speed must also be considered when determining the estimated time. Combined with the estimated time for the target object to enter the preset warning range, the estimated time will change when the speed and position of the vehicle change (such as changing lanes). If the vehicle is parked on the side of the road, the estimated time only needs to be calculated based on the position and speed of the target object.

[0135] S520: At the estimated time, a collision warning is performed according to the vehicle status.

[0136] Since the target object will enter the preset warning range near the estimated time, and since the estimated time will vary according to the movement state and relative position of the target object and the vehicle, the estimated time can reflect the time when the distance between the target object and the vehicle is less than a certain range. Therefore, collision warning at the estimated time can more accurately prompt the driver and avoid false alarms caused by various factors.

[0137] In the fourth embodiment of the present invention, when the vehicle state is stopped, performing a collision warning according to the vehicle state at the estimated time includes:

[0138] determining a target door of the vehicle according to the relative position;

[0139] determining whether there is an operation on the target door at the estimated time point;

[0140] When there is no operation on the target door, generating warning information to warn the driver;

[0141] When there is an operation on the target door, the warning information is generated to warn the driver and lock the opening function of the target door.

[0142] Among them, when the vehicle is in a stopped state, that is, when the vehicle is parked, the risk of collision on the side and rear of the vehicle mainly comes from the collision with the vehicle or pedestrian coming from the side of the vehicle when the door is opened. According to the relative position, determine whether the relative position of the target object is on the left or on the right, corresponding to the left door and the right door respectively. At the estimated time point, the door on the other side of the target door is not affected and can be opened normally. When the target object is far away from the vehicle, the target door can also be opened normally. The user can open the door and get off the vehicle through flashing warning lights and voice reminders. If the occupants in the car do not operate the target door, the driver is prompted by warning information that there is a target on the rear side, such as flashing warning lights, and do not operate the door. When the occupants in the car operate the target door, while prompting the occupants through warning information, the opening function of the target door is temporarily locked to prevent the occurrence of dangerous situations. When the target object leaves the preset area, the opening function of the target door can be unlocked.

[0143] In the fourth embodiment of the present invention, when the vehicle state is driving, performing a collision warning according to the vehicle state at the estimated time includes:

[0144] determining a target direction according to the relative position;

[0145] determining whether there is a first operation action of operating the turn signal toward the target direction or a second operation action of turning the steering wheel toward the target direction at the estimated time point;

[0146] When the first operation action or the second operation action does not exist, generating warning information to warn the driver;

[0147] When the first operation action occurs, generating a voice prompt message to warn the driver;

[0148] When the second operation action occurs, the steering wheel is restricted from rotating in the target direction.

[0149] Among them, when the vehicle is driving in the lane, the target object approaches or enters the preset range at the estimated time point. When the driver does not perform any lane change-related operations and the estimated time point has not arrived, the warning light flashes to remind the user that he can change lanes. Determine whether the target object is to the left or right rear of the vehicle through relative position. When the driver needs to change lanes, turn on the turn signal in the lane change direction by operating the lever. When the driver moves the lever to the target direction, the warning light flashes and a voice broadcast reminds the user that it is not appropriate to change lanes at this time. When the driver operates the steering wheel to change lanes in the opposite direction, there is a risk of collision with the target object. At this time, the steering wheel can be locked to temporarily prevent the driver from changing lanes in the target direction.

[0150] Embodiment 5

[0151] Figure 6 This is a schematic diagram of the structure of a collision warning device provided by Embodiment 5 of the present invention. Figure 6 As shown, the device comprises:

[0152] A disparity map synthesis unit 610 is used to synthesize a disparity map according to images collected by cameras arranged on both sides of the vehicle at each detection time point;

[0153] A disparity map preprocessing unit 620, configured to preprocess the disparity map;

[0154] A target object detection result determination unit 630, configured to determine a target object detection result within a preset range of the vehicle according to the disparity map;

[0155] The target object detection result processing unit 640 is used to perform a collision warning according to the target object detection result.

[0156] In the fifth embodiment of the present invention, when the preprocessing is median filtering, the disparity map preprocessing unit 620 is used to perform:

[0157] Dividing the disparity map into a preset number of filtering sub-regions;

[0158] Determine the median pixel value of all pixels in each of the filtering sub-areas;

[0159] The pixel value of the pixel in each of the filtering sub-areas is replaced by the median pixel value.

[0160] In the fifth embodiment of the present invention, when the preprocessing is edge processing, the disparity map preprocessing unit 620 is used to perform:

[0161] Determining at least two grayscale value levels to be divided in the disparity map;

[0162] Determining a maximum inter-class variance value of the disparity map;

[0163] Determine the value range of each grayscale value level according to the maximum inter-class variance value;

[0164] The grayscale value levels are divided according to the value range, and edge processing is performed on each grayscale value level.

[0165] In the fifth embodiment of the present invention, the target detection result determination unit 630 is used to execute:

[0166] Determining the target object included in the disparity map;

[0167] Determining the relative position and relative speed of the target object and the vehicle;

[0168] The relative position and the relative speed of the target object are used as the target object detection result.

[0169] In the fifth embodiment of the present invention, when determining the relative position and relative speed between the target object and the vehicle, the target object detection result determination unit 630 specifically performs:

[0170] According to the static feature points of the target object;

[0171] Determine the current coordinate position of the static feature point in the preset coordinate system of the disparity map;

[0172] Determining the relative position according to the fixed coordinate position of the vehicle in the coordinate system and the current coordinate position;

[0173] Obtain historical disparity maps at historical detection time points;

[0174] Determine the historical coordinate position of the static feature point of the target object in the historical disparity map;

[0175] The relative speed of the target object is determined according to the historical coordinate position and the current coordinate position.

[0176] In the fifth embodiment of the present invention, the target detection result processing unit 640 is used to execute:

[0177] According to the relative position and the relative speed of the target object, and according to the vehicle state of the vehicle, determining an estimated time for the target object to reach a preset warning range around the vehicle;

[0178] At the estimated time, a collision warning is performed according to the vehicle status.

[0179] In the fifth embodiment of the present invention, when the vehicle state is stopped, the target detection result processing unit 640 is used to execute:

[0180] determining a target door of the vehicle according to the relative position;

[0181] determining whether there is an operation on the target door at the estimated time point;

[0182] When there is no operation on the target door, generating warning information to warn the driver;

[0183] When the target door is operated, the warning information is generated to warn the driver and lock the opening function of the target door;

[0184] In the fifth embodiment of the present invention, when the vehicle state is driving, at the estimated time, the target detection result processing unit 640 is used to execute:

[0185] determining a target direction according to the relative position;

[0186] determining whether there is a first operation action of operating the turn signal toward the target direction or a second operation action of turning the steering wheel toward the target direction at the estimated time point;

[0187] When the first operation action or the second operation action does not exist, generating warning information to warn the driver;

[0188] When the first operation action occurs, generating a voice prompt message to warn the driver;

[0189] When the second operation action occurs, the steering wheel is restricted from rotating in the target direction.

[0190] The collision warning device provided in the embodiment of the present invention can execute the collision warning method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0191] Embodiment 6

[0192] Figure 7 This is a schematic diagram of the structure of a vehicle provided in Embodiment 6 of the present invention. Figure 7 As shown, it includes: a collision warning device 710 and a vehicle body 720, wherein the collision warning device 710 is in communication connection with the vehicle body 720;

[0193] The collision warning device 710 is used to execute the collision warning method described in any one of the above embodiments.

[0194] Embodiment 7

[0195] Figure 8A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0196] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0197] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0198] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a collision warning method.

[0199] In some embodiments, the collision warning method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the collision warning method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the collision warning method in any other appropriate manner (e.g., by means of firmware).

[0200] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0201] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0202] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0203] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0204] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0205] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0206] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0207] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A collision warning method, characterized in that: include: At each detection time point, a disparity map is synthesized based on images collected by cameras arranged on both sides of the vehicle; Preprocessing the disparity map; Determining a target object detection result within a preset range of the vehicle according to the disparity map; Providing a collision warning according to the target object detection result; Determining the target object detection result within the preset range of the vehicle according to the disparity map includes: Determining the target object included in the disparity map; Determining the relative position and relative speed of the target object and the vehicle; According to the relative position and the relative speed of the target object as the target object detection result; The determining the relative position and relative speed of the target object and the vehicle comprises: According to the static feature points of the target object; Determine the current coordinate position of the static feature point in the preset coordinate system of the disparity map; Determining the relative position according to the fixed coordinate position of the vehicle in the coordinate system and the current coordinate position; Obtain historical disparity maps at historical detection time points; Determine the historical coordinate position of the static feature point of the target object in the historical disparity map; Determine the relative speed of the target object according to the historical coordinate position and the current coordinate position; Wherein, performing collision warning according to the target object detection result includes: According to the relative position and the relative speed of the target object, and according to the vehicle state of the vehicle, determining an estimated time for the target object to reach a preset warning range around the vehicle; At the estimated time, providing a collision warning according to the vehicle status; When the vehicle state is stopped, performing a collision warning according to the vehicle state at the estimated time includes: determining a target door of the vehicle according to the relative position; determining whether there is an operation on the target door at the estimated time point; When there is no operation on the target door, generating warning information to warn the driver; When there is an operation on the target door, the warning information is generated to warn the driver and lock the opening function of the target door.

2. The method according to claim 1, characterized in that When the preprocessing is median filtering, the preprocessing of the disparity map includes: Dividing the disparity map into a preset number of filtering sub-regions; Determine the median pixel value of all pixels in each of the filtering sub-areas; The pixel value of the pixel in each of the filtering sub-areas is replaced by the median pixel value.

3. The method according to claim 1, characterized in that When the preprocessing is edge processing, the preprocessing of the disparity map includes: Determining at least two grayscale value levels to be divided in the disparity map; Determining a maximum inter-class variance value of the disparity map; Determine the value range of each grayscale value level according to the maximum inter-class variance value; The grayscale value levels are divided according to the value range, and edge processing is performed on each grayscale value level.

4. The method according to claim 1, characterized in that: When the vehicle state is in driving, performing a collision warning according to the vehicle state at the estimated time includes: determining a target direction according to the relative position; determining whether there is a first operation action of operating the turn signal toward the target direction or a second operation action of turning the steering wheel toward the target direction at the estimated time point; When the first operation action or the second operation action does not exist, generating warning information to warn the driver; When the first operation action occurs, generating a voice prompt message to warn the driver; When the second operation action occurs, the steering wheel is restricted from rotating in the target direction.

5. A collision warning device, characterized in that: include: A disparity map synthesis unit, used to synthesize a disparity map based on images collected by camera devices arranged on both sides of the vehicle at each detection time point; A disparity map preprocessing unit, used for preprocessing the disparity map; A target object detection result determination unit, configured to determine a target object detection result within a preset range of the vehicle according to the disparity map; A target object detection result processing unit, used for performing collision warning according to the target object detection result; The target object detection result determination unit is used to perform: determining the target object included in the disparity map; determining the relative position and relative speed of the target object and the vehicle; and using the relative position and relative speed of the target object as the target object detection result; When determining the relative position and relative speed between the target object and the vehicle, the target object detection result determination unit specifically performs: according to the static feature points of the target object; Determine the current coordinate position of the static feature point in the preset coordinate system of the disparity map; determine the relative position according to the fixed coordinate position of the vehicle in the coordinate system and the current coordinate position; obtain the historical disparity map at the historical detection time point; determine the historical coordinate position of the static feature point of the target object in the historical disparity map; determine the relative speed of the target object according to the historical coordinate position and the current coordinate position; The target object detection result processing unit is used to perform: determining an estimated time for the target object to reach a preset warning range around the vehicle according to the relative position and the relative speed of the target object and the vehicle state of the vehicle; At the estimated time, providing a collision warning according to the vehicle status; When the vehicle is stopped, the target object detection result processing unit is used to perform: determining a target door of the vehicle according to the relative position; determining whether there is an operation on the target door at the estimated time point; When there is no operation on the target door, generating warning information to warn the driver; When there is an operation on the target door, the warning information is generated to warn the driver and lock the opening function of the target door.

6. A vehicle, characterized in that: include: A collision warning device and a vehicle body, wherein the collision warning device is communicatively connected with the vehicle body; The collision warning device is used to execute the collision warning method according to any one of claims 1 to 4.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the collision warning method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the collision warning method according to any one of claims 1 to 4 when executed.

Images