Method and device for detecting blind areas on left side and right side of public transport vehicle and electronic equipment

Abstract

The invention discloses a method and device for detecting blind areas on the left side and the right side of a public transport vehicle and electronic equipment. According to the method, blind area images on the left side and the right side of a bus are obtained, detection areas are arranged in the blind area images, the images are detected through a pre-trained YOLOv8-seg-cls segmentation model, a segmentation result is obtained, whether a dangerous target is located in the detection areas or not is judged, and if the dangerous target is located in the detection areas, the dangerous target is detected. Judging whether the dangerous target is located outside any safety segmentation area or not, if the dangerous target is not located outside any safety segmentation area, judging whether the dangerous target is located in the dangerous segmentation area or not, and if the dangerous target is located in the dangerous segmentation area, giving an alarm that the dangerous target exists in the blind area; the area where the obstacle is located is recognized and pre-judged, when the obstacle is located in the safe area, it is determined that vehicle driving is not affected, and alarming is not conducted, so that the accuracy and practicability of alarming are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a method, device and electronic equipment for detecting blind spots on the left and right sides of a public transport vehicle. Background Art

[0002] The blind spot vehicle discern system (BSD) of the automobile safety system uses millimeter-wave radar sensors to monitor the blind spot area behind the vehicle. Once a vehicle is detected in the blind spot, it will alert the driver through warning signals on the vehicle's exterior rearview mirrors, sounds, and other forms of sound and light.

[0003] For BSD technology, most of the existing technologies use radar or multi-sensor fusion technology, and the detection target is all obstacles in the blind spot. There is no distinction between the areas where the obstacles are located, which will cause some false alarms, reducing the practicality and reliability of the above methods. For example, if pedestrians are on both sides of the road and inside the guardrail, if the road and both sides of the road are not distinguished, the existing technology will alarm, but at this time the pedestrians are safe and there is no need to alarm, causing the driver to no longer trust the alarm information, and the practicality is not strong. Summary of the invention

[0004] The purpose of the present invention is to provide a method, device and electronic equipment for detecting blind spots on the left and right sides of a public transport, so as to solve the problem that the existing technology detects obstacles in all blind spots, does not distinguish the areas where the obstacles are located, causes some false alarms, and reduces practicality and reliability.

[0005] In a first aspect, the present invention provides a method for detecting left and right blind spots of a public transport vehicle, comprising:

[0006] Obtain blind spot images on the left and right sides of the bus;

[0007] Setting a detection area in the blind area image;

[0008] The image is detected by a pre-trained YOLOv8-seg-cls segmentation model to obtain a segmentation result, wherein the segmentation result includes a dangerous target, a dangerous segmentation area, and a safe segmentation area, wherein the dangerous segmentation area and the safe segmentation area are located within the detection area, and the dangerous segmentation area and the safe segmentation area are sequentially arranged from the inside to the outside in a direction away from the bus;

[0009] Determining whether the dangerous target is located in the detection area;

[0010] If the dangerous target is not located in the detection area, terminating the detection of the dangerous target;

[0011] If the dangerous target is located in the detection area, determining whether the dangerous target is located outside any safety segmentation area;

[0012] If the dangerous target is located outside any of the safety segmentation areas, terminating the detection of the dangerous target;

[0013] If the dangerous target is not located outside any of the safe segmented areas, determining whether the dangerous target is located in a dangerous segmented area;

[0014] If the dangerous target is not located in the dangerous segmentation area, terminating the detection of the dangerous target;

[0015] If the dangerous target is located in the dangerous segmentation area, an alarm is issued that a dangerous target exists in the blind area.

[0016] Further, determining whether the dangerous target is located in the detection area includes:

[0017] Extract the coordinates (x, y) of the bottom pixel of the dangerous target, the coordinates (x1, y1) of the upper left corner of the detection area, and the coordinates (x2, y2) of the lower right corner of the detection area;

[0018] Calculate whether x1<x<x2, and y1<y<y2;

[0019] If x1<x<x2, and y1<y<y2 are satisfied, it is determined that the dangerous target is located in the detection area;

[0020] If x1<x<x2 and y1<y<y2 are not satisfied, it is determined that the dangerous target is not located in the detection area.

[0021] Further, determining whether the dangerous target is located outside any safety segmentation area includes:

[0022] Extract the coordinates (x, y) of the bottom 5 pixels of the dangerous target and the coordinates x_safe (xi, yi) of the safe segmentation area, i = 1, 2, 3, 4...n;

[0023] For the blind spot image on the left side of the bus, calculate whether there is i that satisfies xi<x, yi<y;

[0024] If there exists i that satisfies xi<x, yi<y, it is determined that the dangerous target is located outside any safe segmentation area;

[0025] If there is no i that satisfies xi<x, yi<y, it is determined that the dangerous target is not located outside any safe segmentation area;

[0026] For the blind spot image on the right side of the bus, calculate whether there is i that satisfies xi>x, yi>y;

[0027] If there exists i that satisfies xi>x, yi>y, it is determined that the dangerous target is located outside any safe segmentation area;

[0028] If there is no i satisfying xi>x, yi>y, it is determined that the dangerous target is not located outside any safe segmentation area.

[0029] Further, determining whether the dangerous target is located in a dangerous segmentation area includes:

[0030] Extract the coordinates of the bottom five pixels of the dangerous target (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5);

[0031] Get the minimum value of the 5 pixel coordinates (x min ,y min ) and the maximum value coordinate (x max ,y max ),in:

[0032] x min =min(x1,x2,x3,x4,x5);

[0033] y min =min(y1,y2,y3,y4,y5);

[0034] x max =max(x1,x2,x3,x4,x5);

[0035] y max =max(y1,y2,y3,y4,y5);

[0036] According to the minimum coordinate (x min ,y min ) and the maximum value coordinate (x max ,y max ), determine the judgment area;

[0037] Counting the number of pixels in the judgment area, checking the number of pixels belonging to the safe segmentation area and the number of pixels belonging to the dangerous segmentation area;

[0038] according to Calculate the r value, where n_safe is the number of pixels in the safe segmentation area, and n_danger is the number of pixels in the dangerous segmentation area;

[0039] Determine whether r is less than k;

[0040] If r≥k, it is determined that the dangerous target is located in a safe segmentation area;

[0041] If r<k, it is determined that the dangerous target is located in the dangerous segmentation area.

[0042] Further, according to the minimum value coordinate (x min ,y min ) and the maximum value coordinate (x max ,y max ), determine the judgment area, including:

[0043] The minimum coordinate (x min ,y min ) is reduced by 10 to get the x value of the upper left corner of the judgment area lh The coordinates of (X1, Y1), X1 = x min -10, Y1 = y min -10;

[0044] The maximum coordinate (x max ,y max ) increases by 10 to get the x value of the lower right corner of the judgment area rl The coordinates of (X2, Y2), X2 = x max +10, Y2=y max +10.

[0045] According to the upper left corner x of the judgment area lh The coordinates (X1, Y1) and the lower right corner x of the judgment area rl The coordinates (X2, Y2) are used to determine the judgment area.

[0046] Further, after setting the detection area in the blind area image, the method further includes:

[0047] The image is detected by a pre-trained YOLOv8-seg-cls segmentation model to obtain weather classification information, where the weather classification information includes normal weather and severe weather;

[0048] Determining whether the weather classification information is severe weather;

[0049] If the weather classification information is severe weather, a reminder message to pay attention to the rearview mirror is issued;

[0050] If the weather classification information is normal weather, the image is detected by using a pre-trained YOLOv8-seg-cls segmentation model to obtain a segmentation result.

[0051] In a second aspect, the present invention provides a device for detecting blind spots on the left and right sides of a public transport vehicle, comprising:

[0052] An acquisition unit, used for acquiring blind spot images on the left and right sides of the bus;

[0053] A setting unit, used for setting a detection area in the blind area image;

[0054] A detection unit is used to detect the image by using a pre-trained YOLOv8-seg-cls segmentation model to obtain a segmentation result, wherein the segmentation result includes a dangerous target, a dangerous segmentation area, and a safe segmentation area, wherein the dangerous segmentation area and the safe segmentation area are located within the detection area, and the dangerous segmentation area and the safe segmentation area are sequentially arranged from the inside to the outside in a direction away from the public transport vehicle;

[0055] A first judging unit, used to judge whether the dangerous target is located in the detection area;

[0056] a termination unit, configured to terminate the detection of the dangerous target when the dangerous target is not located in the detection area;

[0057] A second judgment unit, configured to judge whether the dangerous target is located outside any safety segmentation area when the dangerous target is located in the detection area;

[0058] The termination unit is further used to terminate the detection of the dangerous target when the dangerous target is located outside any safety segmentation area;

[0059] A third judgment unit, configured to judge whether the dangerous target is located in a dangerous segmentation area when the dangerous target is not located outside any of the safe segmentation areas;

[0060] The termination unit is further used to terminate the detection of the dangerous target when the dangerous target is not located in the dangerous segmentation area;

[0061] The alarm unit is used to issue an alarm that a dangerous target exists in the blind area when the dangerous target is located in the dangerous segmentation area.

[0062] In a third aspect, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method described in the first aspect above.

[0063] Beneficial effects of the present invention: The method, device and electronic equipment for detecting blind spots on the left and right sides of a bus provided by the present invention obtain blind spot images on the left and right sides of the bus, set a detection area in the blind spot image, detect the image through a pre-trained YOLOv8-seg-cls segmentation model, obtain a segmentation result, judge whether a dangerous target is located in the detection area, if the dangerous target is located in the detection area, judge whether the dangerous target is located outside any safe segmentation area, if the dangerous target is not located outside any safe segmentation area, judge whether the dangerous target is located in the dangerous segmentation area, and if the dangerous target is located in the dangerous segmentation area, issue an alarm that a dangerous target exists in the blind spot; the present invention identifies and pre-judges the area where the obstacle is located, and when the obstacle is located in the safe area, determines that it will not affect the driving of the vehicle and will not alarm, thereby improving the accuracy and practicality of the alarm itself. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings are only provided for reference and illustration and are not intended to limit the present invention.

[0065] In the accompanying drawings,

[0066] Figure 1 It is a flow chart of the method for detecting blind spots on the left and right sides of a public transport vehicle of the present invention;

[0067] Figure 2 A flow chart for determining whether a dangerous target is within the detection area;

[0068] Figure 3 A flowchart for determining whether a dangerous target is located outside any safe segmentation area;

[0069] Figure 4 A flowchart for determining whether a dangerous target is located in a dangerous segmentation area;

[0070] Figure 5 A flow chart for determining the judgment area;

[0071] Figure 6 A schematic diagram for determining whether a dangerous target is within the detection area;

[0072] Figure 7 A schematic diagram for determining whether a dangerous target is located outside any safety segmentation area;

[0073] Figure 8 A schematic diagram for determining whether a dangerous target is located in a dangerous segmentation area;

[0074] Fig. 9 It is a block diagram of the left and right blind spot detection device for a public transport vehicle of the present invention;

[0075] Fig.10 It is a block diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0076] To further illustrate the technical means and effects of the present invention, the following is a detailed description in conjunction with the preferred embodiments of the present invention and the accompanying drawings.

[0077] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0078] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0079] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or description". Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0080] There are certain blind spots on both sides of the bus. Due to the large body of the bus, the driver may not be able to see the situation on both sides of the bus during driving, especially the rear and right side of the bus, which are prone to become blind spots. When the vehicle is changing lanes, due to the blind spots in the rearview mirror when turning, the driver cannot fully judge the information of the rear vehicle based on the information in the rearview mirror alone. Bad weather such as rain, snow, fog, and hail increases the difficulty of the driver's judgment and increases the risk of collision or scratching when the car changes lanes.

[0081] Most of the existing BSD technologies are based on radar or multi-sensor fusion technology, and an alarm is issued to the driver when an obstacle is detected in the blind spot. However, in many scenarios, there will be facilities such as curbs, isolation belts, fences, and green belts on the edge of the road. Although pedestrians or non-motor vehicles are in the blind spots on both sides of the bus, they are located on the other side of the curbs, isolation belts, fences, green belts, etc., which belong to the safe area. Then, they will not affect the driving of the bus, there is no safety hazard, and there is no need to alarm. However, the existing BSD technology does not distinguish the area where the obstacle is located, so graying often leads to false alarms, affects the driver's judgment, and reduces the practicality and reliability of the existing BSD technology. Therefore, the left and right blind spot detection method for public transportation vehicles proposed by the present invention identifies and pre-judges the area where the obstacle is located. When the obstacle is located in the safe area, it is determined that it will not affect the driving of the vehicle and will not alarm, thereby improving the accuracy and practicality of the alarm itself. The following is a detailed description of the left and right blind spot detection method for public transportation vehicles of the present invention.

[0082] See also Figure 1 , Figure 1 The flowchart of the method for detecting the left and right blind spots of a public transport vehicle of the present invention is provided. The method for detecting the left and right blind spots of a public transport vehicle of the present invention comprises:

[0083] S101, acquiring blind spot images on the left and right sides of the bus.

[0084] Specifically, the present invention collects blind spot images on the left and right sides of the bus by setting monocular cameras on the left and right sides of the bus. The monocular camera is a module composed of one camera and can only capture images from a single perspective. Compared with binocular cameras, monocular cameras have lower costs and simpler designs, and can meet the needs in the scenarios of the present invention. Compared with the solutions based on radar or binocular cameras in the prior art, the use of monocular cameras is low-cost and easy to promote.

[0085] S102: Setting a detection area in the blind area image.

[0086] S103, detecting the image through a pre-trained YOLOv8-seg-cls segmentation model to obtain a segmentation result, wherein the segmentation result includes a dangerous target, a dangerous segmentation area, and a safe segmentation area, wherein the dangerous segmentation area and the safe segmentation area are located within the detection area, and the dangerous segmentation area and the safe segmentation area are arranged in sequence from the inside to the outside in a direction away from the bus.

[0087] YOLOv8 currently supports image classification, object detection, and instance segmentation tasks. It builds on the success of previous YOLO versions and introduces new features and improvements to further improve performance and flexibility. Specific innovations include a new backbone network, a new Ancher-Free detection head, and a new loss function that can run on a variety of hardware platforms from CPU to GPU.

[0088] The general steps for instance segmentation and classification tasks using YOLOv8 are as follows:

[0089] Prepare a dataset: Prepare a labeled dataset containing bounding box and category information of instances. For each instance, specify its category and bounding box coordinates.

[0090] Adjust the model: Use the pre-trained model of YOLOv8 as a starting point and adjust the model's hyperparameters according to the task requirements. For example, adjust the learning rate, batch size, loss function, etc.

[0091] Training model: Use the prepared data set to train the model. During the training process, you can understand the performance and loss of the model by observing the training log.

[0092] Evaluate the model: Use the test dataset to evaluate the trained model to understand the performance and accuracy of the model. Various evaluation metrics can be used to measure the performance of the model, such as accuracy, mAP (mean average precision), etc.

[0093] Apply the model: Deploy the trained model to the target platform or application for instance segmentation and classification tasks. You can use the inference tools or frameworks provided by YOLOv8 to apply the model to real-time images or video streams.

[0094] Specifically, the present invention obtains a model that can segment pedestrians, non-motor vehicles, curbs, roads, isolation belts, and green belts by training the YOLOv8-seg-cls algorithm; the present invention increases the classification ability of the entire image by modifying the YOLOv8-seg model, and adds a classify classification head in the Segment class of YOLOv8-seg, uses the smallest feature array in the input, inputs it into the classification head, obtains the classification information of the image, and adds the loss of the classification head to the loss function, and the loss function adopts bceloss, so as to obtain the YOLOv8-seg-cls segmentation model.

[0095] bceloss stands for Binary CrossEntropy Loss, which is a commonly used loss function mainly used to deal with binary classification problems. The calculation formula of bceloss is "-ylog(y^hat)-(1-y)log(1-y^hat)", where y is the true value and y^hat is the predicted value. When the true value is 0, in order to make the second half smaller, the predicted value y^hat needs to be as close to 0 as possible; when the true value is 1, in order to make the first half smaller, the predicted value y^hat needs to be as close to 1 as possible. This requires that the output must be between 0 and 1, so in order to ensure that the output of the network is between 0 and 1, a Sigmoid is generally added.

[0096] The dangerous targets segmented by the YOLOv8-seg-cls segmentation model include pedestrians, non-motor vehicles, etc. The dangerous segmentation area includes the road, and the safe segmentation area includes the curb, isolation belt, fence, green belt, etc.

[0097] S104, determining whether the dangerous target is located within the detection area.

[0098] Not all dangerous target segmentation objects need to be judged in the safe segmentation area. For the sake of efficiency in calculation, the present invention proposes the following accelerated screening method. The traditional target detection judgment method is generally based on the center point of the box, but it cannot truly feedback the area where the dangerous target is located. The present invention uses the bottom pixel coordinates of the dangerous target. Taking pedestrians as an example, the coordinates of the human feet are used for judgment, that is, to judge whether the bottom pixel of the dangerous target is located in the dangerous segmentation area.

[0099] Specifically, see Figure 2 and Figure 6 , Figure 2 Flow chart for determining whether a dangerous target is within the detection area. Figure 6 A schematic diagram for determining whether a dangerous target is located within a detection area. Determining whether the dangerous target is located within the detection area includes:

[0100] S1041, extract the coordinates (x, y) of the bottom pixel of the dangerous target, the coordinates (x1, y1) of the upper left corner of the detection area, and the coordinates (x2, y2) of the lower right corner of the detection area.

[0101] S1042, calculate whether x1<x<x2, and y1<y<y2 are satisfied.

[0102] S1043: If x1<x<x2, and y1<y<y2 are satisfied, it is determined that the dangerous target is located in the detection area.

[0103] S1044, if x1<x<x2 and y1<y<y2 are not satisfied, it is determined that the dangerous target is not located in the detection area.

[0104] Figure 6 In the figure, the pedestrian is not located in the detection area, but the non-motor vehicle is located in the detection area.

[0105] S105: If the dangerous target is not located in the detection area, terminate the detection of the dangerous target.

[0106] S106: If the dangerous target is located in the detection area, determine whether the dangerous target is located outside any safe segmentation area.

[0107] For the left image of the bus's forward direction, when the dangerous target is located on the right side of any safe segmentation area in the image, the dangerous target is safe. For the right image of the bus's forward direction, when the dangerous target is located on the left side of any safe segmentation area in the image, the dangerous target is safe.

[0108] Specifically, see Figure 3 , Figure 3 A flowchart for determining whether a dangerous target is located outside any safety segmentation area, wherein determining whether the dangerous target is located outside any safety segmentation area includes:

[0109] S1061, extract the coordinates (x, y) of the bottom 5 pixels of the dangerous target and the coordinates x_safe (xi, yi) of the safe segmentation area, i = 1, 2, 3, 4...n.

[0110] S1062, for the blind spot image on the left side of the bus, calculate whether there is i that satisfies xi<x, yi<y;

[0111] If there exists i satisfying xi<x, yi<y, then S1064 is executed to determine that the dangerous target is located outside any safe segmentation area, that is, the dangerous target is located on the right side of any safe segmentation area.

[0112] If there is no i satisfying xi<x, yi<y, then S1065 is executed to determine that the dangerous target is not located outside any safe segmentation area.

[0113] S1063, for the blind spot image on the right side of the bus, calculate whether there is i that satisfies xi>x, yi>y;

[0114] If there exists i satisfying xi>x, yi>y, then S1064 is executed to determine that the dangerous target is located outside any safe segmentation area, that is, the dangerous target is located on the left side of any safe segmentation area.

[0115] If there is no i satisfying xi>x, yi>y, then S1065 is executed to determine that the dangerous target is not located outside any safe segmentation area.

[0116] S1068: If the dangerous target is located outside any safe segmentation area, execute S105 to terminate the detection of the dangerous target.

[0117] S107: If the dangerous target is not located outside any safe segmented area, determine whether the dangerous target is located in a dangerous segmented area.

[0118] like Figure 7 As shown, the person riding the bicycle is on the right side of the fence. In this case, it is considered safe for the person to ride the bicycle, that is, the coordinate value of the bottom pixel of the person riding the bicycle is greater than the coordinate value of the fence.

[0119] Specifically, see Figure 4 and Figure 8 , Figure 4 Flow chart for determining whether a dangerous target is located in a dangerous segmentation area. Figure 8 A schematic diagram for determining whether a dangerous target is located in a dangerous segmentation area. Determining whether the dangerous target is located in a dangerous segmentation area includes:

[0120] S1071, extract the bottom 5 pixel coordinates of the dangerous target (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5).

[0121] S1072, obtain the minimum value of the five pixel coordinates (x min ,y min ) and the maximum value coordinate (x max ,y max ),in:

[0122] x min =min(x1,x2,x3,x4,x5);

[0123] y min=min(y1,y2,y3,y4,y5);

[0124] x max =max(x1,x2,x3,x4,x5);

[0125] y max =max(y1,y2,y3,y4,y5).

[0126] S1073: according to the minimum value coordinate (x min ,y min ) and the maximum value coordinate (x max ,y max ) to determine the judgment area.

[0127] S1074, counting the number of pixels in the determination area, and checking the number of pixels belonging to the safe segmentation area and the number of pixels belonging to the dangerous segmentation area.

[0128] S1075, according to Calculate the r value, where n_safe is the number of pixels in the safe segmentation area and n_danger is the number of pixels in the dangerous segmentation area.

[0129] S1076, determine whether r is less than k.

[0130] k is a preset threshold value, which can be set as needed.

[0131] S1077: If r≥k, determine that the dangerous target is located in a safe segmentation area.

[0132] When the dangerous target is located in the safe separation area, no alarm will be issued.

[0133] S1078: If r<k, determine that the dangerous target is located in the dangerous segmentation area.

[0134] When a dangerous target is located in a dangerous segmentation area, an alarm is issued.

[0135] Specifically, see Figure 5 , Figure 5 Flow chart for determining the judgment area, according to the minimum value coordinate (x min ,y min ) and the maximum value coordinate (x max ,y max ), determine the judgment area, including:

[0136] S10731, the minimum value coordinate (x min ,y min ) is reduced by 10 to get the x value of the upper left corner of the judgment area lhThe coordinates of (X1, Y1), X1 = x min -10, Y1 = y min -10.

[0137] S10732, the maximum value coordinate (x max ,y max ) increases by 10 to get the x value of the lower right corner of the judgment area rl The coordinates of (X2, Y2), X2 = x max +10, Y2=y max +10.

[0138] S10733, based on the upper left corner x of the judgment area lh The coordinates (X1, Y1) and the lower right corner x of the judgment area rl The coordinates (X2, Y2) are used to determine the judgment area.

[0139] If the dangerous target is not located in the dangerous segmentation area, step S105 is executed to terminate the detection of the dangerous target.

[0140] S108: If the dangerous target is located in the dangerous segmentation area, an alarm is issued that a dangerous target exists in the blind area.

[0141] In bad weather such as rainy days, dark nights, snowy days, foggy weather, etc., the images collected by the monocular cameras installed on both sides of the bus may be unclear, resulting in misjudgment of the area where the dangerous target is located in the blind spot, which increases the danger of driving in bad weather. Therefore, as a preferred embodiment, after setting the detection area in the blind spot image, the method may also include:

[0142] The image is detected by using a pre-trained YOLOv8-seg-cls segmentation model to obtain weather classification information, where the weather classification information includes normal weather and bad weather, where the bad weather may include rainy days, dark nights, snowy days, foggy weather, etc.; whether the weather classification information is bad weather is determined; if the weather classification information is bad weather, a reminder message to pay attention to the rearview mirror is issued; if the weather classification information is normal weather, S103 and subsequent steps are performed normally, and the image is detected by using a pre-trained YOLOv8-seg-cls segmentation model to obtain a segmentation result.

[0143] The present invention determines the weather conditions of the image based on the classification information of the classification detection head. When the image is in normal weather, a normal alarm is sufficient. When it is detected that it is raining heavily and the camera is completely wetted by rain, the driver is directly reminded that the BSD is affected by rain, the alarm may not be accurate, and more attention should be paid to information such as the rearview mirror.

[0144] It can be seen from the above embodiments that the method for detecting blind spots on the left and right sides of a bus provided by the present invention obtains blind spot images on the left and right sides of the bus, sets a detection area in the blind spot image, detects the image through a pre-trained YOLOv8-seg-cls segmentation model, obtains a segmentation result, and determines whether a dangerous target is located in the detection area. If the dangerous target is located in the detection area, it is determined whether the dangerous target is located outside any safe segmentation area. If the dangerous target is not located outside any safe segmentation area, it is determined whether the dangerous target is located in the dangerous segmentation area. If the dangerous target is located in the dangerous segmentation area, an alarm is issued that there is a dangerous target in the blind spot. The present invention identifies and pre-judges the area where the obstacle is located. When the obstacle is located in the safe area, it is determined that it will not affect the driving of the vehicle and no alarm will be issued, thereby improving the accuracy and practicality of the alarm itself.

[0145] In addition, the detection unit of the left and right blind spots of buses of the present invention only uses a monocular camera, which is low-cost and easy to promote. By identifying and prejudging the area where the obstacle is located, when the obstacle is located in a safe area, it is determined that it will not affect the driving of the vehicle and no alarm will be sounded, thereby improving the accuracy and practicality of the alarm itself. Improvements are made to the segmentation technology, and full-image classification recognition is added to the final head part to determine the weather of the detection scene, ensuring that the driver is reminded to pay more attention to driving safety in extreme cases.

[0146] See also Fig. 9 , Fig. 9 The present invention provides a device for detecting the left and right blind spots of a public transport vehicle, comprising:

[0147] An acquisition unit 91 is used to acquire blind spot images on the left and right sides of the bus;

[0148] A setting unit 92, used to set a detection area in the blind area image;

[0149] A detection unit 93 is used to detect the image by using a pre-trained YOLOv8-seg-cls segmentation model to obtain a segmentation result, wherein the segmentation result includes a dangerous target, a dangerous segmentation area, and a safe segmentation area, wherein the dangerous segmentation area and the safe segmentation area are located within the detection area, and the dangerous segmentation area and the safe segmentation area are sequentially arranged from the inside to the outside in a direction away from the bus;

[0150] A first judging unit 94, used to judge whether the dangerous target is located in the detection area;

[0151] A termination unit 95, configured to terminate the detection of the dangerous target when the dangerous target is not located in the detection area;

[0152] A second judgment unit 96 is used to judge whether the dangerous target is located outside any safety segmentation area when the dangerous target is located in the detection area;

[0153] The termination unit 95 is further configured to terminate the detection of the dangerous target when the dangerous target is located outside any of the safety segmentation areas;

[0154] A third judgment unit 97 is used to judge whether the dangerous target is located in a dangerous segmentation area when the dangerous target is not located outside any safe segmentation area;

[0155] The termination unit 95 is further configured to terminate the detection of the dangerous target when the dangerous target is not located in the dangerous segmentation area;

[0156] The alarm unit 98 is used to issue an alarm that a dangerous target exists in a blind spot when the dangerous target is located in the dangerous segmentation area.

[0157] In this embodiment, the first judgment unit further includes:

[0158] The first extraction subunit is used to extract the coordinates (x, y) of the bottom pixel of the dangerous target, the coordinates (x1, y1) of the upper left corner of the detection area, and the coordinates (x2, y2) of the lower right corner of the detection area;

[0159] A second calculation subunit is used to calculate whether x1<x<x2 and y1<y<y2 are satisfied;

[0160] The first determination subunit is used to determine that the dangerous target is located in the detection area when x1<x<x2 and y1<y<y2 are satisfied; and to determine that the dangerous target is not located in the detection area when x1<x<x2 and y1<y<y2 are not satisfied.

[0161] In this embodiment, the second determination unit further includes:

[0162] The second extraction subunit is used to extract the coordinates (x, y) of the bottom 5 pixels of the dangerous target and the coordinates x_safe (xi, yi) of the safe segmentation area, i = 1, 2, 3, 4...n;

[0163] The second calculation subunit is used to calculate whether there is i satisfying xi<x, yi<y for the blind spot image on the left side of the bus;

[0164] The second determination subunit is used to determine that the dangerous target is located outside any safe segmentation area when i satisfies xi<x, yi<y; and to determine that the dangerous target is not located outside any safe segmentation area when i does not satisfy xi<x, yi<y.

[0165] The second calculation subunit is further used to calculate whether there is i satisfying xi>x, yi>y for the blind spot image on the right side of the bus;

[0166] The second determination subunit is further used to determine that the dangerous target is located outside any safety segmentation area when i satisfies xi>x, yi>y; and to determine that the dangerous target is not located outside any safety segmentation area when i does not satisfy xi>x, yi>y.

[0167] In this embodiment, the third judgment unit further includes:

[0168] The third extraction subunit is used to extract the bottom 5 pixel coordinates of the dangerous target (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5);

[0169] Get the subunit for obtaining the minimum value of the 5 pixel coordinates (x min ,y min ) and the maximum value coordinate (x max ,y max ),in:

[0170] x min =min(x1,x2,x3,x4,x5);

[0171] y min =min(y1,y2,y3,y4,y5);

[0172] x max =max(x1,x2,x3,x4,x5);

[0173] y max =max(y1,y2,y3,y4,y5);

[0174] The first determining subunit is used to determine the minimum value coordinate (x min ,y min ) and the maximum value coordinate (x max ,y max ), determine the judgment area;

[0175] A counting subunit, used for counting the number of pixels in the judgment area, checking the number of pixels belonging to the safe segmentation area and the number of pixels belonging to the dangerous segmentation area;

[0176] The third computing subunit is used to Calculate the r value, where n_safe is the number of pixels in the safe segmentation area, and n_danger is the number of pixels in the dangerous segmentation area;

[0177] A judgment subunit, used to judge whether r is less than k;

[0178] The second determination subunit is used to determine that the dangerous target is located in the safe segmentation area when r≥k; the second determination subunit is also used to determine that the dangerous target is located in the dangerous segmentation area when r<k.

[0179] In this embodiment, the first determining subunit further includes:

[0180] The subunit is used to reduce the minimum value of the five pixel coordinates (x min ,y min ) is reduced by 10 to get the x value of the upper left corner of the judgment area lh The coordinates of (X1, Y1), X1 = x min -10, Y1 = y min -10;

[0181] Add a subunit to convert the maximum value of the 5 pixel coordinates (x max ,y max ) increases by 10 to get the x value of the lower right corner of the judgment area rl The coordinates of (X2, Y2), X2 = x max +10, Y2=y max +10.

[0182] The judgment area determines the subunit, which is used to determine the x value of the upper left corner of the judgment area. lh The coordinates (X1, Y1) and the lower right corner x of the judgment area rl The coordinates (X2, Y2) are used to determine the judgment area.

[0183] In this embodiment, the left and right blind spot detection device for a public transport vehicle further includes:

[0184] A weather detection unit, configured to, after setting a detection area in the blind area image, detect the image by using a pre-trained YOLOv8-seg-cls segmentation model to obtain weather classification information, wherein the weather classification information includes normal weather and severe weather;

[0185] A weather judgment unit, used to judge whether the weather classification information is bad weather;

[0186] The reminder unit is used to send a reminder message to pay attention to the rearview mirror when the weather classification information is bad weather. If the weather classification information is normal weather, the image is detected by a pre-trained YOLOv8-seg-cls segmentation model to obtain a segmentation result.

[0187] It can be known from the above embodiments that the left and right blind spot detection device for a bus of the present invention can obtain the blind spot images on the left and right sides of the bus through the acquisition unit; set the detection area in the blind spot image through the setting unit; detect the image through the pre-trained YOLOv8-seg-cls segmentation model through the detection unit to obtain the segmentation result, and the segmentation result includes a dangerous target, a dangerous segmentation area and a safe segmentation area. The dangerous segmentation area and the safe segmentation area are located in the detection area, and the dangerous segmentation area and the safe segmentation area are sequentially arranged from the inside to the outside in the direction away from the bus; determine whether the dangerous target is located in the detection area through the first judgment unit; terminate the detection of the dangerous target if the dangerous target is not located in the detection area through the termination unit; and terminate the detection of the dangerous target if the dangerous target is located in the detection area through the second judgment unit. In the case of a dangerous target, it is determined whether the dangerous target is located outside any safety segmentation area through the termination unit. When the dangerous target is located outside any safety segmentation area, the detection of the dangerous target is terminated. When the dangerous target is not located outside any safety segmentation area, it is determined whether the dangerous target is located in the dangerous segmentation area through the third judgment unit. When the dangerous target is not located in the dangerous segmentation area, the detection of the dangerous target is terminated through the termination unit. When the dangerous target is located in the dangerous segmentation area, an alarm is issued through the alarm unit that there is a dangerous target in the blind spot. The left and right side blind spot detection device for a public bus of the present invention identifies and pre-judges the area where the obstacle is located. When the obstacle is located in the safe area, it is determined that it will not affect the vehicle's driving and will not alarm, thereby improving the accuracy and practicality of the alarm itself.

[0188] See also Fig.10 , Fig.10 This is a block diagram of an electronic device of the present invention. An embodiment of the present invention further provides an electronic device, a memory 100 and a processor 200. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the above method for detecting blind spots on the left and right sides of a public transport vehicle.

[0189] The embodiment of the present invention further provides a storage medium, and the embodiment of the present invention further provides a storage medium, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, some or all of the steps in each embodiment of the method for detecting blind spots on the left and right sides of a public transportation vehicle provided by the present invention are implemented. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0190] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

[0191] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the embodiment of the left and right blind spot detection device for public transportation vehicles, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0192] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. A method for detecting blind spots on the left and right sides of a public transport vehicle, characterized in that: include: Obtain blind spot images on the left and right sides of the bus; Setting the detection area in the blind area image; The image is detected by the pre-trained YOLOv8-seg-cls segmentation model to obtain the segmentation results, which include dangerous targets, dangerous segmentation areas and safe segmentation areas. The dangerous segmentation areas and safe segmentation areas are located in the detection area, and the dangerous segmentation areas and safe segmentation areas are arranged from the inside to the outside in the direction away from the bus. Determine whether the dangerous target is within the detection area; If the dangerous target is not located in the detection area, the detection of the dangerous target is terminated; If the dangerous target is within the detection area, determine whether the dangerous target is outside any of the safety segmentation areas; If the dangerous target is located outside any safe segmentation area, the detection of the dangerous target is terminated; If the dangerous target is not located outside any of the safe segmentation areas, determine whether the dangerous target is located in the dangerous segmentation area; If the dangerous target is not located in the dangerous segmentation area, the detection of the dangerous target is terminated; If a dangerous target is located in the dangerous segmentation area, an alarm is issued that there is a dangerous target in the blind area.

2. The method for detecting blind spots on the left and right sides of a public transportation vehicle according to claim 1, characterized in that: Determine whether the dangerous target is within the detection area, including: Extract the coordinates (x, y) of the bottom pixel of the dangerous target, the coordinates (x1, y1) of the upper left corner of the detection area, and the coordinates (x2, y2) of the lower right corner of the detection area; Calculate whether x1<x<x2, and y1<y<y2; If x1<x<x2, and y1<y<y2, it is determined that the dangerous target is within the detection area; If x1<x<x2 and y1<y<y2 are not satisfied, it is determined that the dangerous object is not located in the detection area.

3. The method for detecting blind spots on the left and right sides of a public transportation vehicle according to claim 1, characterized in that: Determine whether the dangerous target is located outside any safe segmentation area, including: Extract the coordinates (x, y) of the bottom 5 pixels of the dangerous target and the coordinates x_safe (xi, yi) of the safe segmentation area, i = 1, 2, 3, 4...n; For the blind spot image on the left side of the bus, calculate whether there is i that satisfies xi<x, yi<y; If there exists i that satisfies xi<x, yi<y, it is determined that the dangerous target is located outside any safe segmentation area; If there is no i that satisfies xi<x, yi<y, it is determined that the dangerous target is not located outside any safe segmentation area; For the blind spot image on the right side of the bus, calculate whether there is i that satisfies xi>x, yi>y; If there exists i that satisfies xi>x, yi>y, it is determined that the dangerous target is located outside any safe segmentation area; If there is no i that satisfies xi>x, yi>y, it is determined that the dangerous target is not located outside any of the safe segmentation areas.

4. The method for detecting left and right blind spots of a public transportation vehicle according to claim 1, characterized in that: Determine whether the dangerous target is located in the dangerous segmentation area, including: Extract the coordinates of the bottom five pixels of the dangerous target (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5); Get the minimum value of the 5 pixel coordinates (x min ,y min ) and the maximum value coordinate (x max ,y max ),in: x min =min(x1,x2,x3,x4,x5); <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> min <h2 style=";text-align:left;direction:ltr"> (min(y1,y2,y3,y4,y5)) x max =max(x1,x2,x3,x4,x5); <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> max <h2 style=";text-align:left;direction:ltr"> (max(y1, y2, y3, y4, y5)) According to the minimum coordinate of the 5 pixel coordinates (x min ,y min ) and the maximum value coordinate (x max ,y max ), determine the judgment area; Count the number of pixels in the judgment area and check the number of pixels belonging to the safe segmentation area and the number of pixels belonging to the dangerous segmentation area; according to Calculate the r value, where n_safe is the number of pixels in the safe segmentation area, and n_danger is the number of pixels in the dangerous segmentation area; Determine whether r is less than k; If r ≥ k, it is determined that the dangerous target is located in the safe segmentation area; If r<k, it is determined that the dangerous target is located in the dangerous segmentation area.

5. The method for detecting blind spots on the left and right sides of a public transportation vehicle as claimed in claim 4, characterized in that: According to the minimum coordinate of the 5 pixel coordinates (x min ,y min ) and the maximum value coordinate (x max ,y max ), determine the judgment area, including: The minimum coordinate (x min ,y min ) is reduced by 10 to get the x value of the upper left corner of the judgment area lh The coordinates of (X1, Y1), X1 = x min -10, Y1 = y min -10; The maximum coordinate (x max ,y max ) plus 10 to get the x value of the lower right corner of the judgment area rl The coordinates of (X2, Y2), X2 = x max +10, Y2=y max +10. According to the upper left corner x of the judgment area lh The coordinates (X1, Y1) and the lower right corner x of the judgment area rl The coordinates (X2, Y2) are used to determine the judgment area.

6. The method for detecting blind spots on the left and right sides of a public transportation vehicle according to claim 1, characterized in that: After setting the detection area in the blind area image, the method further includes: The image is detected through the pre-trained YOLOv8-seg-cls segmentation model to obtain weather classification information, which includes normal weather and severe weather; Determine whether the weather classification information is severe weather; If the weather classification information is severe weather, a reminder message to pay attention to the rearview mirror will be issued; If the weather classification information is normal weather, the image is detected through the pre-trained YOLOv8-seg-cls segmentation model to obtain the segmentation result.

7. A device for detecting blind spots on the left and right sides of a public transport, characterized in that: include: An acquisition unit, used for acquiring blind spot images on the left and right sides of the bus; A setting unit, used for setting a detection area in a blind area image; A detection unit is used to detect the image through a pre-trained YOLOv8-seg-cls segmentation model to obtain a segmentation result, wherein the segmentation result includes a dangerous target, a dangerous segmentation area, and a safe segmentation area. The dangerous segmentation area and the safe segmentation area are located within the detection area, and the dangerous segmentation area and the safe segmentation area are arranged in sequence from the inside to the outside in a direction away from the bus; A first judgment unit, used to judge whether a dangerous target is located in a detection area; A termination unit, used for terminating the detection of the dangerous target when the dangerous target is not located in the detection area; A second judgment unit is used to judge whether the dangerous target is located outside any safety segmentation area when the dangerous target is located in the detection area; The termination unit is also used to terminate the detection of the dangerous target when the dangerous target is located outside any of the safety segmentation areas; A third judgment unit is used to judge whether the dangerous target is located in the dangerous segmentation area when the dangerous target is not located outside any of the safe segmentation areas; The termination unit is further used to terminate the detection of the dangerous target when the dangerous target is not located in the dangerous segmentation area; The alarm unit is used to issue an alarm that a dangerous target exists in a blind spot when the dangerous target is located in the dangerous segmentation area.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the methods of claims 1 to 6.

Images