Head collision prevention alarm method and device, electronic equipment and storage medium

Abstract

The invention provides an anti-head-collision alarm method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a back row video of a vehicle within a preset duration; identifying a user in the back-row image of the back-row video to obtain user information of the user in each frame of back-row image; screening out a user of which the human body position is located in an aisle area from a current frame of back row image in the back row video to obtain a target user; according to the user identifier of the target user, judging whether a user which is the same as the user identifier and of which the human body position is located at a rear-row seat exists in other frames of rear-row images in the rear-row video or not; and if the judgment result is yes, giving an alarm when the occlusion area in the current frame of rear-row image is greater than a preset occlusion area threshold value.

Description

Technical Field

[0001] The present disclosure relates to the technical field of anti-collision head alarm in vehicles, and in particular to an anti-collision head alarm method, device, electronic equipment and storage medium. Background Art

[0002] In related anti-head collision technology, taking the scenario of preventing a person's head from colliding with an obstacle (entertainment screen) in a vehicle as an example: a Time of Flight (TOF) camera is usually used to obtain depth information between the person's head and the camera. The distance between the person's head and the camera indicated by the depth information is used to determine whether the person's head has entered the target area where there is a risk of head collision, and then the probability of an anti-head collision alarm is determined based on the judgment result.

[0003] However, the entertainment screen is usually set between the front seats and the second row of seats, and the TOF camera is usually set twenty centimeters behind the entertainment screen; therefore, the TOF camera cannot capture depth images within twenty centimeters of the entertainment screen in the aisle, which results in the inability to accurately determine the alarm probability corresponding to the rear-seat user when he gets off the car if he is within twenty centimeters of the entertainment screen. Summary of the Invention

[0004] The present disclosure provides an anti-head collision alarm method, device, electronic device and storage medium.

[0005] According to a first aspect of the present disclosure, there is provided an anti-head collision alarm method, comprising:

[0006] Obtain rear-seat video of the vehicle within a preset time period; rear-seat video refers to video captured by a video collector set behind an obstacle;

[0007] Identify users in the rear-row images of the rear-row video and obtain user information of the users in each frame of the rear-row image; the user information includes the user's body position and user ID; in each frame of the rear-row image of the rear-row video, different users have different user IDs;

[0008] Filter out users whose bodies are located in the aisle area from the back row image of the current frame in the back row video to obtain the target user;

[0009] Based on the user ID of the target user, determining whether there is a user with the same user ID and whose body position is in the back row seat in other frames of the back row video; other frames of the back row image refer to images in the back row video other than the current frame of the back row image;

[0010] If the judgment result is yes, an alarm is issued when the occlusion area in the rear row image of the current frame is larger than a preset occlusion area threshold.

[0011] In one embodiment, when the occlusion area in the rear row of images of the current frame is larger than a preset occlusion area threshold, an alarm is issued, including:

[0012] Get the pixel value of each pixel in the back row of the current frame;

[0013] Comparing the pixel value of the pixel point with the preset pixel value, determining the number of pixel points in the back row image of the current frame whose pixel value is less than the preset pixel value;

[0014] Determine whether the number of pixel points whose pixel values ​​are greater than a preset pixel value is greater than the preset number of pixel points; the pixel points whose pixel values ​​are greater than the preset pixel value are the pixel points of the occlusion area;

[0015] If the judgment result is yes, an alarm is issued.

[0016] In one embodiment, after determining whether the number of pixels having pixel values ​​greater than a preset pixel value is greater than a preset number of pixels, the method provided by the present disclosure includes:

[0017] If the result of the judgment is negative, the target user's head bounding box is determined to be within the target area based on the position coordinates of the target user's head bounding box; the target area refers to the risk area where the rear seat user may collide with obstacles when getting out of the vehicle;

[0018] If the judgment result is yes, determine whether the area of ​​the target user's head bounding box is smaller than a preset area threshold;

[0019] If the judgment result is no, an alarm is issued.

[0020] In one embodiment, identifying a user in a rear-row image of a rear-row video and obtaining user information of the user in each frame of the rear-row image includes:

[0021] Identifying at least one user in a first frame of a rear-row image in a rear-row video, and marking different users in the first frame of the rear-row image with different user identifiers;

[0022] Identifying at least one user in a second frame of the rear-row image in the rear-row video, and determining, using a target tracking method, whether there is a user among the users identified in the second frame of the rear-row image that matches the user identified in the first frame of the rear-row image;

[0023] If the judgment result is yes, the user in the second frame of the rear image that can be matched with the user in the first frame of the rear image is marked with the same user ID;

[0024] The second frame of the rear-row image is updated to the first frame of the rear-row image, and each frame of the rear-row image in the rear-row video is traversed to obtain the user identifier of the user in each frame of the rear-row image.

[0025] In one embodiment, identifying user information of a user in each frame of a rear-row image of a rear-row video to obtain the user information of the user in each frame of the rear-row image includes:

[0026] Identify the human body and human head in each frame of the back-row image of the back-row video, and obtain the external bounding box of the human body and the external bounding box of the human head in each frame of the back-row image;

[0027] Determine the intersection-over-union (IoU) of each person's body bounding box and each head bounding box in each frame of the back row image, and match the person corresponding to the person's body bounding box with the head bounding box whose IoU is not less than a preset IoU threshold;

[0028] The center point position of the user's body circumference frame is determined as the user's body position, and the center point position of the head circumference frame of the head matching the body is determined as the user's head position.

[0029] In one embodiment, before comparing the size of the occlusion area in the rear row image of the current frame with a preset occlusion area threshold, the method provided by the present disclosure includes:

[0030] Obtain the pressure value of each rear seat at the time corresponding to the capture moment of each rear row image frame in the rear row video within a preset time length;

[0031] Obtaining a pressure change value for each rear seat according to the pressure value of each rear seat at a corresponding moment of acquisition of each rear image frame in the rear video;

[0032] It is determined that the user corresponding to the rear seat whose pressure change value is greater than the preset pressure change value is the target user.

[0033] In one embodiment, a method for calibrating a target area includes:

[0034] Obtain multiple training images, where the training images are images of the head of a rear-seat user at the boundary of the risk area when the user exits the vehicle.

[0035] Recognize the human heads in the training image to obtain multiple head bounding boxes in the training image; wherein each head corresponds to a head bounding box;

[0036] Determine the center point position of each head bounding box, and obtain the center point positions of multiple head bounding boxes;

[0037] The closed area formed by the lines connecting multiple center points is determined as the target area.

[0038] According to a second aspect of the present disclosure, there is provided an anti-head collision alarm device, comprising:

[0039] An acquisition unit, which acquires rear-seat video of the vehicle within a preset time period; the rear-seat video refers to video acquired by a video collector set behind the obstacle;

[0040] an identification unit for identifying users in the rear-row images of the rear-row video and obtaining user information of the users in each frame of the rear-row image; the user information includes the user's body position and user identification; different users have different user identifications in each frame of the rear-row image of the rear-row video;

[0041] A screening unit, screening out users whose bodies are located in the aisle area from the back row image of the current frame in the back row video, to obtain a target user;

[0042] a determination unit, based on the user identifier of the target user, determining whether there is a user with the same user identifier and whose body position is in the back row seat in other frame back row images in the back row video; the other frame back row images refer to images in the back row video other than the current frame back row image;

[0043] The determination unit, if the judgment result is yes, issues an alarm when the occlusion area in the rear row image of the current frame is larger than a preset occlusion area threshold.

[0044] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0045] at least one processor; and

[0046] a memory communicatively connected to at least one processor; wherein,

[0047] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect.

[0048] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method of the aforementioned first aspect.

[0049] The present disclosure provides an anti-head collision alarm method, device, electronic device and storage medium. The method provided by the present disclosure includes: obtaining a rear-seat video of a vehicle within a preset time length; the rear-seat video refers to a video collected by a video collector set behind an obstacle. Identifying the user in the rear-seat image of the rear-seat video, and obtaining the user information of the user in each frame of the rear-seat image; the user information includes the user's body position and user identification; in each frame of the rear-seat image of the rear-seat video, the user identification of different users is different; filtering out the user whose body position is in the aisle area from the rear-seat image of the current frame in the rear-seat video, and obtaining the target user; judging, based on the user identification of the target user, whether there is a user with the same user identification and whose body position is in the rear seat in the rear-seat image of other frames in the rear-seat video; the rear-seat image of other frames refers to the image in the rear-seat video other than the rear-seat image of the current frame; if the judgment result is yes, an alarm is issued when the occlusion area in the rear-seat image of the current frame is larger than the preset occlusion area threshold.

[0050] According to the solution of the present disclosure, the solution provided by the present disclosure determines whether the user in the aisle has moved from the back seat to the aisle user through the user identification in the back video, and then determines the alarm probability based on the occlusion area of ​​the acquired back image. Therefore, when the user in the back seat moves to within twenty centimeters of an obstacle, the alarm probability can also be determined based on the occlusion area of ​​the back image, thereby ensuring accurate determination of the alarm probability corresponding to the user getting off the back seat.

[0051] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0053] Figure 1 A flowchart of an anti-head collision alarm method provided by an embodiment of the present disclosure;

[0054] Figure 2 A schematic diagram of the seat distribution in the cabin provided by an embodiment of the present disclosure;

[0055] Figure 3 A schematic diagram of the structure of key points of the human body provided by an embodiment of the present disclosure;

[0056] Figure 4 A flowchart of a first method for determining an alarm probability provided by an embodiment of the present disclosure;

[0057] Figure 5 A flowchart of a second method for determining an alarm probability provided by an embodiment of the present disclosure;

[0058] Figure 6 A flowchart of the first method for obtaining user information provided by an embodiment of the present disclosure;

[0059] Figure 7 A flowchart of a second method for obtaining user information provided by an embodiment of the present disclosure;

[0060] Figure 8 A flow chart of a method for determining a pressure change value provided by an embodiment of the present disclosure;

[0061] Figure 9 A flowchart of a method for obtaining a target area provided by an embodiment of the present disclosure;

[0062] Figure 10 A schematic structural diagram of an anti-head collision alarm device provided by an embodiment of the present disclosure;

[0063] Figure 11 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0064] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0065] The anti-collision head alarm method provided in the embodiments of the present disclosure can be applied to vehicles equipped with non-depth video collectors, such as IR cameras. The executor of the method can be a processor set on the vehicle side or in the cloud, such as the processor of the anti-collision head alarm module in the vehicle, or the vehicle controller.

[0066] like Figure 1 As shown, an anti-head collision alarm method provided by an embodiment of the present disclosure includes the following steps:

[0067] Step 101, obtaining a rear-seat video of a vehicle within a preset time period;

[0068] In one embodiment, the rear video refers to a video captured by a video collector disposed behind an obstacle;

[0069] In one embodiment, the video collector is a non-depth image collector. Compared with non-depth image collectors, when processing the depth image collected by the depth image collector, it is usually necessary to convert the depth information in the depth image into the distance information between the target (human body, human head) in the depth image and the depth image collector, which consumes a lot of computing resources.

[0070] In one embodiment, the cabin seat distribution diagram is as follows: Figure 2 As shown, the front seats refer to seats 0 and 1, the second row seats refer to seats 2 and 3, the back seats refer to seats 4 and 5, and the aisle area refers to seat 7.

[0071] In one embodiment, since the area under the aisle can be captured by a video collector set behind an obstacle and will not be blocked by front-row items and obstacles (entertainment screen), the video collector set behind the obstacle is responsible for collecting rear-row videos within a preset time period corresponding to the second row seats, rear seats and aisle area inside the cabin.

[0072] In one embodiment, the preset duration is preferably 5 seconds, because a user can usually stand up from a back seat and move to the front of the entertainment screen within 5 seconds.

[0073] In one embodiment, the user standing up from the rear seat and moving to the front of the entertainment screen means that the user moves from seat 4 or seat 5 to seat 7.

[0074] In one embodiment, the rear-row video includes multiple frames of continuous rear-row images.

[0075] In one embodiment, there may be one or more video collectors positioned behind the obstacle. Since a single video collector positioned behind the obstacle can capture rear-row video of the entire cabin, including the second row of seats, the rear seats, and the aisle area, for a preset duration, the number of video collectors positioned behind the obstacle is preferably one. This disclosure does not limit the number of video collectors positioned behind the obstacle.

[0076] Step 102: Identify users in the rear-row images of the rear-row video and obtain user information of the users in each frame of the rear-row image;

[0077] In one embodiment, the user information includes the user's body position and user identification.

[0078] In one embodiment, the user information may further include a bounding box around the user's head.

[0079] In one embodiment, the human body position is used to indicate coordinate information of a bounding box of the human body.

[0080] In one embodiment, a human body detection and classification network may be used to identify the human body in each frame of the back-row image in the back-row video, and obtain the human body position of each user in each frame of the back-row image in the back-row video.

[0081] In one embodiment, a human head detection and classification network can also be used to identify the human bodies and heads of users in the back row images, and at the same time obtain the human body bounding box and human body confidence corresponding to each user, and obtain the head bounding box and head confidence corresponding to each head.

[0082] In one embodiment, the confidence level is used to indicate the credibility of the human body bounding box or the human head bounding box.

[0083] In one embodiment, a confidence threshold may be preset, and the human body circumference frames and human head circumference frames with confidence lower than the preset confidence threshold are deleted, and only credible human body circumference frames and human head circumference frames remain.

[0084] In one embodiment, the user corresponding to the human body bounding box and the head bounding box whose intersection-over-union ratio (IoU) between the human body bounding box and the head bounding box is greater than a preset IoU threshold are matched.

[0085] In one embodiment, the area selected by the human body external frame can also be cropped from the back row image to obtain the user image. Then, the key points of the user image are detected using the human body key point detection network; Figure 3 As shown, the positions of points 15, 16, 17 and 18 among the key points of the human body are used as the positions of the human body.

[0086] In one embodiment, the user ID represents the unique ID of the user in each frame of the rear-seat image within a preset time period. That is, even if the user gets up from the rear seat and moves to the front of the entertainment screen, the user ID will not change during this process.

[0087] In one embodiment, the user identification may be obtained by any one of a Kalman filter method, a convolutional neural network method, and a multi-target tracker method.

[0088] In one embodiment, in each frame of the back-row image of the back-row video, the user identifiers of different users are different.

[0089] Step 103, filtering out users whose bodies are located in the aisle area from the back row image of the current frame in the back row video to obtain target users;

[0090] In one embodiment, the user's body position information is used to match the body with the rear seat or aisle area.

[0091] In one embodiment, if there is no human body in the aisle area, there is no need to trigger the anti-head collision alarm.

[0092] In one embodiment, if there is a human body in the aisle area, the human body position information that matches the aisle information is filtered out from the human head position information in the rear row image of the current frame.

[0093] Step 104 , based on the user ID of the target user, it is determined whether there is a user with the same user ID and whose body position is in the back row seat in the back row images of other frames in the back row video.

[0094] In one embodiment, if there is a user with the same user ID in the back row images of other frames in the back row video and the body position is in the back seat, it means that the user in the current aisle area has moved from the back seat.

[0095] In one embodiment, the other frame rear-row images refer to images in the rear-row video except the current frame rear-row image.

[0096] In one embodiment, there is no user with the same user ID in the back row images of other frames in the back row video, and the user's body position is in the back row seat, which means that the body of the user corresponding to the head that highly obscures the video collector is not the person who stood up in the back row, and no anti-head collision alarm is issued.

[0097] Step 105: If the judgment result is yes, an alarm is issued when the occlusion area in the rear row image of the current frame is larger than a preset occlusion area threshold.

[0098] In one embodiment, the preset occlusion area threshold is half of the total area of ​​the rear row image.

[0099] In one embodiment, if the occlusion area in the rear row of images of the current frame is smaller than a preset occlusion area threshold, it can be determined that the video collector is not highly occluded at the current moment.

[0100] In one embodiment, if the occlusion area in the rear image of the current frame is larger than a preset occlusion area threshold, it can be determined that the video collector is highly occluded at the current moment, and the human head is very close to the entertainment screen, and an anti-head collision alarm is issued.

[0101] In one embodiment, the alarm may be an intelligent voice announcement.

[0102] The present disclosure provides a head collision prevention alarm method, device, electronic device, and storage medium. The method provided by the present disclosure includes: obtaining a rear-seat video of a vehicle within a preset time period; the rear-seat video refers to a video captured by a video collector set behind an obstacle; identifying users in the rear-seat image of the rear-seat video, and obtaining user information of the users in each frame of the rear-seat image; the user information includes the user's body position and user ID; different users have different user IDs in each frame of the rear-seat image of the rear-seat video; filtering out users whose body positions are located in the aisle area from the current frame of the rear-seat image in the rear-seat video to obtain a target user; based on the user ID of the target user, determining whether there is a user with the same user ID and whose body position is located in the rear seat in other frames of the rear-seat image in the rear-seat video; other frames of the rear-seat image refer to images in the rear-seat video other than the current frame of the rear-seat image; if the judgment result is yes, an alarm is issued when the occlusion area in the current frame of the rear-seat image is greater than a preset occlusion area threshold.

[0103] According to the solution of the present disclosure, the solution provided by the present disclosure determines whether the user in the aisle has moved from the back seat to the aisle user through the user identification in the back video, and then determines the alarm probability based on the occlusion area of ​​the acquired back image. Therefore, when the user in the back seat moves to within twenty centimeters of an obstacle, the alarm probability can also be determined based on the occlusion area of ​​the back image, thereby ensuring accurate determination of the alarm probability corresponding to the user getting off the back seat.

[0104] In one embodiment, if Figure 4 As shown, step 105 includes:

[0105] Step 401, obtaining the pixel value of each pixel in the back row image of the current frame;

[0106] In one embodiment, the pixel value of each pixel in the rear-row image of the current frame can be obtained by traversing the pixel value of each pixel in the rear-row image of the current frame through image processing software or a function in MATLAB.

[0107] In one embodiment, the pixel value of each pixel in the rear-row image of the current frame may also be obtained by reading specific pixel values ​​according to the row and column coordinates of the pixels in the rear-row image of the current frame in the image matrix.

[0108] Step 402 , comparing the pixel value of the pixel point with a preset pixel value, and determining the number of pixel points in the rear row image of the current frame whose pixel value is less than the preset pixel value;

[0109] In one embodiment, the preset pixel value is the pixel value of a pixel in the rear image when the video collector is highly blocked, such as any value between 0 and 120. When the pixel value is 0, it means that the video collector is completely blocked and the rear image is black.

[0110] In one embodiment, when the video collector is highly obscured, the light flux obtained by the video collector is greatly reduced, causing its imaging to become darker. At this time, the number of pixel points in the rear image of the current frame whose pixel values ​​are greater than the preset pixel value is greater than the preset number of pixel points.

[0111] Step 403, determining whether the number of pixels whose pixel values ​​are greater than a preset pixel value is greater than a preset number of pixels;

[0112] In one embodiment, pixel points with pixel values ​​greater than a preset pixel value are pixel points in the occlusion area.

[0113] In one embodiment, if the number of pixel points whose pixel values ​​in the rear row of the current frame are less than the preset pixel value is less than the preset number of pixel points, it can be determined that the video collector is not highly blocked at the current moment.

[0114] In one embodiment, if the number of pixel points whose pixel values ​​in the rear row of the current frame are greater than the preset pixel value is greater than the preset number of pixel points, it can be determined that at the current moment, the video collector is highly blocked and the human head is very close to the entertainment screen, and an anti-head collision alarm is issued.

[0115] In one embodiment, the preset number of pixels may be half of the total number of pixels in the rear image. Typically, if half of the area in the rear image is too dark, it means that the rear image is highly obscured.

[0116] In one embodiment, the preset number of pixels may be set to other values ​​according to actual alarm requirements, such as one-third of the total number of pixels in the rear image, etc., which is not limited in the present disclosure.

[0117] Step 404: If the judgment result is yes, an alarm is issued.

[0118] In one embodiment, if the judgment result is no, it can be determined that at the current moment, the video collector is not highly blocked, the human head is far away from the entertainment screen, and the anti-head collision alarm is not triggered.

[0119] In one embodiment, the pixel values ​​in steps 401 to 403 may also be the grayscale values ​​of pixels in the grayscale image after the rear image is converted into a grayscale image. Since the grayscale value in the grayscale image represents the brightness of the pixel, the grayscale value of the pixel can indicate whether the area corresponding to the pixel is an occluded area.

[0120] In one embodiment, if Figure 5 As shown, after step 403, the following steps are included:

[0121] Step 501: If the result of the judgment is no, determine whether the bounding box of the target user's head is located in the target area based on the position coordinates of the bounding box of the target user's head;

[0122] In one embodiment, the target area refers to a risk area where rear seat users may collide with obstacles when getting out of the vehicle.

[0123] In one embodiment, it is necessary to determine whether to issue an anti-head collision alarm only when the external frame of the human head is located in the target area.

[0124] In one embodiment, when the head frame is not located in the target area, the head collision prevention alarm is not triggered.

[0125] In one embodiment, if the judgment result of step 501 is yes, the number of pixel points whose pixel values ​​in the rear row image of the current frame are greater than the preset pixel value is greater than the preset number of pixel points, then it can be determined that at the current moment, the video collector is highly blocked and the human head is very close to the entertainment screen, and the first alarm probability is determined to be 1, triggering the anti-head collision alarm.

[0126] Step 502: If the result of the judgment is yes, determine whether the area of ​​the target user's head bounding box is smaller than a preset area threshold;

[0127] In one embodiment, since the rear images are not captured by the depth image collector, the size of the head bounding box corresponding to the head in the rear images is highly correlated with the distance between the head and the obstacle. Specifically, the larger the head bounding box, the greater the distance between the head and the video collector, and the greater the distance between the head and the obstacle. Conversely, the smaller the head bounding box, the smaller the distance between the head and the video collector, and the smaller the distance between the head and the obstacle.

[0128] In one embodiment, the preset area threshold is the area value of the head circumscribed frame on the rear row image when the head is relatively close to the entertainment screen.

[0129] In one embodiment, the preset area threshold may be obtained from historical experience of head area thresholds in related anti-collision head alarm technologies.

[0130] Step 503: If the judgment result is no, an alarm is issued;

[0131] In one embodiment, if Figure 6 As shown, step 102 includes:

[0132] Step 601: identifying at least one user in a first frame of a rear-row image in a rear-row video, and marking different users in the first frame of the rear-row image with different user identifiers;

[0133] In one embodiment, based on the user identifier, it can be determined whether the users in the rear-row images of different frames are the same user.

[0134] In one embodiment, user IDs of different users may be represented by consecutive Arabic numerals.

[0135] Step 602: Identify at least one user in a second frame of the rear-row image in the rear-row video, and use a target tracking method to determine whether there is a user among the users identified in the second frame of the rear-row image that matches the user identified in the first frame of the rear-row image.

[0136] In one embodiment, the target tracking method may be any one of a Kalman filter method, a convolutional neural network method, and a multi-target tracker method.

[0137] Step 603: If the judgment result is yes, the user in the second frame of the rear-row image that can be matched with the user in the first frame of the rear-row image is marked with the same user identifier;

[0138] In one embodiment, the user in the second frame of rear-row image that can be matched with the user in the first frame of rear-row image is the same user, and their user identifiers are the same.

[0139] In one embodiment, if the judgment result is negative, a different user identifier is marked for the user in the second frame of the rear-row image that cannot be matched with the user in the first frame of the rear-row image;

[0140] Step 604: Update the second frame of the rear-row image to the first frame of the rear-row image, traverse each frame of the rear-row image in the rear-row video, and obtain the user identifier of the user in each frame of the rear-row image.

[0141] In one embodiment, by traversing each frame of the rear-row image in the rear-row video, the user identification of the user in each frame of the rear-row image is obtained, and the body position and head information of the user corresponding to the user identification is stored in the database. The user information in the rear-row image can be quickly found according to the user identification, providing conditions for improving the efficiency of anti-collision head alarm judgment.

[0142] In one embodiment, if Figure 7 As shown, step 102 includes:

[0143] Step 701: Identify the human body and the human head in each frame of the rear-row image of the rear-row video, and obtain the external bounding box of the human body and the external bounding box of the human head in each frame of the rear-row image;

[0144] In one embodiment, a human head classification detection network may be used to identify the human bodies and heads of users in each frame of the rear-row image to obtain the human body circumference frame of each user's human body and the head circumference frame of each head in each frame of the rear-row image.

[0145] In one embodiment, the human body circumference frame and the head circumference frame are usually represented by their corresponding position coordinates. Taking the human body circumference frame as a rectangle as an example, the position coordinates of the upper left vertex and the lower right vertex of the rectangle are usually used to represent the corresponding human body circumference frame.

[0146] In one embodiment, the human head classification detection network can also determine the confidence level of each human body bounding box and each head bounding box. The confidence level indicates the credibility of the human body bounding box or the head bounding box.

[0147] In one embodiment, a confidence threshold may be preset, and the human body and head circumference frames with confidences less than the preset confidence threshold are deleted, leaving only the human body and head circumference frames of credible users.

[0148] Step 702: Determine the intersection-over-union (IoU) of the bounding box of each user's body and the bounding box of each head in each rear-row image frame, and match the body corresponding to the bounding box of the body with the head corresponding to the bounding box of the head whose IoU is not less than a preset IoU threshold.

[0149] In one embodiment, no matter the user's posture is sitting, standing or other postures (bending over, lowering the head, turning the head, etc.), the human body external bounding box and the human head external bounding box usually have an overlapping area.

[0150] In one embodiment, the overlapping area of ​​the human body circumference frame and the head circumference frame is defined as the intersection area, and the total area occupied by the human body circumference frame and the head circumference frame is defined as the union area; the intersection-to-union ratio refers to the ratio of the intersection area to the union area.

[0151] In one embodiment, a larger IoU indicates greater overlap between the body's bounding box and the head's bounding box, i.e., a greater probability that the body and head belong to the same user. Therefore, based on a large number of calibrations, the IoU of the body's bounding box and the head's bounding box in different poses of the same user can be determined and used as a preset IoU threshold.

[0152] In one embodiment, in the matching results of the user corresponding to the human body bounding box and the head corresponding to the head bounding box whose intersection-over-union ratio is not less than a preset intersection-over-union ratio threshold, the human body and head corresponding to the human body bounding box and the head bounding box of the user that can be matched belong to the same user.

[0153] Step 703 : determining the center point position of the user's body circumference frame as the user's body position, and determining the center point position of the head circumference frame of the head that matches the body as the user's head position.

[0154] In one embodiment, if the center point of the user's body external bounding box is located at the back seat, it is determined that the user's body is located at the back seat.

[0155] In one embodiment, if the center point of the circumscribed frame of the user's head is located in the back seat, it is determined that the body position corresponding to the head is located in the back seat.

[0156] In one embodiment, if Figure 8 As shown, before step 105, the following steps are included:

[0157] Step 801, obtaining the pressure value of each rear seat at the time corresponding to the capture moment of each rear image frame in the rear video within a preset time period;

[0158] In one embodiment, the preset duration is preferably 5 seconds, because a person can usually stand up from a back seat and move to the front of the entertainment screen within 5 seconds.

[0159] In one embodiment, the pressure value of each rear seat may be acquired by a pressure sensor disposed on the rear seat.

[0160] Step 802, obtaining a pressure change value for each rear row seat based on the pressure value of each rear row seat at the time corresponding to the acquisition time of each rear row image frame in the rear row video;

[0161] In one embodiment, whether there is someone in the rear seat can be determined by the pressure sensor signal of the rear seat. For example, when the pressure sensor signal is 1, it indicates that the pressure value of the rear seat is very high and there is someone in the rear seat. When the pressure sensor signal is 0, it indicates that the pressure value of the rear seat is negligible and there is no one in the rear seat.

[0162] In one embodiment, when the pressure value of the rear seat is 1 and does not change within a preset time period, it indicates that there is no one in the rear seat or there is no action of the user leaving the seat and getting off the vehicle.

[0163] In one embodiment, when the pressure value of the rear seat changes from 1 to 0 within a preset time period, it indicates that a user has left the rear seat.

[0164] Step 803 : Determine that the user corresponding to the rear seat whose pressure change value is greater than the preset pressure change value is the target user.

[0165] In one embodiment, when the rear seat pressure change value is greater than a preset pressure change value, it indicates that the rear seat has changed from being occupied by a user to being empty, that is, there is an action of a user getting up from the rear seat, and the user corresponding to the rear seat is the target user.

[0166] In one embodiment, if the seat information of the target user is changed to the aisle area, it is necessary to perform an anti-head collision alarm judgment based on the occlusion status of the video collector.

[0167] In one embodiment, if Figure 9 As shown, the target area calibration method includes:

[0168] Step 901: Acquire multiple training images, where the training images are images of rear-seat users' heads at the boundary of a risk area when they get off the vehicle.

[0169] In one embodiment, the boundaries of the risk area are typically defined by product managers.

[0170] In one embodiment, the boundary position of the risk area refers to the position where the anti-collision head alarm is triggered.

[0171] Step 902: Recognize the human head in the training image to obtain multiple human head bounding boxes in the training image; wherein each human head corresponds to a human head bounding box;

[0172] In one embodiment, a human head classification detection network is used to identify the human head in each training image to obtain a bounding box of the human head in each training image.

[0173] Step 903: Determine the center point position of each head circumscribed frame to obtain the center point positions of multiple head circumscribed frames;

[0174] In one embodiment, the center point position is used to indicate the center coordinate point of the circumscribed frame of the head.

[0175] In one embodiment, the center point position of the head circumference frame may be obtained through the position of the head circumference frame.

[0176] Step 904: Determine a closed area formed by connecting lines of multiple center points as the target area.

[0177] In one embodiment, the size of the target area is determined by the size of the area enclosed by all head positions at the boundary positions of the risk area in all training images.

[0178] Figure 10 This is a schematic diagram of the structure of the anti-collision head alarm device provided by the embodiment of the present disclosure, as shown in FIG. Figure 10 As shown, the anti-head collision alarm device 1000 includes:

[0179] The acquisition unit 1001 acquires rear-seat video of the vehicle within a preset time period; the rear-seat video refers to video captured by a video collector set behind the obstacle;

[0180] Identification unit 1002 identifies users in the rear-row images of the rear-row video and obtains user information of the users in each frame of the rear-row image; the user information includes the user's body position and user identification; different users have different user identifications in each frame of the rear-row image of the rear-row video;

[0181] A screening unit 1003 filters out users whose body positions are located in the aisle area from the back row image of the current frame in the back row video to obtain a target user;

[0182] The determining unit 1004 determines, based on the user identifier of the target user, whether there is a user with the same user identifier and whose body position is in the back row seat in any of the back row images in other frames of the back row video; the back row images in other frames refer to images in the back row video other than the back row image in the current frame;

[0183] If the determination result is yes, the determination unit 1005 generates an alarm when the occlusion area in the rear row image of the current frame is larger than a preset occlusion area threshold.

[0184] In one embodiment, the determining unit 1005 is specifically configured to:

[0185] Get the pixel value of each pixel in the back row of the current frame;

[0186] Comparing the pixel value of the pixel point with the preset pixel value, determining the number of pixel points in the back row image of the current frame whose pixel value is greater than the preset pixel value;

[0187] Determine whether the number of pixel points whose pixel values ​​are greater than a preset pixel value is greater than the preset number of pixel points; the pixel points whose pixel values ​​are greater than the preset pixel value are the pixel points of the occlusion area;

[0188] If the judgment result is yes, an alarm is issued.

[0189] In one embodiment, the determining unit 1005 is specifically configured to:

[0190] After determining whether the number of pixels having pixel values ​​greater than a preset pixel value is greater than a preset number of pixels, the method provided by the present disclosure includes:

[0191] If the result of the judgment is negative, the target user's head bounding box is determined to be within the target area based on the position coordinates of the target user's head bounding box; the target area refers to the risk area where the rear seat user may collide with obstacles when getting out of the vehicle;

[0192] If the judgment result is yes, determine whether the area of ​​the target user's head bounding box is smaller than a preset area threshold;

[0193] If the judgment result is no, an alarm is issued.

[0194] In one embodiment, the identification unit 1002 is specifically configured to:

[0195] Identifying at least one user in a first frame of a rear-row image in a rear-row video, and marking different users in the first frame of the rear-row image with different user identifiers;

[0196] Identifying at least one user in a second frame of the rear-row image in the rear-row video, and determining, using a target tracking method, whether there is a user among the users identified in the second frame of the rear-row image that matches the user identified in the first frame of the rear-row image;

[0197] If the judgment result is yes, the user in the second frame of the rear image that can be matched with the user in the first frame of the rear image is marked with the same user ID;

[0198] The second frame of the rear-row image is updated to the first frame of the rear-row image, and each frame of the rear-row image in the rear-row video is traversed to obtain the user identifier of the user in each frame of the rear-row image.

[0199] In one embodiment, the identification unit 1002 is specifically configured to:

[0200] Identify the human body and human head in each frame of the back-row image of the back-row video, and obtain the external bounding box of the human body and the external bounding box of the human head in each frame of the back-row image;

[0201] Determine the intersection-over-union (IoU) of each person's body bounding box and each head bounding box in each frame of the back row image, and match the person corresponding to the person's body bounding box with the head bounding box whose IoU is not less than a preset IoU threshold;

[0202] The center point position of the user's body circumference frame is determined as the user's body position, and the center point position of the head circumference frame of the head matching the body is determined as the user's head position.

[0203] In one embodiment, the anti-head collision alarm device 1000 further includes a pressure change value determining unit, which is configured to:

[0204] Obtain the pressure value of each rear seat at the time corresponding to the capture moment of each rear row image frame in the rear row video within a preset time length;

[0205] Obtaining a pressure change value for each rear seat according to the pressure value of each rear seat at a corresponding moment of acquisition of each rear image frame in the rear video;

[0206] It is determined that the user corresponding to the rear seat whose pressure change value is greater than the preset pressure change value is the target user.

[0207] In one embodiment, the anti-head collision alarm device 1000 further includes a target area determination unit, which is configured to:

[0208] Obtain multiple training images, where the training images are images of the head of a rear-seat user at the boundary of the risk area when the user exits the vehicle.

[0209] Recognize the human heads in the training image to obtain multiple head bounding boxes in the training image; wherein each head corresponds to a head bounding box;

[0210] Determine the center point position of each head bounding box, and obtain the center point positions of multiple head bounding boxes;

[0211] The closed area formed by the lines connecting multiple center points is determined as the target area.

[0212] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principles are the same, which is not limited in this embodiment.

[0213] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a non-transitory computer-readable storage medium storing computer instructions, and a computer program product.

[0214] Specifically, an embodiment of the present disclosure provides an electronic device, including:

[0215] at least one processor; and

[0216] a memory communicatively connected to at least one processor; wherein,

[0217] The memory stores instructions that can be executed by at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the steps of the aforementioned anti-collision head alarm method.

[0218] An embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the steps of the aforementioned anti-collision head alarm method.

[0219] An embodiment of the present disclosure provides a computer program product, including a computer program, which implements the steps of the aforementioned anti-collision head alarm method when executed by a processor.

[0220] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, in-vehicle devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0221] like Figure 11As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1102 or a computer program loaded from a storage unit 1108 into a RAM (Random Access Memory) 1103. Various programs and data required for the operation of the device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An I / O (Input / Output) interface 1105 is also connected to the bus 1104.

[0222] Various components in device 1100 are connected to an I / O interface 1105, including an input unit 1104, such as a keyboard and mouse; an output unit 1107, such as various types of displays and speakers; a storage unit 1108, such as a magnetic disk and optical disk; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows device 1100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0223] The computing unit 1101 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), various specialized AI (Artificial Intelligence) computing chips, various computing units that run machine learning model algorithms, a DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as the head collision prevention alarm method. For example, in some embodiments, the head collision prevention alarm method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the aforementioned anti-head collision alarm method in any other appropriate manner (for example, by means of firmware).

[0224] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System on Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0225] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0226] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0227] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0228] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A head collision prevention alarm method, characterized in that: include: Get the rear video of the vehicle within a preset time period; The rear video refers to the video collected by the video collector set behind the obstacle; Identifying a user in a rear-row image of the rear-row video, and obtaining user information of the user in each frame of the rear-row image; the user information includes a body position and a user identifier of the user; and the user identifiers of different users are different in each frame of the rear-row image of the rear-row video; Filtering out the user whose body position is located in the aisle area from the back row image of the current frame in the back row video to obtain the target user; determining, based on the user identifier of the target user, whether there is a user with the same user identifier and whose body position is in the back row seat in other frame back row images in the back row video; the other frame back row images refer to images in the back row video other than the current frame back row image; If the judgment result is yes, an alarm is issued when the occlusion area in the rear row image of the current frame is larger than a preset occlusion area threshold.

2. The method according to claim 1, characterized in that The alarm is issued when the occlusion area in the rear row image of the current frame is larger than a preset occlusion area threshold, including: Obtaining the pixel value of each pixel in the back row image of the current frame; Comparing the pixel value of the pixel point with a preset pixel value, and determining the number of pixel points in the rear row image of the current frame whose pixel value is less than the preset pixel value; Determine whether the number of pixel points whose pixel values ​​are greater than the preset pixel value is greater than the preset number of pixel points; the pixel points whose pixel values ​​are greater than the preset pixel value are the pixel points of the occlusion area; If the judgment result is yes, an alarm is issued.

3. The method according to claim 2, characterized in that After determining whether the number of pixel points whose pixel values ​​are greater than the preset pixel value is greater than the preset number of pixel points, the method includes: If the judgment result is no, determining whether the target user's head bounding box is within a target area based on the position coordinates of the target user's head bounding box; the target area refers to a risk area where rear-seat users may collide with obstacles when getting off the vehicle; If the judgment result is yes, determining whether the area of ​​the target user's head circumference frame is smaller than a preset area threshold; If the judgment result is no, an alarm is issued.

4. The method according to claim 1, wherein The identifying of the user in the rear-row image of the rear-row video and obtaining the user information of the user in each frame of the rear-row image includes: Identifying at least one user in a first frame of a rear-row image in the rear-row video, and marking different user identifiers for different users in the first frame of the rear-row image; Identifying at least one user in a second frame of rear-row image in the rear-row video, and determining, using a target tracking method, whether there is a user among the users identified in the second frame of rear-row image that matches the user identified in the first frame of rear-row image; If the judgment result is yes, marking the user in the second frame of rear-row image that can be matched with the user in the first frame of rear-row image with the same user identifier; The second frame of rear-row image is updated to the first frame of rear-row image, and each frame of rear-row image in the rear-row video is traversed to obtain the user identifier of the user in each frame of rear-row image.

5. The method according to claim 3, characterized in that The identifying user information of the user in each frame of the rear-row image of the rear-row video to obtain the user information of the user in each frame of the rear-row image includes: Identify the human body and the human head in each frame of the rear-row image of the rear-row video, and obtain the external bounding box of the human body and the external bounding box of the human head in each frame of the rear-row image; Determine an intersection-and-union (IoU) ratio between each of the human body bounding boxes and each of the head bounding boxes in each frame of the rear-row image, and match the human body corresponding to the human body bounding box with the head bounding box whose IoU ratio is not less than a preset IoU threshold; The center point position of the body circumference frame of the user is determined as the body position of the user, and the center point position of the head circumference frame of the head matching the body is determined as the head position of the user.

6. The method according to claim 1, wherein Before the alarm is issued when the occlusion area in the rear row image of the current frame is larger than the preset occlusion area threshold, the method includes: Obtaining the pressure value of each rear seat at a time corresponding to the capture time of each frame of the rear image in the rear video within a preset time period; Obtaining a pressure change value for each rear seat according to a pressure value of each rear seat at a corresponding moment of acquisition of each frame of the rear image in the rear video; It is determined that the user corresponding to the rear seat whose pressure change value is greater than the preset pressure change value is the target user.

7. The method according to claim 3, characterized in that The target area calibration method includes: Acquire multiple training images, wherein the training images are images of the head of a rear-seat user at a boundary position of the risk area when the user gets out of the vehicle; Recognize the human heads in the training image to obtain a plurality of human head bounding boxes in the training image; wherein one human head corresponds to one human head bounding box; Determine the center point position of each of the head circumference frames to obtain the center point positions of multiple head circumference frames; A closed area formed by connecting lines of multiple center points is determined as a target area.

8. An anti-head collision alarm device, characterized in that: include: An acquisition unit acquires rear-seat video of a vehicle within a preset time period; The rear video refers to the video collected by a video collector arranged behind the obstacle. an identification unit, configured to identify user information of a user in each frame of the rear-row image in the rear-row video, and obtain the user information of the user in each frame of the rear-row image; the user information includes a body position and a user identifier of the user; and the user identifiers of different users are different in each frame of the rear-row image in the rear-row video; a screening unit, screening out the user whose body position is located in the aisle area from the back row image of the current frame in the back row video to obtain a target user; a judgment unit, configured to judge, based on the user identifier of the target user, whether there is a user with the same user identifier as the target user and whose body position is located in the back row seat in the back row images of other frames in the back row video; The other frame rear images refer to images in the rear video except the current frame rear image; The determination unit, if the judgment result is yes, issues an alarm when the occlusion area in the rear row image of the current frame is larger than a preset occlusion area threshold.

9. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images