Anti-collision reminding method, device and system, electronic equipment and storage medium

Abstract

The invention provides an anti-collision reminding method, device and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring a depth video stream through an image collector; acquiring a first depth value of the target user from each frame of depth image; in response to the condition that the first depth value of the target user in the current frame depth image meets a second condition, judging whether the first depth value of the target user in the current frame is smaller than the first depth value of the target user in the previous frame; when the judgment result is yes, anti-collision reminding is sent out; according to the scheme, the first depth value of the target user is acquired from each frame of depth image of the depth video stream, and the first depth value of the target user of the current frame is responded to meet the second condition. Whether the first depth value of the target user of the current frame is smaller than the first depth value of the target user of the previous frame is judged, and if the first depth value of the current frame is smaller than the first depth value of the previous frame, anti-collision reminding is performed on the target user, so that the collision risk is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an anti-collision reminder method, device, system, electronic device and storage medium. Background Art

[0002] As the target user moves toward the obstacle, the distance between them gradually decreases. When the distance between them reaches the preset danger distance, the target user is at risk of collision.

[0003] Take the passenger getting off the bus as an example. When the front row entertainment screen is not folded, the rear passengers get up from their seats and walk through the front aisle, getting closer and closer to the front row entertainment screen. If the passengers are not reminded of the entertainment screen approaching in a certain distance from the entertainment screen, there is a risk of collision. Summary of the invention

[0004] The present invention provides an anti-collision reminder method, device, system, electronic device and storage medium.

[0005] According to a first aspect of the present disclosure, there is provided an anti-collision reminder method, comprising:

[0006] Acquire a depth video stream consisting of continuous frame depth images of the target user through an image collector;

[0007] Obtain a first depth value of the target user from each frame of the depth image;

[0008] In response to the first depth value of the target user in the current frame depth image satisfying the second condition, determining whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image; the second condition is that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance;

[0009] When the judgment result is yes, an anti-collision reminder is issued; the anti-collision reminder refers to a reminder that there are obstacles in the moving direction of the target user.

[0010] In some embodiments of the present disclosure, obtaining a depth video stream consisting of continuous frame depth images of a target user through an image collector includes:

[0011] In response to a first condition, the image collector is controlled to acquire a depth video stream consisting of continuous frame depth images of the target user; the first condition refers to the target user leaving the target seat.

[0012] In some embodiments of the present disclosure, in response to the first condition, controlling the image collector to obtain a depth video stream consisting of continuous frame depth images of the target user includes:

[0013] When the pressure signal of the target seat changes from being greater than the first preset pressure value to being less than the first preset pressure value, the image collector is controlled to acquire a depth video stream consisting of continuous frame depth images of the target user.

[0014] In some embodiments of the present disclosure, before obtaining the first depth value of the target user from each frame of the depth image, the method further includes:

[0015] Obtaining a target area; the target area refers to a moving area where the target user moves from the target seat to the obstacle;

[0016] Cutting out a first area corresponding to the target area from each frame of the depth image;

[0017] Obtaining the first depth value of the target user from each frame of the depth image includes:

[0018] A first depth value of the target user is obtained from a first area corresponding to each frame of the depth image.

[0019] In some embodiments of the present disclosure, obtaining a first depth value of a target user from each depth image frame includes:

[0020] Obtain all the pixels constituting the target user from each frame of the depth image;

[0021] Determine the effective area of ​​the target user in each frame of the depth image according to the sum of the areas of all pixels constituting the target user;

[0022] Determine a depth image whose effective area is greater than a preset effective area as a valid frame depth image;

[0023] A first depth value of the target user is obtained from each valid frame depth image.

[0024] In some embodiments of the present disclosure, obtaining a first depth value of a target user from each depth image frame includes:

[0025] Acquire a first depth value of each pixel constituting the target user from each frame of the depth image;

[0026] An average value of the first depth values ​​of all pixels constituting the target user is determined as the first depth value of the target user in each frame of the depth image.

[0027] In some embodiments of the present disclosure, the image collector includes a time-of-flight TOF camera;

[0028] Obtaining the first depth value of the target user from each frame of the depth image includes:

[0029] Obtain the depth parameters of the target user in each frame of the depth image; the depth parameters include the second depth values ​​of the target user in the three directions of X, Y, and Z of the camera coordinate system; the camera coordinate system refers to a coordinate system with the center of the TOF camera as the origin and the direction perpendicular to the installation plane of the TOF camera as the Z direction;

[0030] A first depth value of the target user in the depth image is determined according to the second depth value in the Z direction in the camera coordinate system.

[0031] In some embodiments of the present disclosure, determining a first depth value of a target user in a depth image according to a second depth value in a Z direction in a camera coordinate system includes:

[0032] Get the angle value between the installation plane of the TOF camera and the vertical plane;

[0033] According to the angle value, the second depth value in the Z direction in the camera coordinate system is converted into the first depth value in the Z direction in the world coordinate system; the world coordinate system refers to a coordinate system with the center of the TOF camera as the origin and the direction perpendicular to the plane where the obstacle is located as the Z direction;

[0034] A first depth value in the Z direction in the world coordinate system is determined as a first depth value of the target user in the depth image.

[0035] In some embodiments of the present disclosure, in response to the first depth value of the target user in the current frame depth image satisfying the second condition, determining whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image includes:

[0036] Obtaining a first distance between the image collector and the obstacle;

[0037] Determine a second distance between the target user and the obstacle using the first distance and a first depth value of the target user in the current frame depth image;

[0038] In response to the second distance being less than the preset danger distance, it is determined whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image.

[0039] According to a second aspect of the present disclosure, there is provided an anti-collision reminder device, comprising:

[0040] A first acquisition module is used to acquire a depth video stream consisting of continuous frame depth images of a target user through an image collector;

[0041] A second acquisition module is used to acquire a first depth value of the target user from each frame of the depth image;

[0042] A judgment module, configured to judge whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image in response to the first depth value of the target user in the current frame depth image satisfying a second condition; the second condition being that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance;

[0043] The execution module is used to issue an anti-collision reminder when the judgment result is yes; the anti-collision reminder refers to a reminder that there are obstacles in the moving direction of the target user.

[0044] According to a third aspect of the present disclosure, there is provided an anti-collision reminder system, comprising:

[0045] Controller and image acquisition device;

[0046] The image collector is arranged in front of or behind the obstacle, and is used to collect the depth image in front of the obstacle;

[0047] A controller, used for acquiring a depth video stream consisting of continuous frame depth images of a target user through an image collector;

[0048] Obtain a first depth value of the target user from each frame of the depth image;

[0049] In response to the first depth value of the target user in the current frame depth image satisfying the second condition, determining whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image; the second condition is that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance;

[0050] When the judgment result is yes, an anti-collision reminder is issued; the anti-collision reminder refers to a reminder that there are obstacles in the moving direction of the target user.

[0051] In some embodiments of the present disclosure, the image collector includes a time-of-flight TOF camera.

[0052] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0053] at least one processor; and

[0054] a memory communicatively connected to at least one processor; wherein,

[0055] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to execute the aforementioned first aspect method.

[0056] According to a fifth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method of the first aspect.

[0057] The present disclosure provides an anti-collision reminder method, device, system, electronic device and storage medium. The method includes: obtaining a depth video stream consisting of continuous frame depth images of a target user through an image collector; obtaining a first depth value of the target user from each frame of the depth image; in response to the first depth value of the target user in the current frame of the depth image satisfying a second condition, judging whether the first depth value of the target user in the current frame of the depth image is less than the first depth value of the target user in the previous frame of the depth image; the second condition means that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance; when the judgment result is yes, issuing an anti-collision reminder; the anti-collision reminder refers to a reminder that there is an obstacle in the moving direction of the target user.

[0058] According to the scheme of the present disclosure, the first depth value of the target user from the image collector can be obtained through the depth image collected by the image collector. When the first depth value indicates that the distance between the target user and the obstacle is within the preset danger distance, it is determined whether the target user continues to move forward by whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image. If the judgment result is yes, that is, the target user is still moving forward, there is a risk that the target user will bump into the obstacle. At this time, a bump reminder is issued to the target user, thereby reducing the risk of bumping.

[0059] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0061] Figure 1 A flowchart of an anti-collision reminder method provided by an embodiment of the present disclosure;

[0062] Figure 2 A flowchart of a method for obtaining a first depth value of a target user from each valid frame depth image provided by an embodiment of the present disclosure;

[0063] Figure 3 A flowchart of a method for obtaining a first depth value of a target user from a first area corresponding to each frame of a depth image provided by an embodiment of the present disclosure;

[0064] Figure 4A flowchart of a method for determining whether a first depth value of a target user in a current frame depth image is less than a first depth value of the target user in a previous frame depth image provided by an embodiment of the present disclosure;

[0065] Figure 5 A schematic diagram of the structure of the anti-collision reminder system provided in an embodiment of the present disclosure;

[0066] Figure 6 A flowchart of an anti-collision reminder method provided for an application example of the present disclosure;

[0067] Figure 7 A schematic diagram of the structure of an anti-collision reminder device provided in an embodiment of the present disclosure;

[0068] Figure 8 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0069] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0070] In order to help those skilled in the art to better understand the technical solutions described in the embodiments of the present disclosure, the technical terms in the embodiments of the present disclosure are explained as follows before introducing the embodiments of the present disclosure.

[0071] Time of Flight (TOF) camera: records the flight time of light waves and captures images containing depth values ​​in the three directions of the camera coordinate system X, Y, and Z.

[0072] The following describes the anti-collision reminder method, device, system, electronic device and storage medium of the embodiments of the present disclosure with reference to the accompanying drawings.

[0073] As the target user moves toward the obstacle, the distance between them gradually decreases. When the distance between them reaches the preset danger distance, the target user is at risk of collision.

[0074] Take the passenger getting off the bus as an example. When the front row entertainment screen is not folded, the rear passengers get up from their seats and walk through the front aisle. They are getting closer and closer to the front row entertainment screen. If the passengers are not reminded in time that the entertainment screen is close to them within a certain distance, there is a risk of collision.

[0075] For example, in a vehicle with three rows of seats, the door is usually located in front of the second row of seats, and the entertainment screen is usually located on the top of the vehicle in front of the second row of seats for the second and third row passengers to watch. When the third row of passengers get off the vehicle, the entertainment screen is turned on and is located on the path of the third row of passengers getting off. If the third row of passengers getting off the vehicle is not reminded in time, there is a risk that the third row of passengers will bump into the entertainment screen.

[0076] In order to reduce the risk of collision for the target user, an anti-collision reminder method is provided in an embodiment of the present disclosure. The method can be applied to a vehicle equipped with a TOF camera. The executor of the method can be a separate controller or a vehicle controller of the vehicle.

[0077] Figure 1 A flowchart of an anti-collision reminder method provided in an embodiment of the present disclosure.

[0078] like Figure 1 As shown, the anti-collision reminder method provided by the embodiment of the present disclosure includes the following steps:

[0079] Step 101: Acquire a depth video stream consisting of continuous frame depth images of a target user through an image collector.

[0080] In one embodiment, the image collector is a camera capable of acquiring a depth image of the target user, such as a TOF camera. The present disclosure does not limit the specific image collector. The depth image can be an image including a grayscale image and depth values ​​in the three directions of X, Y, and Z of the camera coordinate system.

[0081] In one embodiment, the image collector is used to collect the depth image of the area corresponding to the target user's moving path. Therefore, the image collector can be set in front of or behind the obstacle, and the distance value between the target user and the obstacle can be obtained according to the depth value collected by the image collector and the distance value between the image collector and the obstacle.

[0082] In one embodiment, taking the above scenario of passengers in three rows getting off a bus as an example:

[0083] If a second-row passenger extends his arm to control the entertainment screen, the image collector can also detect an object approaching the entertainment screen, which may lead to a false reminder. To avoid false reminders, the passenger standing up from the third row of seats can be used as a triggering condition for the reminder.

[0084] Based on this, in one embodiment, a depth video stream consisting of continuous frame depth images of a target user is obtained through an image collector, including:

[0085] In response to a first condition, the image collector is controlled to acquire a depth video stream consisting of continuous frame depth images of the target user; the first condition refers to the target user leaving the target seat.

[0086] In one embodiment, the target seat may be any seat in the three rows.

[0087] In one embodiment, the change of the pressure signal on the seat may be used to determine whether the target user leaves the target seat.

[0088] Based on this, in one embodiment, in response to the first condition, controlling the image collector to obtain a depth video stream consisting of continuous frame depth images of the target user includes:

[0089] When the pressure signal of the target seat changes from being greater than the first preset pressure value to being less than the first preset pressure value, the image collector is controlled to acquire a depth video stream consisting of continuous frame depth images of the target user.

[0090] In one embodiment, a pressure signal corresponding to the target seat may be acquired through a pressure sensor installed on the target seat.

[0091] In one embodiment, one pressure sensor or multiple pressure sensors may be disposed on the target seat.

[0092] In one embodiment, if a pressure sensor is disposed on the target seat, the pressure sensor may be disposed at the center of the target seat.

[0093] In one embodiment, after the pressure sensor obtains the pressure signal of the target seat, the pressure signal can be transmitted to the controller. When the pressure signal changes from greater than a first preset pressure value to less than the first preset pressure value, the controller controls the image collector to obtain a depth video stream consisting of continuous frame depth images of the target user.

[0094] In one embodiment, the setting of the first preset pressure value can be obtained by laboratory personnel through multiple experiments or from official platform data, and the present disclosure is not limited thereto. For example, if the minimum passenger weight obtained from official platform data is 30 kg, and the items placed on the seat do not exceed 30 kg, the first preset pressure value can be set to the pressure value corresponding to 30 kg, thereby avoiding the situation where the passenger takes the items off the seat and triggers the image collector action.

[0095] Step 102, obtaining a first depth value of the target user from each frame of the depth image;

[0096] In one embodiment, the depth value of the depth image refers to the distance between the target user and the image collector in the depth image.

[0097] In one embodiment, when a second row passenger extends his arm to control the entertainment screen, the area of ​​the target user in the depth image captured by the image collector may only be the area of ​​the arm. In order to avoid this situation and further reduce the risk of false alerts, when obtaining the first depth value of the target user from each frame of the depth image, it is also necessary to determine whether the depth image is valid based on the area of ​​the target user in the depth image.

[0098] Based on this, in one embodiment, if Figure 2 As shown, step 102 includes:

[0099] Step 201, obtaining all pixel points constituting the target user from each frame of the depth image;

[0100] In one embodiment, all pixel points constituting the target user may be acquired from each frame of the depth image through a target detection algorithm.

[0101] Step 202, determining the effective area of ​​the target user in each frame of the depth image according to the sum of the areas of all pixels constituting the target user;

[0102] In one embodiment, the area of ​​each pixel constituting the target user can be obtained from the size of the depth image collected by the image collector and its resolution. Specifically, the area value of each pixel is the same, and its area value is the ratio of the size of the depth image to its resolution.

[0103] Step 203, determining that a depth image having an effective area greater than a preset effective area is a valid frame depth image;

[0104] In one embodiment, the preset effective area is a threshold for confirming that a target user has passed through, and its size can be set according to actual needs.

[0105] In one embodiment, the preset effective area is mainly used to filter the area of ​​the arm, etc. Therefore, the effective area can be calibrated according to the area of ​​the arm in the depth image.

[0106] Step 204: Acquire a first depth value of the target user from each valid frame depth image.

[0107] In one embodiment, since the depth values ​​of different parts of the target user in the depth image may be different, in order to obtain a more accurate and reliable first depth value, thereby improving the accuracy and reliability of the anti-collision reminder, the average depth value of all pixels constituting the target user may be used as the first depth value.

[0108] Based on this, in one embodiment, step 102 may further include:

[0109] Acquire a first depth value of each pixel constituting the target user from each frame of the depth image;

[0110] An average value of the first depth values ​​of all pixels constituting the target user is determined as the first depth value of the target user in each frame of the depth image.

[0111] In one embodiment, the image collector may adopt a TOF camera, which can capture an image of the target object including depth values ​​in three directions of the camera coordinate system X, Y, and Z by recording the flight time of light waves.

[0112] Based on this, in one embodiment, the image collector includes a time-of-flight TOF camera;

[0113] Accordingly,

[0114] Obtaining the first depth value of the target user from each frame of the depth image includes:

[0115] Obtain the depth parameters of the target user in each frame of the depth image; the depth parameters include the second depth values ​​of the target user in the three directions of X, Y, and Z of the camera coordinate system; the camera coordinate system refers to a coordinate system with the center of the TOF camera as the origin and the direction perpendicular to the installation plane of the TOF camera as the Z direction;

[0116] In one embodiment, only the second depth value of the target user in the Z direction of the camera coordinate system may be obtained to reduce the load of the controller and alleviate the data calculation pressure of the controller.

[0117] A first depth value of the target user in the depth image is determined according to the second depth value in the Z direction in the camera coordinate system.

[0118] In one embodiment, the TOF camera has a certain distance measurement range, so a valid interval for depth data can be set, and all pixel points outside the interval are set to invalid values.

[0119] In one embodiment, the depth data valid interval may be set according to the specification of the TOF camera.

[0120] In one embodiment, since the TOF camera may be installed at a certain tilt angle, the second depth value in the Z direction in the camera coordinate system cannot be directly used as the distance between the target user and the TOF camera.

[0121] Based on this, in one embodiment, determining a first depth value of a target user in a depth image according to a second depth value in a Z direction in a camera coordinate system includes:

[0122] Get the angle value between the installation plane of the TOF camera and the vertical plane;

[0123] In one embodiment, the installation plane of the TOF camera refers to a plane where a lens of the TOF camera is located, and the installation plane of the TOF camera is used to indicate a tilt angle of the TOF camera relative to a vertical plane.

[0124] According to the angle value, the second depth value in the Z direction in the camera coordinate system is converted into the first depth value in the Z direction in the world coordinate system; the world coordinate system refers to a coordinate system with the center of the TOF camera as the origin and the direction perpendicular to the plane where the obstacle is located as the Z direction;

[0125] In one embodiment, the plane where the obstacle is located is the plane where the entertainment screen is in the open state. Usually, the plane when the entertainment screen is in the open state is determined as a vertical plane.

[0126] A first depth value in the Z direction in the world coordinate system is determined as a first depth value of the target user in the depth image.

[0127] In one embodiment, the world coordinate system refers to a coordinate system with the center of the TOF camera as the origin and the direction perpendicular to the plane where the obstacle is located as the Z direction. The first depth value in the Z direction in the world coordinate system is the first depth value of the target user in the depth image, that is, the distance between the target user and the TOF camera. According to the distance between the target user and the TOF camera, and the distance between the TOF camera and the obstacle, the actual distance between the target user and the obstacle can be determined.

[0128] In one embodiment, since the image area collected by the image collector is relatively large, it includes not only the area corresponding to the moving path of the target user, that is, the corridor area between the second row of seats and the third row of seats, but also the second row of seats and the third row of seats. In other words, the second row of seats and the third row of seats in the depth image are not the target area. If the depth values ​​of the second row of seats and the third row of seats are obtained at the same time, the load of the controller will be increased, and the efficiency of obtaining the first depth value of the target user will be reduced.

[0129] Based on this, in one embodiment, if Figure 3 As shown, before step 102, the anti-collision reminder method further includes:

[0130] Step 301, obtaining a target area; the target area refers to a moving area where the target user moves from the target seat to the obstacle;

[0131] In one embodiment, the target area can be determined by on-site commissioning. The length of the target area is the distance between the passenger and the entertainment screen, the width is the width of the middle corridor, and the height is the distance to the vehicle roof.

[0132] Step 302, intercepting a first area corresponding to the target area from each frame of the depth image;

[0133] In one embodiment, a first area corresponding to the target area may be intercepted from each frame of the depth image by using an image segmentation algorithm.

[0134] Accordingly,

[0135] In one embodiment, obtaining a first depth value of a target user from each depth image frame includes:

[0136] A first depth value of the target user is obtained from a first area corresponding to each frame of the depth image.

[0137] In one embodiment, step 102 may be used to obtain a first depth value of the target user from a first area corresponding to each frame of the depth image.

[0138] Step 103, in response to the first depth value of the target user in the current frame depth image satisfying the second condition, determine whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image; the second condition means that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance.

[0139] In one embodiment, before step 103, the method may further include determining whether the first depth value of the target user in the current frame depth image satisfies a second condition.

[0140] In one embodiment, since the first depth value refers to the distance between the target user and the image collector, in order to determine whether the first depth value of the target user in the current frame depth image meets the second condition, it is also necessary to obtain the first distance between the image collector and the obstacle.

[0141] Based on this, in one embodiment, if Figure 4 As shown, step 103 includes:

[0142] Step 401, obtaining a first distance between the image collector and the obstacle;

[0143] In one embodiment, the image collector can be installed on the same vertical plane as the obstacle, or in front of or behind the obstacle. Preferably, due to the limitation of the installation space, the image collector can be installed behind the obstacle. The rear refers to the front of the target user's moving direction.

[0144] Step 402, determining a second distance between the target user and the obstacle by using the first distance and a first depth value of the target user in the current frame depth image;

[0145] In one embodiment, if the image collector is installed on the same plane as the obstacle, the first depth value of the target user in the current frame depth image is the second distance between the target user and the obstacle; if the image collector is installed in front of the obstacle, the sum of the first distance and the first depth value of the target user in the current frame depth image is the second distance between the target user and the obstacle; if the image collector is installed behind the obstacle, the difference between the first depth value of the target user in the current frame depth image and the first distance is the second distance between the target user and the obstacle.

[0146] Step 403 : in response to the second distance being less than the preset danger distance, determining whether the first depth value of the target user in the current depth image is less than the first depth value of the target user in the previous depth image.

[0147] In one embodiment, it is determined whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image. If so, it indicates that the target user is moving and approaching an obstacle.

[0148] Step 104, when the judgment result is yes, an anti-collision reminder is issued; the anti-collision reminder refers to a reminder that there is an obstacle in the moving direction of the target user.

[0149] In one embodiment, the target users may be three-row passengers, the obstacle may be an entertainment screen, and the anti-collision reminder may be a voice broadcast reminder, a light flashing reminder, or the like.

[0150] The anti-collision reminder method provided by the embodiment of the present disclosure includes: obtaining a depth video stream consisting of continuous frame depth images of a target user through an image collector; obtaining a first depth value of the target user from each frame of the depth image; in response to the first depth value of the target user in the current frame of the depth image satisfying a second condition, judging whether the first depth value of the target user in the current frame of the depth image is less than the first depth value of the target user in the previous frame of the depth image; the second condition means that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance; when the judgment result is yes, issuing an anti-collision reminder; the anti-collision reminder refers to a reminder that there is an obstacle in the moving direction of the target user.

[0151] According to the solution disclosed in this disclosure:

[0152] First, through the depth image collected by the image collector, the first depth value of the target user from the image collector can be obtained. When the first depth value indicates that the distance between the target user and the obstacle is within the preset danger distance, it is determined whether the target user continues to move forward by whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image. If the judgment result is yes, that is, the target user is still moving forward, there is a risk that the target user will bump into the obstacle. At this time, a bump reminder is issued to the target user, thereby reducing the risk of bumping.

[0153] Secondly, the change of the target seat pressure signal can be used to trigger the image collector to obtain a depth video stream consisting of continuous frame depth images of the target user, thereby reducing the risk of false alerts.

[0154] Again, if the area of ​​the object in the target area reaches the threshold, it means that a passenger has passed by. At this time, an anti-collision reminder is issued to further reduce the occurrence of false detection and improve the robustness of the algorithm.

[0155] Again, based on the angle value between the installation plane of the TOF camera and the vertical plane, the second depth value in the Z direction in the camera coordinate system is converted into the first depth value in the Z direction in the world coordinate system, thereby improving the reliability of the anti-collision reminder.

[0156] Thirdly, the first depth value of the target user in each frame of the depth image is represented by the average value of the first depth values ​​of all the pixel points of the target user, thereby improving the accuracy of the anti-collision reminder.

[0157] The embodiment of the present disclosure also provides an anti-collision reminder system. The anti-collision reminder system provided by the embodiment of the present disclosure can be applied to a vehicle equipped with an entertainment screen, which is installed behind the door. When the rear passengers get off the vehicle with the entertainment screen turned on, there is a risk of bumping into the entertainment screen. The anti-collision system provided by the embodiment of the present disclosure is used to remind the rear passengers that there is an entertainment screen in front of them, thereby reducing the risk of bumping of the rear passengers.

[0158] like Figure 5 As shown, the anti-collision reminder system provided by the embodiment of the present disclosure includes a controller 501 and an image collector 502;

[0159] The image collector 502 is arranged in front of or behind the obstacle, and is used to collect the depth image in front of the obstacle;

[0160] The controller 501 is used to obtain a depth video stream consisting of continuous frame depth images of a target user through an image collector;

[0161] Obtain a first depth value of the target user from each frame of the depth image;

[0162] In response to the first depth value of the target user in the current frame depth image satisfying the second condition, determining whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image; the second condition is that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance;

[0163] When the judgment result is yes, an anti-collision reminder is issued; the anti-collision reminder refers to a reminder that there are obstacles in the moving direction of the target user.

[0164] In one embodiment, the image collector 502 includes a time-of-flight TOF camera.

[0165] Applied to the above-mentioned anti-collision reminder system, the application example disclosed herein also provides an anti-collision reminder method.

[0166] like Figure 6 As shown, Figure 6 A flowchart of an anti-collision reminder method provided for an application example of the present disclosure, the anti-collision reminder method comprising the following steps:

[0167] Step 601, collecting a pressure signal of a target seat through a pressure sensor disposed on the target seat;

[0168] In one embodiment, the target seat is a rear seat.

[0169] Step 602, determining whether the pressure signal changes from being greater than a first preset pressure value to being less than the first preset pressure value;

[0170] In one embodiment, because the behavior of the target user leaving the target seat is a non-continuous behavior, if the judgment result is yes, and the time when the target user gets off the car is the first time period, then when the judgment result is yes, the pressure signal may no longer be collected in the first time period thereafter. The first time period may be obtained through a limited number of experiments, such as 10 seconds.

[0171] If the judgment result is yes, then go to step 603;

[0172] If the judgment result is no, return to step 601;

[0173] Step 603, controlling the TOF camera to collect a depth video stream consisting of continuous frame depth images of the target user;

[0174] Step 604, judging whether each frame of the depth image is a valid frame of the depth image according to the area of ​​the target user in each frame of the depth video stream;

[0175] If the judgment result is yes, then go to step 605;

[0176] If the judgment result is no, return to step 603;

[0177] Step 605, obtaining a second depth value of the target user in the Z direction in the camera coordinate system in the first area of ​​the effective frame depth image;

[0178] In one embodiment, the first area refers to an area corresponding to the target area in the effective frame depth image; the target area refers to a moving area where the target user moves from the target seat to the obstacle;

[0179] In one embodiment, the target area can be determined by on-site debugging. For example, the length of the target area is the distance between the target seat and the entertainment screen, which can be 800 mm; the width of the target area is the width of the middle corridor; and the height of the target area is the height of the vehicle roof, which can be 280 mm.

[0180] Step 606, converting the camera coordinate system into the world coordinate system according to the angle value between the installation plane of the TOF camera and the vertical plane;

[0181] Step 607, determining the depth value of the target image in the Z direction in the world coordinate system as a first depth value;

[0182] In one embodiment, the first depth value refers to an average value of the first depth values ​​of all pixels constituting the target user;

[0183] Step 608, determining whether the first depth value of the target user in the current frame depth image satisfies the second condition;

[0184] In one embodiment, the second condition means that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance;

[0185] If the judgment result is yes, then go to step 609;

[0186] If the judgment result is no, return to step 605;

[0187] Step 609, determining whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image;

[0188] If yes, proceed to step 610;

[0189] If not, return to step 605;

[0190] Step 610, issue an anti-collision reminder.

[0191] Corresponding to the above anti-collision reminder method, the present invention also provides an anti-collision reminder device. Since the device embodiment of the present invention corresponds to the above method embodiment, the details not disclosed in the device embodiment can be referred to the above method embodiment, and will not be repeated in the present invention.

[0192] Figure 7 A schematic diagram of the structure of an anti-collision reminder device provided in an embodiment of the present disclosure is shown in FIG. Figure 7 As shown, the anti-collision reminder device 700 includes:

[0193] The first acquisition module 701 is used to acquire a depth video stream consisting of continuous frame depth images of a target user through an image collector;

[0194] A second acquisition module 702 is used to acquire a first depth value of a target user from each frame of the depth image;

[0195] The judging module 703 is configured to judge whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image in response to the first depth value of the target user in the current frame depth image satisfying a second condition; the second condition is that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance;

[0196] The execution module 704 is used to issue an anti-collision reminder when the judgment result is yes; the anti-collision reminder refers to a reminder that there is an obstacle in the moving direction of the target user.

[0197] In one embodiment, the first acquisition module 701 is specifically configured to:

[0198] In response to a first condition, the image collector is controlled to collect a depth video stream consisting of continuous frame depth images of the target user; the first condition refers to the target user leaving the target seat.

[0199] In one embodiment, the control module is further configured to:

[0200] When the pressure signal of the target seat changes from being greater than the first preset pressure value to being less than the first preset pressure value, the image collector is controlled to acquire a depth video stream consisting of continuous frame depth images of the target user.

[0201] In one embodiment, the anti-collision reminder device 700 further includes a third acquisition module, which is used to:

[0202] Obtaining a target area; the target area refers to a moving area where the target user moves from the target seat to the obstacle;

[0203] Cutting out a first area corresponding to the target area from each frame of the depth image;

[0204] Accordingly, the second acquisition module 702 is specifically configured to:

[0205] A first depth value of the target user is obtained from a first area corresponding to each frame of the depth image.

[0206] In one embodiment, the second acquisition module 702 is specifically configured to:

[0207] Obtain all the pixels constituting the target user from each frame of the depth image;

[0208] Determine the effective area of ​​the target user in each frame of the depth image according to the sum of the areas of all pixels constituting the target user;

[0209] Determine a depth image whose effective area is greater than a preset effective area as a valid frame depth image;

[0210] A first depth value of the target user is obtained from each valid frame depth image.

[0211] In one embodiment, the second acquisition module 702 is specifically configured to:

[0212] Acquire a first depth value of each pixel constituting the target user from each frame of the depth image;

[0213] An average value of the first depth values ​​of all pixels constituting the target user is determined as the first depth value of the target user in each frame of the depth image.

[0214] In one embodiment, the image collector includes a time-of-flight TOF camera;

[0215] Accordingly,

[0216] In one embodiment, the second acquisition module 702 is specifically configured to:

[0217] Obtain the depth parameters of the target user in each frame of the depth image; the depth parameters include the second depth values ​​of the target user in the three directions of X, Y, and Z of the camera coordinate system; the camera coordinate system refers to a coordinate system with the center of the TOF camera as the origin and the direction perpendicular to the installation plane of the TOF camera as the Z direction;

[0218] A first depth value of the target user in the depth image is determined according to the second depth value in the Z direction in the camera coordinate system.

[0219] In one embodiment, the second acquisition module 702 is specifically configured to:

[0220] Get the angle value between the installation plane of the TOF camera and the vertical plane;

[0221] According to the angle value, the second depth value in the Z direction in the camera coordinate system is converted into the first depth value in the Z direction in the world coordinate system; the world coordinate system refers to a coordinate system with the center of the TOF camera as the origin and the direction perpendicular to the plane where the obstacle is located as the Z direction;

[0222] A first depth value in the Z direction in the world coordinate system is determined as a first depth value of the target user in the depth image.

[0223] In one embodiment, the determination module 703 is specifically configured to:

[0224] Obtaining a first distance between the image collector and the obstacle;

[0225] Determine a second distance between the target user and the obstacle using the first distance and a first depth value of the target user in the current frame depth image;

[0226] In response to the second distance being less than the preset danger distance, it is determined whether the first depth value of the target user in the current frame depth image is less than the first depth value of the target user in the previous frame depth image.

[0227] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0228] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a non-transitory computer-readable storage medium storing computer instructions.

[0229] Specifically, an embodiment of the present disclosure provides an electronic device, including:

[0230] at least one processor; and

[0231] a memory communicatively connected to at least one processor; wherein,

[0232] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can perform the steps of the aforementioned anti-collision reminder method.

[0233] An embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the steps of the aforementioned anti-collision reminder method.

[0234] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, vehicle-mounted devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0235] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 802 or a computer program loaded from a storage unit 808 to a RAM (Random Access Memory) 803. In RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An I / O (Input / Output) interface 805 is also connected to the bus 804.

[0236] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 804, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0237] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the anti-collision reminder method. For example, in some embodiments, the anti-collision reminder method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or the communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the aforementioned anti-collision reminder method in any other appropriate manner (for example, by means of firmware).

[0238] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0239] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0240] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0241] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0242] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for preventing collisions, characterized in that: include: Acquire a depth video stream consisting of continuous frame depth images of the target user through an image collector; Acquire a first depth value of the target user from each frame of the depth image; In response to the first depth value of the target user in the depth image of the current frame satisfying a second condition, determining whether the first depth value of the target user in the depth image of the current frame is less than the first depth value of the target user in the depth image of the previous frame; the second condition being that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance; When the judgment result is yes, an anti-collision reminder is issued; the anti-collision reminder refers to a reminder that there is an obstacle in the moving direction of the target user.

2. The method according to claim 1, characterized in that The method of obtaining a depth video stream consisting of continuous frame depth images of a target user through an image collector includes: In response to a first condition, the image collector is controlled to acquire a depth video stream consisting of continuous frame depth images of the target user; the first condition refers to the target user leaving the target seat.

3. The method according to claim 2, characterized in that In response to the first condition, controlling the image collector to obtain a depth video stream consisting of continuous frame depth images of the target user includes: When the pressure signal of the target seat changes from being greater than the first preset pressure value to being less than the first preset pressure value, the image collector is controlled to acquire a depth video stream consisting of continuous frame depth images of the target user.

4. The method according to claim 1, characterized in that: Before acquiring the first depth value of the target user from each frame of the depth image, the method further includes: Acquire a target area; the target area refers to a moving area where the target user moves from the target seat to the obstacle; Intercepting a first area corresponding to the target area from each frame of the depth image; The obtaining a first depth value of the target user from each frame of the depth image includes: A first depth value of the target user is obtained from a first area corresponding to each frame of the depth image.

5. The method according to claim 1, characterized in that The obtaining a first depth value of the target user from each frame of the depth image includes: Acquire all pixel points constituting the target user from each frame of the depth image; Determine the effective area of ​​the target user in each frame of the depth image according to the sum of the areas of all the pixels constituting the target user; Determine that the depth image having the effective area greater than a preset effective area is a valid frame depth image; A first depth value of the target user is obtained from each frame of the valid frame depth image.

6. The method according to claim 1, characterized in that The obtaining a first depth value of the target user from each frame of the depth image includes: Acquire a first depth value of each pixel constituting the target user from each frame of the depth image; An average value of the first depth values ​​of all pixels constituting the target user is determined as the first depth value of the target user in each frame of the depth image.

7. The method according to claim 1, characterized in that The image collector includes a time-of-flight TOF camera; The obtaining a first depth value of the target user from each frame of the depth image includes: Obtaining the depth parameters of the target user in each frame of the depth image; the depth parameters include the second depth values ​​of the target user in the three directions of X, Y, and Z of the camera coordinate system; the camera coordinate system refers to a coordinate system with the center of the TOF camera as the origin and the direction perpendicular to the installation plane of the TOF camera as the Z direction; A first depth value of the target user in the depth image is determined according to the second depth value in the Z direction in the camera coordinate system.

8. The method according to claim 3, characterized in that The determining, according to the second depth value in the Z direction in the camera coordinate system, a first depth value of the target user in the depth image includes: Obtaining the angle value of the angle between the installation plane of the TOF camera and the vertical plane; According to the angle value, converting the second depth value in the Z direction in the camera coordinate system into a first depth value in the Z direction in the world coordinate system; the world coordinate system refers to a coordinate system with the center of the TOF camera as the origin and the direction perpendicular to the plane where the obstacle is located as the Z direction; A first depth value in the Z direction in the world coordinate system is determined as a first depth value of the target user in the depth image.

9. The method according to claim 1, characterized in that: In response to the first depth value of the target user in the depth image of the current frame satisfying the second condition, determining whether the first depth value of the target user in the depth image of the current frame is less than the first depth value of the target user in the depth image of the previous frame, comprises: Acquire a first distance between the image collector and the obstacle; Determine a second distance between the target user and the obstacle by using the first distance and a first depth value of the target user in the depth image of the current frame; In response to the second distance being smaller than the preset danger distance, it is determined whether the first depth value of the target user in the depth image of the current frame is smaller than the first depth value of the target user in the depth image of the previous frame.

10. An anti-collision reminder device, characterized in that: include: A first acquisition module is used to acquire a depth video stream consisting of continuous frame depth images of a target user through an image collector; A second acquisition module, used to acquire a first depth value of the target user from each frame of the depth image; A judgment module, configured to judge whether the first depth value of the target user in the depth image of the current frame is less than the first depth value of the target user in the depth image of the previous frame in response to the first depth value of the target user in the depth image of the current frame satisfying a second condition; the second condition being that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance; The execution module is used to issue an anti-collision reminder when the judgment result is yes; the anti-collision reminder refers to a reminder to remind the target user that there is an obstacle in the moving direction.

11. An anti-collision reminder system, characterized in that: Including a controller and an image collector; The image collector is arranged in front of or behind the obstacle, and is used to collect a depth image in front of the obstacle; The controller is used to obtain a depth video stream consisting of continuous frame depth images of the target user through an image collector; Acquire a first depth value of the target user from each frame of the depth image; In response to the first depth value of the target user in the depth image of the current frame satisfying a second condition, determining whether the first depth value of the target user in the depth image of the current frame is less than the first depth value of the target user in the depth image of the previous frame; the second condition being that the first depth value of the target user indicates that the distance between the target user and the obstacle is within a preset danger distance; When the judgment result is yes, an anti-collision reminder is issued; the anti-collision reminder refers to a reminder that there is an obstacle in the moving direction of the target user.

12. The system according to claim 11, characterized in that The image collector includes a time-of-flight TOF camera.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

Images