Anti-collision reminding method, device and system, electronic equipment and storage medium
By acquiring and analyzing depth images and non-deep images, combining the judgment of depth values, width values and head area values, the unnecessary anti-collision reminder problem caused by misjudgment in the prior art is solved, and a more accurate reminder effect is achieved.
Patent Information
- Application Number
- CN202311519551.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-15
- Publication Date
- 2025-05-16
AI Technical Summary
The existing anti-collision reminder methods cannot effectively determine whether the target approaching the entertainment screen is the passenger's torso, resulting in misjudgment and issuing unnecessary anti-collision reminders.
By acquiring the depth image and non-deep image of the target user, it is determined whether the depth value and width value of the first part of the target user meet specific conditions, and whether an anti-collision reminder is issued based on the head area value.
It effectively avoids unnecessary anti-collision reminders due to misjudgment, and improves the accuracy and reliability of reminders.
Smart Images
Figure CN120014607A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to an anti-collision reminder method, device, electronic device, storage medium and vehicle. Background Art
[0002] In vehicles with entertainment screens, when the entertainment screen is turned on, it may collide with the driver and passengers. For example, in vehicles with three rows of seats, the entertainment screen is usually set in front of the upper second row of seats for the second and third row passengers to watch. When the torsos of the second and third row passengers move and the movement trajectory interferes with the entertainment screen, the torsos of the passengers may collide with the entertainment screen.
[0003] In the related anti-collision reminder method, a time of flight (TOF) camera is usually set near the entertainment screen, and the TOF camera is used to collect the depth image of the passenger under the entertainment screen in real time. Then, the distance between the passenger and the entertainment screen is determined based on the depth image. When the distance between the passenger and the entertainment screen reaches a preset dangerous distance, an anti-collision reminder is issued, thereby reducing the risk of passenger collision.
[0004] However, since the relevant anti-collision reminder method does not include a method to determine whether the target approaching the entertainment screen (obstacle) is the passenger's torso, when the second-row passenger reaches out to control the entertainment screen, the relevant anti-collision reminder method will usually misjudge the passenger's arms and other limbs as the passenger's torso, thereby causing an anti-collision reminder to be issued due to misjudgment. Summary of the invention
[0005] The present disclosure provides an anti-collision reminder method, device, electronic equipment, storage medium and vehicle.
[0006] The first aspect of the present disclosure provides an anti-collision reminder method, the method comprising:
[0007] Acquire a depth image and a non-depth image of the target user at the same time; the depth image refers to a depth image of the target user located below the obstacle acquired by the first image collector; the non-depth image refers to a non-depth image including the head of the target user located below the obstacle acquired by the second image collector;
[0008] Acquire a first depth value and a width value of a first part of the target user from the depth image, and acquire a head area value of the target user from the non-depth image;
[0009] In response to the first depth value of the first part of the target user satisfying the first condition, determining whether the width value and the head area value of the first part of the target user satisfy the second condition; the first condition refers to the first depth value indicating that the distance between the first part of the target user and the obstacle is within a preset danger distance; the second condition refers to width thresholds corresponding to different head area values, wherein the larger the head area value, the smaller the corresponding width threshold;
[0010] In response to the width value of the first part and the head area value of the target user satisfying the second condition, an anti-collision reminder is issued.
[0011] In some embodiments of the present disclosure, acquiring a first depth value of a first part of the target user from the depth image includes:
[0012] Dividing the depth image into a plurality of grids;
[0013] Taking the average depth value of all pixels of each grid in the plurality of grids as the second depth value of the corresponding grid;
[0014] The part corresponding to the smallest depth value among the plurality of second depth values is determined as the first part of the target user, and the smallest depth value among the plurality of second depth values is determined as the first depth value of the first part of the target user.
[0015] In some embodiments of the present disclosure, acquiring a first depth value of a first part of the target user from the depth image includes:
[0016] Acquire a depth parameter of a target user in the depth image; the depth parameter includes a third depth value of the target user in the Y direction of the camera coordinate system;
[0017] Obtaining an angle value of an angle between an installation plane of the first image collector and a horizontal plane;
[0018] According to the angle value, converting the third depth value in the Y direction in the camera coordinate system into the first depth value in the Y direction in the world coordinate system; the world coordinate system refers to a coordinate system with the center of the first image collector as the origin and the direction perpendicular to the plane where the obstacle is located as the Y direction;
[0019] Determine a minimum value of the first depth values in the Y direction in the world coordinate system as the first depth value of the first part of the target user.
[0020] In some embodiments of the present disclosure, acquiring a first depth value of a first part of the target user from the depth image includes:
[0021] Determine whether the depth image is a valid image, where the valid image refers to a depth image whose error is within a preset valid error range;
[0022] If the judgment result is yes, a first depth value of the first part of the target user is obtained from the depth image.
[0023] In some embodiments of the present disclosure, the determining whether the depth image is a valid image includes:
[0024] Obtain fourth depth values of two preset reference points in the Z direction in the camera coordinate system; the fourth depth values of the two reference points in the Z direction in the camera coordinate system are the same;
[0025] Obtaining an average depth value of all pixels in a first preset area in the depth image in the Z direction in the camera coordinate system; the first preset area refers to an area in the depth image centered on the midpoint of a line connecting two reference points;
[0026] The average depth value is subtracted from the fourth depth value, and whether the depth image is a valid image is determined based on whether the difference is within a preset effective error range.
[0027] In some embodiments of the present disclosure, acquiring the width value of the first part of the target user from the depth image includes:
[0028] Determine at least one first grid in the depth image that is associated with a first part of the target user in an X direction in the camera coordinate system;
[0029] The sum of the depth values of all the first grids in the X direction of the camera coordinate system is determined to be the width value of the first part of the target user.
[0030] A second aspect of the present disclosure provides an anti-collision reminder device, the device comprising:
[0031] A first acquisition module is used to acquire a depth image and a non-depth image of the target user at the same time; the depth image refers to a depth image of the target user located below the obstacle acquired by the first image collector; the non-depth image refers to a non-depth image including the head of the target user located below the obstacle acquired by the second image collector;
[0032] a second acquisition module, configured to acquire a first depth value and a width value of a first part of the target user from the depth image, and to acquire a head area value of the target user from the non-depth image; the first part refers to a part of the target user in the depth image with the smallest depth value;
[0033] a judgment module, configured to judge whether the width value of the first part of the target user satisfies a second condition by using the width value of the first part of the target user and the head area value in response to the first depth value of the first part of the target user satisfying a first condition; the first condition refers to that the first depth value indicates that the distance between the first part of the target user and an obstacle is within a preset danger distance; the second condition refers to width thresholds corresponding to different head area values, wherein the larger the head area value, the smaller the corresponding width threshold;
[0034] The reminder module is used to issue an anti-collision reminder in response to the width value of the first part and the head area value of the target user satisfying the second condition.
[0035] A third aspect of the present disclosure provides an electronic device, including:
[0036] at least one processor; and
[0037] a memory communicatively connected to the at least one processor; wherein,
[0038] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect of the present disclosure.
[0039] An embodiment of the fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect of the present disclosure.
[0040] An embodiment of the fifth aspect of the present disclosure provides a vehicle, comprising at least one of the device described in the second aspect of the present disclosure, the electronic device described in the third aspect of the present disclosure, and the non-transitory computer-readable storage medium described in the fourth aspect of the present disclosure.
[0041] The present disclosure provides an anti-collision reminder method, device, electronic device, storage medium and vehicle, the method comprising: obtaining a depth image and a non-depth image of a target user at the same time; the depth image refers to a depth image of the target user located below an obstacle through a first image collector; the non-depth image refers to a non-depth image including the head of the target user located below the obstacle and collected by a second image collector; obtaining a first depth value and a width value of a first part of the target user from the depth image, and obtaining a head area value of the target user from the non-depth image; in response to the first depth value of the first part of the target user satisfying a first condition, determining whether the width value and the head area value of the first part of the target user satisfy a second condition; the first condition refers to the first depth value indicating that the distance between the first part of the target user and the obstacle is within a preset danger distance; the second condition refers to width thresholds corresponding to different head area values, wherein the larger the head area value, the smaller the corresponding width threshold; in response to the width value and the head area value of the first part of the target user satisfying the second condition, issuing an anti-collision reminder.
[0042] According to the scheme provided by the present disclosure, the first depth value and width value of the first part of the target user can be obtained through the depth image collected by the first image collector, and the head area value of the target user can be obtained through the non-depth image collected by the second image collector. When the first depth value of the first part of the target user indicates that the distance between the first part of the target user and the obstacle is within the preset danger distance, according to the imaging principle that the closer the target user is to the second image collector, the larger the head area value, when the head area value of the target user and the width value of the first part of the target user meet the second condition, an anti-collision reminder is issued. Among them, since the second condition refers to the width threshold corresponding to different head area values, that is, when the head area value of the target user is small, it means that the distance between the head of the target user and the obstacle is far, which further means that the first part corresponding to the first depth value that meets the first condition at this time is more likely to be a limb part of the target user, rather than a torso part. At this time, a larger width threshold can effectively avoid the occurrence of an anti-collision reminder due to misjudgment.
[0043] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0045] Figure 1 A flowchart of the anti-collision reminder method provided by an embodiment of the present disclosure;
[0046] Figure 2 A schematic diagram of a flow chart of a first method for obtaining a first depth value provided by an embodiment of the present disclosure;
[0047] Figure 3 A schematic diagram of a flow chart of a second method for obtaining a first depth value provided by an embodiment of the present disclosure;
[0048] Figure 4 A schematic diagram of a flow chart of a third method for obtaining a first depth value provided by an embodiment of the present disclosure;
[0049] Figure 5 A flowchart of the anti-collision reminder method provided by the application example of the present disclosure;
[0050] Figure 6 A schematic diagram of the structure of the anti-collision reminder device provided in an embodiment of the present disclosure;
[0051] Figure 7 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION
[0052] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0053] In vehicles with entertainment screens, when the entertainment screen is turned on, it may collide with the driver and passengers. For example, in vehicles with three rows of seats, the entertainment screen is usually set in front of the upper second row of seats for the second and third row passengers to watch. When the torsos of the second and third row passengers move and the movement trajectory interferes with the entertainment screen, the torsos of the passengers may collide with the entertainment screen.
[0054] In the related anti-collision reminder method, a time of flight (TOF) camera is usually set near the entertainment screen, and the TOF camera is used to collect the depth image of the passenger under the entertainment screen in real time. Then, the distance between the passenger and the entertainment screen is determined based on the depth image. When the distance between the passenger and the entertainment screen reaches a preset dangerous distance, an anti-collision reminder is issued, thereby reducing the risk of passenger collision.
[0055] However, since the relevant anti-collision reminder method does not include a method to determine whether the target approaching the entertainment screen (obstacle) is the passenger's torso, when the second-row passenger reaches out to control the entertainment screen, the relevant anti-collision reminder method will usually misjudge the passenger's arms and other limbs as the passenger's torso, thereby causing an anti-collision reminder to be issued due to misjudgment.
[0056] In order to avoid the situation where an anti-collision reminder is issued due to misjudgment, the embodiments of the present disclosure provide an anti-collision reminder method, device, electronic device, storage medium and vehicle.
[0057] In one embodiment, the anti-collision reminder method provided by the embodiment of the present disclosure can be applied to a vehicle that is equipped with both a depth camera and a non-depth camera. The executor of the method can be a separate controller or a vehicle controller of the vehicle.
[0058] like Figure 1 As shown, the anti-collision reminder method provided by the embodiment of the present disclosure includes the following steps:
[0059] Step 101, obtaining a depth image and a non-depth image of a target user at the same time; the depth image refers to a depth image of the target user located below an obstacle acquired by a first image collector; the non-depth image refers to a non-depth image including the head of the target user located below the obstacle acquired by a second image collector;
[0060] In one embodiment, the first image collector may be a depth camera, such as a TOF camera;
[0061] In one embodiment, the first image collector may be disposed in front of or behind the obstacle to collect a depth image of the target user under the obstacle;
[0062] In one embodiment, the depth image may be an image including a grayscale image and depth values in three directions of X, Y, and Z of a camera coordinate system;
[0063] In one embodiment, the distance value between the target user and the obstacle can be obtained according to the depth value of the depth image acquired by the first image collector and the distance value between the first image collector and the obstacle.
[0064] In one embodiment, the second image collector may be a non-depth camera, such as an infrared camera;
[0065] In one embodiment, the second image collector may be disposed in front of or behind the obstacle to collect a non-depth image of the target user under the obstacle.
[0066] Step 102, obtaining a first depth value and a width value of a first part of the target user from the depth image, and obtaining a head area value of the target user from the non-depth image;
[0067] In one embodiment, the first part refers to a part of the target user in the depth image where the depth value is the smallest;
[0068] In one embodiment, the first depth value refers to the distance between the target user and the image collector in the depth image.
[0069] In one embodiment, the first depth value refers to a depth value in the Y direction in a camera coordinate system; the camera coordinate system refers to a coordinate system with the first image collector as the origin and a direction perpendicular to the installation plane of the first image collector as the Y direction;
[0070] In one embodiment, the width value refers to a width value of the target for the first part, and the width value of the first part can be obtained from a width value of the first part in the X direction of the camera coordinate system.
[0071] In one embodiment, obtaining the head area value of the target user from the non-depth image includes:
[0072] First, a target recognition model, such as a face recognition model or a head recognition model, is used to recognize the head of the target user from the non-depth image; secondly, the head area value of the target user is calculated based on the recognized head of the target user.
[0073] Step 103, in response to the first depth value of the first part of the target user satisfying the first condition, determining whether the width value and the head area value of the first part of the target user satisfy the second condition; the first condition refers to the first depth value indicating that the distance between the first part of the target user and the obstacle is within a preset danger distance; the second condition refers to width thresholds corresponding to different head area values, wherein the larger the head area value, the smaller the corresponding width threshold;
[0074] In one embodiment, when the first depth value indicates that the distance between the first part of the target user and the obstacle is within the preset danger distance, it means that the torso or limbs of the target user are close to the entertainment screen and there is a risk of bumping the entertainment screen. At this time, there is a need to issue an anti-bumping reminder. However, in order to reduce the probability of misjudgment, it is also necessary to filter the case where the target user is physically operating the entertainment screen.
[0075] In one embodiment, according to the imaging principle, the farther the target user's head is from the second image collector, the larger the target user's head area value is. That is, when the first depth value satisfies the first condition, if the head area is larger, it is more likely that the target user uses his limbs to operate the entertainment screen. Therefore, the width threshold at this time can be set larger to filter this situation.
[0076] Step 104: In response to the width value of the first part and the head area value of the target user satisfying the second condition, an anti-collision reminder is issued.
[0077] In one embodiment, the anti-collision reminder can be a voice broadcast reminder, a light flashing reminder, or a combination of a voice broadcast reminder and a light flashing reminder.
[0078] The present disclosure provides an anti-collision reminder method, comprising: acquiring a depth image and a non-depth image of a target user at the same time; the depth image refers to a depth image of the target user located below an obstacle through a first image collector; the non-depth image refers to a non-depth image including the head of the target user located below the obstacle and collected by a second image collector; acquiring a first depth value and a width value of a first part of the target user from the depth image, and acquiring a head area value of the target user from the non-depth image; the first part refers to a part of the target user with the smallest depth value in the depth image; in response to the first depth value of the first part of the target user satisfying a first condition, using the width value of the first part of the target user and the head area value to judge whether the width value of the first part of the target user satisfies a second condition; the first condition refers to the first depth value indicating that the distance between the first part of the target user and the obstacle is within a preset danger distance; the second condition refers to width thresholds corresponding to different head area values, wherein the larger the head area value, the smaller the corresponding width threshold; in response to the width value and the head area value of the first part of the target user satisfying the second condition, issuing an anti-collision reminder.
[0079] According to the scheme provided by the present disclosure, the first depth value and width value of the first part of the target user can be obtained through the depth image collected by the first image collector, and the head area value of the target user can be obtained through the non-depth image collected by the second image collector. When the first depth value of the first part of the target user indicates that the distance between the first part of the target user and the obstacle is within the preset danger distance, according to the imaging principle that the closer the target user is to the second image collector, the larger the head area value, when the head area value of the target user and the width value of the first part of the target user meet the second condition, an anti-collision reminder is issued. Among them, since the second condition refers to the width threshold corresponding to different head area values, that is, when the head area value of the target user is small, it means that the distance between the head of the target user and the obstacle is far, which further means that the first part corresponding to the first depth value that meets the first condition at this time is more likely to be a limb part of the target user, rather than a torso part. At this time, a larger width threshold can effectively avoid the occurrence of an anti-collision reminder due to misjudgment.
[0080] In one embodiment, if Figure 2As shown, obtaining a first depth value of a first part of the target user from the depth image includes the following steps:
[0081] Step 201, dividing the depth image into a plurality of grids;
[0082] In one embodiment, the depth image may be divided into a plurality of grids using an image gridding method;
[0083] In one embodiment, the number of the grids may be twenty or thirty, which is not limited in the present disclosure. Preferably, the depth image may be evenly divided into twenty grids according to the size of the depth image.
[0084] Step 202, taking the average depth value of all pixels of each grid in the plurality of grids as the second depth value of the corresponding grid;
[0085] In one embodiment, since the depth image is a depth image of the target user, the depth values of the same part of the target user may also be different. Taking the depth value in the Y direction of the camera coordinate system as an example, when the target user uses a limb to operate the entertainment screen, such as an arm, when the arm is raised, the depth value of the wrist is obviously smaller than the depth value of the elbow. Therefore, the average depth value can more accurately reflect the first depth value of the first part of the target user, thereby providing a more reliable anti-collision reminder.
[0086] Step 203: determine the part corresponding to the smallest depth value among the plurality of second depth values as the first part of the target user, and determine the smallest depth value among the plurality of second depth values as the first depth value of the first part of the target user.
[0087] In one embodiment, since the depth value refers to the distance between the target user and the first image collector, the part corresponding to the smallest depth value in the second depth values is the first part of the target user.
[0088] In one embodiment, if Figure 3 As shown, the obtaining a first depth value of a first part of the target user from the depth image includes:
[0089] Step 301, obtaining a depth parameter of a target user in the depth image; the depth parameter includes a third depth value of the target user in the Y direction of the camera coordinate system;
[0090] In one embodiment, the camera coordinate system refers to a coordinate system with the center of the first image collector as the origin and the direction perpendicular to the installation plane of the first image collector as the Y direction;
[0091] In one embodiment, the installation plane of the first image collector refers to the plane where the collection lens of the first image collector is located;
[0092] In one embodiment, the Y direction refers to a downward direction perpendicular to the installation plane of the first image collector.
[0093] Step 302, obtaining the angle value of the angle between the installation plane of the first image collector and the horizontal plane;
[0094] In one embodiment, the angle value between the installation plane of the first image collector and the horizontal plane can reflect the installation inclination of the first image collector. According to this angle value, the camera coordinate system can be converted into the world coordinate system.
[0095] Step 303: converting the third depth value in the Y direction in the camera coordinate system into the first depth value in the Y direction in the world coordinate system according to the angle value; the world coordinate system refers to a coordinate system with the center of the first image collector as the origin and the direction perpendicular to the plane where the obstacle is located as the Y direction;
[0096] In one embodiment, the camera coordinate system may be converted into the world coordinate system according to the Pythagorean theorem.
[0097] Step 304: determine the minimum value of the first depth values in the Y direction in the world coordinate system as the first depth value of the first part of the target user.
[0098] In one embodiment, the first depth value in the Y direction in the world coordinate system can reflect the distance of the target user in the vertical direction, that is, a more accurate and reliable distance.
[0099] In one embodiment, obtaining a first depth value of a first part of the target user from the depth image includes:
[0100] Determine whether the depth image is a valid image, where the valid image refers to a depth image whose error is within a preset valid error range;
[0101] In one embodiment, if Figure 4 As shown, the determining whether the depth image is a valid image includes:
[0102] Step 401, obtaining fourth depth values of two preset reference points in the Z direction in the camera coordinate system; the fourth depth values of the two reference points in the Z direction in the camera coordinate system are the same;
[0103] In one embodiment, the Z direction in the camera coordinate system refers to the Z direction which is close to the front-rear direction of the vehicle and perpendicular to the Y direction of the camera coordinate system;
[0104] In one embodiment, taking a vehicle with three rows of seats as an example, the two reference points may be selected as the midpoints of the B-pillars of the vehicle.
[0105] Step 402, obtaining an average depth value of all pixels in a first preset area in the depth image in the Z direction in the camera coordinate system; the first preset area refers to an area in the depth image centered on the midpoint of a line connecting two reference points;
[0106] In one embodiment, the first preset area may be a pixel block of 20*20 size, or a pixel block of other sizes, which is not limited in the present disclosure.
[0107] In one embodiment, the average depth value refers to the average of the depth values of all pixels in the first preset area in the Z direction in the camera coordinate system.
[0108] Step 403: Subtract the average depth value from the fourth depth value, and determine whether the depth image is a valid image based on whether the difference is within a preset effective error range.
[0109] If the judgment result is yes, a first depth value of the first part of the target user is obtained from the depth image.
[0110] In one embodiment, obtaining the width value of the first part of the target user from the depth image includes:
[0111] Determine at least one first grid in the depth image that is associated with a first part of the target user in an X direction in the camera coordinate system;
[0112] In one embodiment, the first grid refers to the grid where the first part is located;
[0113] In one embodiment, the first part may exist in multiple grids, and each grid where the first part exists may be regarded as the first grid.
[0114] The sum of the depth values of all the first grids in the X direction of the camera coordinate system is determined to be the width value of the first part of the target user.
[0115] The anti-collision reminder method provided by the present disclosure is further described below with an example.
[0116] like Figure 5 As shown, the anti-collision reminder method provided by the application example of the present disclosure is applied to a vehicle with three rows of seats and equipped with a TOF camera, an infrared camera and an entertainment screen. The TOF camera is used to collect the depth image of the target user under the entertainment screen, and the infrared camera is used to collect the non-depth image of the target user under the entertainment screen.
[0117] The anti-collision reminder method provided by the disclosed application example includes the following steps:
[0118] Step 501, at the same time, using a TOF camera to collect a depth image of the target user, and using an infrared camera to collect a non-depth image of the target user including the head;
[0119] Step 502, determining whether the depth image is a valid image;
[0120] If yes, go to step 503;
[0121] If not, return to step 501;
[0122] In an application example, it may be first determined whether the depth image is a valid image by using the depth value indicated by the depth image, such as the depth value in the Y direction of the camera coordinate system.
[0123] Specifically, the parameters of the TOF camera determine that the depth value collected by the TOF camera has a valid depth value range. Whether the depth image is a valid image can be determined based on whether the depth value in the Y direction of the camera coordinate system is within the valid depth value range.
[0124] In an application example, determining whether the depth image is a valid image may also include:
[0125] Get the reference depth value of the midpoint of the B-pillars on both sides of the vehicle in the Z direction in the camera coordinate system;
[0126] Get the actual mean value of the depth value of all pixels in the Z direction in the camera coordinate system in the 20*20 pixel block area where the midpoint of the line connecting the midpoints of the B-pillars on both sides of the vehicle in the depth image is located;
[0127] The reference depth value is subtracted from the actual mean value, and whether the depth image is a valid image is determined based on whether the difference is within a preset effective error range.
[0128] Step 503, converting the camera coordinate system into the world coordinate system according to the angle value between the installation plane of the TOF camera and the horizontal plane;
[0129] Step 504, determining a depth value of a first part of the target user in the Y direction in the world coordinate system as a first depth value;
[0130] Step 505, determining whether a first depth value of a first part of the target user satisfies a first condition;
[0131] If yes, proceed to step 506;
[0132] If not, return to step 501;
[0133] Step 506, using the width value of the first part of the target user and the head area value, determining whether the width value of the first part of the target user meets the second condition;
[0134] If yes, proceed to step 507;
[0135] If not, return to step 501;
[0136] Step 507, issuing an anti-collision reminder.
[0137] Corresponding to the aforementioned anti-collision reminder method, such as Figure 6 As shown, the embodiment of the present disclosure further provides an anti-collision reminder device 600, comprising:
[0138] The first acquisition module 601 is used to acquire a depth image and a non-depth image of the target user at the same time; the depth image refers to a depth image of the target user located below the obstacle acquired by the first image collector; the non-depth image refers to a non-depth image including the head of the target user located below the obstacle acquired by the second image collector;
[0139] A second acquisition module 602 is used to acquire a first depth value and a width value of a first part of the target user from the depth image, and to acquire a head area value of the target user from the non-depth image; the first part refers to a part of the target user in the depth image with the smallest depth value;
[0140] The judgment module 603 is used to judge whether the width value of the first part of the target user meets a second condition by using the width value of the first part of the target user and the head area value in response to the first depth value of the first part of the target user meeting a first condition; the first condition refers to that the first depth value indicates that the distance between the first part of the target user and the obstacle is within a preset danger distance; the second condition refers to width thresholds corresponding to different head area values, wherein the larger the head area value, the smaller the corresponding width threshold;
[0141] The reminder module 604 is used to issue an anti-collision reminder in response to the width value of the first part and the head area value of the target user satisfying the second condition.
[0142] In one embodiment, the second acquisition module 602 is specifically configured to:
[0143] Dividing the depth image into a plurality of grids;
[0144] Taking the average depth value of all pixels of each grid in the plurality of grids as the second depth value of the corresponding grid;
[0145] The part corresponding to the smallest depth value among the plurality of second depth values is determined as the first part of the target user, and the smallest depth value among the plurality of second depth values is determined as the first depth value of the first part of the target user.
[0146] In one embodiment, the second acquisition module 602 is further specifically configured to:
[0147] Acquire a depth parameter of the target user in the depth image; the depth parameter includes a third depth value of the target user in the Y direction of the camera coordinate system; the camera coordinate system refers to a coordinate system with the center of the first image collector as the origin and the direction perpendicular to the installation plane of the first image collector as the Y direction;
[0148] Obtaining an angle value of an angle between an installation plane of the first image collector and a horizontal plane;
[0149] According to the angle value, converting the third depth value in the Y direction in the camera coordinate system into the first depth value in the Y direction in the world coordinate system; the world coordinate system refers to a coordinate system with the center of the first image collector as the origin and the direction perpendicular to the plane where the obstacle is located as the Y direction;
[0150] Determine a minimum value of the first depth values in the Y direction in the world coordinate system as the first depth value of the first part of the target user.
[0151] In one embodiment, the second acquisition module 602 is further specifically configured to:
[0152] Determine whether the depth image is a valid image, where the valid image refers to a depth image whose error is within a preset valid error range;
[0153] If the judgment result is yes, a first depth value of the first part of the target user is obtained from the depth image.
[0154] In one embodiment, the second acquisition module 602 is further specifically configured to:
[0155] Obtaining fourth depth values of two preset reference points in the Z direction in the camera coordinate system; the fourth depth values of the two reference points in the Z direction in the camera coordinate system are the same;
[0156] Obtaining an average depth value of all pixels in a first preset area in the depth image in the Z direction in the camera coordinate system; the first preset area refers to an area in the depth image centered on the midpoint of a line connecting two reference points;
[0157] The average depth value is subtracted from the fourth depth value, and whether the depth image is a valid image is determined based on whether the difference is within a preset effective error range.
[0158] In one embodiment, the second acquisition module 602 is further specifically configured to:
[0159] Determine at least one first grid in the depth image that is associated with a first part of the target user in an X direction in the camera coordinate system;
[0160] The sum of the depth values of all the first grids in the X direction of the camera coordinate system is determined to be the width value of the first part of the target user.
[0161] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.
[0162] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a non-transitory computer-readable storage medium storing computer instructions, and a vehicle.
[0163] Specifically, an embodiment of the present disclosure provides an electronic device, including:
[0164] at least one processor; and
[0165] a memory communicatively connected to the at least one processor; wherein,
[0166] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the aforementioned anti-collision reminder method.
[0167] An embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the steps of the aforementioned anti-collision reminder method.
[0168] An embodiment of the present disclosure provides a vehicle, comprising at least one of the aforementioned anti-collision reminder device, an electronic device, and a non-transitory computer-readable storage medium.
[0169] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, vehicle-mounted devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0170] like Figure 7As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 702 or a computer program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. In RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An I / O (Input / Output) interface 705 is also connected to the bus 704.
[0171] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 704, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0172] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the anti-collision reminder method. For example, in some embodiments, the anti-collision reminder method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or the communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the aforementioned anti-collision reminder method in any other appropriate manner (for example, by means of firmware).
[0173] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0174] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0175] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0176] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0177] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for preventing collisions, characterized in that: include: Obtain the depth image and non-depth image of the target user at the same time; The depth image refers to a depth image of the target user located below the obstacle acquired by the first image collector; the non-depth image refers to a non-depth image including the head of the target user located below the obstacle acquired by the second image collector; Acquire a first depth value and a width value of a first part of the target user from the depth image, and acquire a head area value of the target user from the non-depth image; In response to the first depth value of the first part of the target user satisfying the first condition, determining whether the width value and the head area value of the first part of the target user satisfy the second condition; the first condition refers to the first depth value indicating that the distance between the first part of the target user and the obstacle is within a preset danger distance; the second condition refers to width thresholds corresponding to different head area values, wherein the larger the head area value, the smaller the corresponding width threshold; In response to the width value of the first part and the head area value of the target user satisfying the second condition, an anti-collision reminder is issued.
2. The method according to claim 1, characterized in that The acquiring a first depth value of a first part of the target user from the depth image includes: Dividing the depth image into a plurality of grids; Taking an average depth value of pixels of each grid in the plurality of grids as a second depth value of the corresponding grid; The part corresponding to the smallest depth value among the plurality of second depth values is determined as the first part of the target user, and the smallest depth value among the plurality of second depth values is determined as the first depth value of the first part of the target user.
3. The method according to claim 1, characterized in that: The acquiring a first depth value of a first part of the target user from the depth image includes: Acquire a depth parameter of a target user in the depth image; the depth parameter includes a third depth value of the target user in the Y direction of the camera coordinate system; Obtaining an angle value of an angle between an installation plane of the first image collector and a horizontal plane; According to the angle value, converting the third depth value in the Y direction in the camera coordinate system into the first depth value in the Y direction in the world coordinate system; the world coordinate system refers to a coordinate system with the center of the first image collector as the origin and the direction perpendicular to the plane where the obstacle is located as the Y direction; Determine a minimum value of the first depth values in the Y direction in the world coordinate system as the first depth value of the first part of the target user.
4. The method according to claim 1, characterized in that: The acquiring a first depth value of a first part of the target user from the depth image includes: Determine whether the depth image is a valid image, where the valid image refers to a depth image whose error is within a preset valid error range; If the judgment result is yes, a first depth value of the first part of the target user is obtained from the depth image.
5. The method according to claim 4, characterized in that The determining whether the depth image is a valid image includes: Obtain fourth depth values of two preset reference points in the Z direction in the camera coordinate system; the fourth depth values of the two reference points in the Z direction in the camera coordinate system are the same; Obtaining an average depth value of all pixels in a first preset area in the depth image in the Z direction in the camera coordinate system; the first preset area refers to an area in the depth image centered on the midpoint of a line connecting two reference points; The average depth value is subtracted from the fourth depth value, and whether the depth image is a valid image is determined based on whether the difference is within a preset effective error range.
6. The method according to claim 2, characterized in that The acquiring the width value of the first part of the target user from the depth image includes: Determine at least one first grid in the depth image that is associated with a first part of the target user in an X direction in the camera coordinate system; The sum of the depth values of all the first grids in the X direction of the camera coordinate system is determined to be the width value of the first part of the target user.
7. An anti-collision reminder device, characterized in that: include: A first acquisition module is used to acquire a depth image and a non-depth image of a target user at the same time; The depth image refers to a depth image of the target user located below the obstacle acquired by the first image collector; the non-depth image refers to a non-depth image including the head of the target user located below the obstacle acquired by the second image collector; A second acquisition module, configured to acquire a first depth value and a width value of a first part of the target user from the depth image, and acquire a head area value of the target user from the non-depth image; The first part refers to a part of the target user in the depth image where the depth value is the smallest; a judgment module, configured to judge whether the width value of the first part of the target user satisfies a second condition by using the width value of the first part of the target user and the head area value in response to the first depth value of the first part of the target user satisfying a first condition; the first condition refers to that the first depth value indicates that the distance between the first part of the target user and an obstacle is within a preset danger distance; the second condition refers to width thresholds corresponding to different head area values, wherein the larger the head area value, the smaller the corresponding width threshold; The reminder module is used to issue an anti-collision reminder in response to the width value of the first part and the head area value of the target user satisfying the second condition.
8. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.
10. A vehicle, characterized in that: Includes at least one of the device of claim 7, the electronic device of claim 8, and the non-transitory computer-readable storage medium of claim 9.