Anti-collision alarm method and apparatus, and electronic device, storage medium and vehicle

Abstract

Provided in the present disclosure are an anti-collision alarm method and apparatus, and an electronic device, a storage medium and a vehicle. The method comprises: acquiring a target image; and when a key human body position in the target image falls into an alarm area, outputting alarm information, wherein the alarm information comprises basic user information.

Description

Anti-collision alarm method, device, electronic device, storage medium and vehicle

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to a Chinese patent application filed with the Patent Office of China on March 26, 2024, with application number 202410355252.X and entitled “Anti-head collision alarm method, device, electronic device, storage medium and vehicle,” the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present disclosure relates to the field of alarm technology, and in particular to an anti-collision alarm method, device, electronic equipment, storage medium, and vehicle. Background Art

[0004] In related alarm technologies, a camera is usually used to capture a target image of a target object; the key point positions of the target object are determined based on the target image, and then the key point positions of the target object are used to generate the action of the target object; the action of the target object is then compared with preset dangerous actions to determine whether the action of the target object is a dangerous action; if the judgment result is yes, an alarm message such as a flashing light alarm or a rapid buzzing alarm is issued.

[0005] However, in related alarm technologies, flashing lights or rapid beeps are issued to all users. Since there are usually multiple users in a vehicle, when these alarm technologies are used in a vehicle, the target users inside the vehicle do not know whether the alarm message is directed at them, and therefore their perception of the alarm message is weak. Summary of the Invention

[0006] The present disclosure provides an anti-collision alarm method, device, electronic equipment, storage medium and vehicle.

[0007] According to a first aspect of the present disclosure, there is provided an anti-collision alarm method, comprising:

[0008] Acquire the target image;

[0009] When a key position of a human body in the target image falls into an alarm area, an alarm message is output, where the alarm message includes basic information of the user.

[0010] According to a second aspect of the present disclosure, there is provided an anti-collision alarm method, comprising:

[0011] Acquire the target image;

[0012] Determining the alarm scene where the user is located according to the human body position in the target image;

[0013] When the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene, an alarm message is output.

[0014] According to a third aspect of the present disclosure, there is provided an anti-head collision alarm method, comprising:

[0015] Acquire a vehicle interior image; wherein the vehicle interior image refers to a vehicle interior image acquired by at least one non-depth image collector disposed in the vehicle cabin;

[0016] Identifying user information of at least one user in the vehicle interior image to obtain user information of each user; wherein the user information includes location information and basic user information of the user; the location information includes a body position and a head position, and the basic user information includes at least one of a user name and a user attribute;

[0017] Traversing the position information of each user in the vehicle interior image, and determining the alarm scene of the currently traversed user according to the body position of the currently traversed user;

[0018] According to the currently traversed head position of the user, determining whether the currently traversed head position of the user falls into the alarm area corresponding to the alarm scene; wherein different alarm scenes correspond to different alarm areas;

[0019] If the judgment result is yes, the alarm information of the user currently traversed is output; the alarm information includes the basic user information of the user currently traversed.

[0020] According to a fourth aspect of the present disclosure, there is provided an anti-head collision alarm device, comprising:

[0021] A first acquisition unit is configured to acquire a vehicle interior image; wherein the vehicle interior image refers to a vehicle interior image acquired by at least one non-depth image acquirer disposed in the vehicle cabin;

[0022] a first recognition unit, configured to recognize user information of at least one user in the vehicle interior image and obtain user information of each user; wherein the user information includes location information and basic user information of the user; the location information includes a body position and a head position; and the basic user information includes at least one of a user name and a user attribute;

[0023] a first determining unit, configured to traverse the position information of each user in the vehicle interior image, and determine an alarm scene in which the currently traversed user is located based on the body position of the currently traversed user;

[0024] a judgment unit, configured to judge, based on the currently traversed head position of the user, whether the currently traversed head position of the user falls into an alarm area corresponding to the alarm scenario; wherein different alarm scenarios correspond to different alarm areas;

[0025] The first output unit is configured to output the alarm information of the currently traversed user if the judgment result is yes; the alarm information includes the basic user information of the currently traversed user.

[0026] According to a fifth aspect of the present disclosure, there is provided an anti-collision alarm device, comprising:

[0027] A second acquisition unit, configured to acquire a target image;

[0028] The second output unit is configured to output alarm information when a key position of a human body in the target image falls into an alarm area, wherein the alarm information includes basic user information.

[0029] According to a sixth aspect of the present disclosure, there is provided an anti-collision alarm device, comprising:

[0030] A third acquisition unit, configured to acquire a target image;

[0031] a second determining unit, configured to determine an alarm scene in which the user is located based on a human body position in the target image;

[0032] The third output unit is configured to output alarm information when a key position of a human body in the target image falls into an alarm area corresponding to the alarm scene.

[0033] According to a seventh aspect of the present disclosure, there is provided an electronic device, including:

[0034] at least one processor; and

[0035] a memory communicatively connected to at least one processor; wherein,

[0036] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to perform the method of the first aspect, the second aspect, or the third aspect.

[0037] According to an eighth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method of the aforementioned first aspect, second aspect, or third aspect.

[0038] According to a ninth aspect of the present disclosure, a vehicle is provided, comprising the device provided in the fourth aspect, the fifth aspect, or the sixth aspect of the present disclosure or the electronic device provided in the seventh aspect of the present disclosure.

[0039] The present disclosure provides a collision avoidance alarm method, device, electronic device, storage medium, and vehicle. The method provided by the present disclosure includes: acquiring a vehicle interior image; wherein the vehicle interior image refers to a vehicle interior image captured by at least one non-depth image collector set in the vehicle cabin; identifying user information of at least one user in the vehicle interior image to obtain user information of each user; wherein the user information includes the user's location information and basic user information; the location information includes the body position and head position of the user, and the basic user information includes at least one of the user's name and user attributes; traversing the location information of each user in the vehicle interior image, and determining the alarm scene of the currently traversed user based on the body position of the currently traversed user; and determining whether the currently traversed user's head position falls within an alarm area corresponding to the alarm scene based on the currently traversed user's head position; wherein different alarm scenes correspond to different alarm areas; and if the judgment result is yes, outputting alarm information of the currently traversed user; the alarm information includes at least one of the currently traversed user's body position, head position, or user attributes.

[0040] According to the solution disclosed herein, by identifying user information in the vehicle interior image, the user's location information and basic user information can be obtained; the location information includes the body position and head position, and the basic user information includes at least one of the user's name and user attributes. Furthermore, based on the user's body position, the alarm scenario in which the user is located can be determined. When the user's head position falls within the alarm area corresponding to the alarm scenario, an alarm message containing the user's basic information can be output. Because the alarm message includes at least one of the user's name and user attributes, compared to flashing light alarms or rapid beeping alarms in related technologies, users at risk of head collision can accurately perceive that the alarm message is directed at them, improving their awareness of the alarm message.

[0041] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0043] FIG1 is a schematic diagram of the structure of the seat distribution in the vehicle cabin provided by an embodiment of the present disclosure;

[0044] FIG2 is a flow chart of an anti-head collision alarm method provided by an embodiment of the present disclosure;

[0045] FIG3 is a schematic structural diagram of key points of a human body provided by an embodiment of the present disclosure;

[0046] FIG4 is a flow chart of a method for determining a user name provided by an embodiment of the present disclosure;

[0047] FIG5 is a flow chart of a first method for obtaining an original alarm probability provided by an embodiment of the present disclosure;

[0048] FIG6 is a flow chart of a second method for obtaining an original alarm probability provided by an embodiment of the present disclosure;

[0049] FIG7 is a flow chart of a third method for obtaining an original alarm probability provided by an embodiment of the present disclosure;

[0050] FIG8 is a flow chart of a fourth method for obtaining an original alarm probability provided by an embodiment of the present disclosure;

[0051] FIG9 is a flowchart of a fifth method for obtaining an original alarm probability provided by an embodiment of the present disclosure;

[0052] FIG10 is a schematic structural diagram of an anti-head collision alarm device provided in an embodiment of the present disclosure;

[0053] FIG11 is a flow chart of a method for preventing collisions and providing an alarm according to an embodiment of the present disclosure;

[0054] FIG12 is a schematic diagram of a structure of an anti-collision alarm device provided by an embodiment of the present disclosure;

[0055] FIG13 is a second flow chart of an anti-collision alarm method provided by an embodiment of the present disclosure;

[0056] FIG14 is a second structural diagram of an anti-collision alarm device provided by an embodiment of the present disclosure;

[0057] FIG15 is a schematic block diagram of an example electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0059] Example 1

[0060] An anti-collision head alarm method provided in an embodiment of the present disclosure can be applied to vehicles equipped with a non-depth image collector, such as an infrared (IR) camera. Its execution entity can be a processor on the vehicle side or in the cloud, such as the processor of the vehicle's anti-collision head alarm module, or the vehicle's entire vehicle controller.

[0061] In one embodiment, the seating arrangement within a vehicle cabin is shown in Figure 1 . The seats within the vehicle cabin include the front row, second row, and third row; the second and third rows are collectively referred to as the rear row. The front row refers to seats 0 and 1, the second row refers to seats 2 and 3, the third row refers to seats 4 and 5, and the aisle area refers to seat 7. An entertainment screen is typically located between the front and second row seats, and is typically located at the top of the cabin.

[0062] In one embodiment, the aisle area can be divided into an upper aisle area and a lower aisle area.

[0063] In one embodiment, the IR camera inside the vehicle can be arranged in front of the front seat, such as in front of or behind the rearview mirror in the vehicle. The IR camera arranged in front of the front seat is used to collect front row video or front row image.

[0064] In one embodiment, the IR camera inside the vehicle may also be set behind the entertainment screen, for example, twenty centimeters behind the entertainment screen; the IR camera set behind the entertainment screen is used to collect rear video or rear images of the second row of seats, third row of seats and aisle area inside the cabin.

[0065] In one embodiment, the number of cameras located in front of the front seats can be one or more. Since a single camera can capture complete front-row video or images, the number of cameras located in front of the front seats is preferably one. This disclosure does not limit the number of cameras. Similarly, the number of cameras located behind the entertainment screen is preferably one.

[0066] In one embodiment, IR cameras may also be provided in front of the front seats and behind the entertainment screen.

[0067] In one embodiment, since the seats in the cabin are divided into the front row, the second row, and the third row, and obstacles such as the entertainment screen are typically located between the front and second rows, at the top of the cabin, users in the front row who lean back or users in the second row who lean forward may collide with the entertainment screen. However, if users in the third row lean forward, they are too far away from the entertainment screen and therefore do not face the risk of collision. Only when exiting the third row, they may move toward the door near the second row, at which point there is a risk of collision with the entertainment screen. Furthermore, users in the aisle may be in the upper aisle area, which can only be captured by the non-depth image collector located in front of the front row, or in the lower aisle area, which can only be captured by the non-depth image collector located behind the obstacle. Regardless of whether the user is in the upper aisle area or the lower aisle area, there is a risk of collision with the entertainment screen.

[0068] Based on this, according to the user's body position in the vehicle, the scenarios in which the user may hit his head in the vehicle can generally be divided into front seat alarm scenarios, second row seat alarm scenarios, upper aisle area alarm scenarios, lower aisle area alarm scenarios and door alarm scenarios; correspondingly, the risk areas where the user's head hits the entertainment screen generally include the alarm area corresponding to the front seat alarm scenario, the alarm area corresponding to the second row seat alarm scenario, the alarm area corresponding to the upper aisle area alarm scenario, the alarm area corresponding to the lower aisle area alarm scenario and the alarm area corresponding to the door alarm scenario.

[0069] As shown in FIG2 , the anti-head collision alarm method provided by the embodiment of the present disclosure includes the following steps:

[0070] Step 201, obtaining an image of the interior of a vehicle;

[0071] In one embodiment, the vehicle interior image refers to a vehicle interior image captured by at least one non-depth image collector disposed in a vehicle cabin;

[0072] In one embodiment, the non-depth image collector may be positioned in front of the front seats, such as in front of or behind the rearview mirror. The non-depth image collector positioned in front of the front seats is used to capture a first vehicle interior image of the front seats, the second row of seats, and the aisle area within the cabin.

[0073] In one embodiment, the non-depth image collector may also be set behind an obstacle, such as an entertainment screen, for example, twenty centimeters behind the entertainment screen; the non-depth image collector set behind the entertainment screen is used to collect a second vehicle interior image of the second row of seats, the third row of seats and the area under the aisle inside the cabin.

[0074] Based on this, in one embodiment, step 201 includes:

[0075] Acquire a first vehicle interior image and / or a second vehicle interior image; the first vehicle interior image refers to a vehicle interior image acquired by a non-depth image collector disposed in front of the front seat, and the second vehicle interior image refers to a vehicle interior image acquired by a non-depth image collector disposed behind an obstacle;

[0076] Step 202: Identify user information of at least one user in the vehicle interior image and obtain user information of each user; wherein the user information includes location information and basic user information of the user; the location information includes body position and head position, and the basic user information includes at least one of a user name and user attributes;

[0077] In one embodiment, a human body detection network and a human body key point recognition network may be used to identify human bodies and heads in the vehicle interior image, and then the identified human bodies and heads are matched to obtain the human body position and head position of each user; specifically:

[0078] A human body detection network may be used to identify the human body of each user in the vehicle interior image.

[0079] In one embodiment, the human detection network typically consists of a backbone network, a shoulder network, and a head network, and is used to detect the presence of users in the vehicle interior image and output the coordinates of the bounding box corresponding to each user, such as the coordinates of the upper left and lower right vertices of the bounding box.

[0080] In one embodiment, the human body detection network can also output the confidence of the human body bounding box corresponding to each user. Based on this, before outputting the coordinates of the human body bounding box corresponding to each user, the human body bounding box whose confidence is less than the preset human body bounding box confidence must be deleted from the human body bounding box output by the human body detection network.

[0081] In one embodiment, after obtaining the coordinates of the human body external bounding box, an image of each selected position of the human body external bounding box may be cropped from the vehicle interior image to obtain at least one user image; then, a human body key point recognition network is used to recognize each user image to obtain the coordinates of the human body key points of each user image;

[0082] In one embodiment, as shown in FIG3 , key points of a human body refer to points that can represent key positions of a human body, such as points 1 to 24 in FIG3 , where points 1 to 4 represent key points of a human head.

[0083] Determine the circumscribed frame corresponding to the head key point among the human body key points of each user image as the head circumscribed frame, and obtain at least one head circumscribed frame.

[0084] Determine the intersection-and-union ratio of the human body external bounding box and the head external bounding box, and determine the human body corresponding to the human body external bounding box and the head corresponding to the head external bounding box whose intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold as the human body and head of the same user; in this way, the human body and head bound to each user are obtained.

[0085] Since the body position is usually the center point of the body bounding box, and the head position is usually the center point of the head bounding box, the body position and head position of the user can be bound for the user.

[0086] In one embodiment, the intersection-over-union (IoU) of the human body's circumference frame and the head's circumference frame can reflect the degree of overlap between the human body's circumference frame and the head's circumference frame; therefore, a human body and head with an IoU greater than a preset IoU threshold are usually the human body and head of the same user.

[0087] In one embodiment, the intersection-and-union ratio between a human body circumference frame and each head circumference frame can be determined, and the human body and head corresponding to the human body circumference frame and the head circumference frame whose intersection-and-union ratio is greater than a preset intersection-and-union ratio are determined to be the human body and head of the same user.

[0088] In one embodiment, if the vehicle interior image is captured by a non-depth image collector disposed in front of a front seat, identifying user information of at least one user in the vehicle interior image includes:

[0089] User information of at least one user in the first vehicle interior image is identified.

[0090] In one embodiment, if the vehicle interior image is captured by a non-depth image collector disposed behind an entertainment screen, identifying user information of at least one user in the vehicle interior image includes:

[0091] User information of at least one user in the second vehicle interior image is identified.

[0092] In one embodiment, if non-depth image collectors are provided in front of the front seats and behind the entertainment screen, identifying user information of at least one user in the vehicle interior image includes:

[0093] User information of at least one user in the first vehicle interior image and the second vehicle interior image is identified.

[0094] In one embodiment, a face recognition network can be used to identify the face of each user in the vehicle interior image to obtain the user name of each user; then, the user basic information matching the user name is searched from the user basic information database; in this way, the user's body position, head position and user basic information can be bound.

[0095] In one embodiment, the user basic information includes at least one of the user name or user attributes; the user attributes include at least one of the user age feature and the user gender feature; the user age feature is used to indicate whether the user is an adult or a child; the user gender feature is used to indicate whether the user is male or female.

[0096] In one embodiment, the user name of each user may also be obtained by recognizing the voice in the vehicle;

[0097] Specifically, using a sound recognition network to recognize the sound in the vehicle, and obtaining the voiceprint characteristics and sound source location corresponding to the sound;

[0098] Determining the user name corresponding to the voiceprint feature using a mapping relationship between the voiceprint feature and the user name;

[0099] Determine the distance between the position coordinates of the sound source position and the position coordinates of the human body position;

[0100] If the distance is less than a preset distance value, the name of the user corresponding to the human body position is determined to be the user name.

[0101] Step 203, traversing the position information of each user in the vehicle interior image, and determining the alarm scene of the currently traversed user according to the body position of the currently traversed user;

[0102] In one embodiment, the user's body external bounding box that is currently traversed can be obtained first, and then the center point position of the body external bounding box can be used as the user's body position; then, the area where the user is located can be determined based on the position coordinates of the center point position of the user's body external bounding box that is currently traversed.

[0103] In one embodiment, the waist key point or the hip key point among the currently traversed user's body key points may also be used as the user's body position, such as at least one of points 15 to 18 in FIG3 .

[0104] In one embodiment, as described above, based on the user's body position in the vehicle, the alarm scenarios in which the user in the vehicle may hit his head can generally be divided into front seat alarm scenarios, second row seat alarm scenarios, upper aisle area alarm scenarios, lower aisle area alarm scenarios and door alarm scenarios.

[0105] In one embodiment, if the currently traversed user is located in the front seat, then the alarm scene in which the user is located is determined to be the front seat alarm scene;

[0106] In one embodiment, if the currently traversed user is located in the second row of seats, it is determined that the alarm scene in which the user is located is the second row of seats alarm scene;

[0107] In one embodiment, if the currently traversed user is located in the upper aisle area, it is determined that the alarm scene in which the user is located is the upper aisle area alarm scene;

[0108] In one embodiment, if the currently traversed user is located in the upper aisle area, it is determined that the alarm scene in which the user is located is the lower aisle area alarm scene;

[0109] In one embodiment, if the currently traversed user is located at a car door, it is determined that the alarm scene in which the user is located is a car door alarm scene.

[0110] Step 204: judging whether the currently traversed head position of the user falls within the alarm area corresponding to the alarm scenario based on the currently traversed head position of the user;

[0111] In one embodiment, corresponding to the alarm scenarios, different alarm scenarios correspond to different alarm areas.

[0112] In one embodiment, the external bounding box of the head of the currently traversed user can be obtained first, and then the center point position of the external bounding box of the head can be used as the head position of the currently traversed user; then, based on the position coordinates of the center point position of the external bounding box of the head of the currently traversed user, it is determined whether the head position of the currently traversed user falls into the alarm area corresponding to the alarm scene.

[0113] In one embodiment, corresponding to the alarm scenario, the risk areas where the user's head collides with the entertainment screen generally include the alarm area corresponding to the front seat alarm scenario, the alarm area corresponding to the second row seat alarm scenario, the alarm area corresponding to the aisle area alarm scenario, the alarm area corresponding to the lower aisle area alarm scenario, and the alarm area corresponding to the door alarm scenario.

[0114] Specifically, in one embodiment,

[0115] If the alarm scene currently traversed by the user is the front seat alarm scene, then judging whether the head position of the currently traversed user falls into the alarm area corresponding to the front seat alarm scene according to the head position of the currently traversed user;

[0116] If the alarm scene currently traversed by the user is the second-row seat alarm scene, then judging whether the head position of the currently traversed user falls into the alarm area corresponding to the second-row seat alarm scene according to the head position of the currently traversed user;

[0117] If the alarm scene currently traversed by the user is the aisle area alarm scene, then judging whether the head position of the currently traversed user falls into the alarm area corresponding to the aisle area alarm scene according to the head position of the currently traversed user;

[0118] If the alarm scene currently traversed by the user is the aisle area alarm scene, then judging whether the head position of the currently traversed user falls into the alarm area corresponding to the aisle area alarm scene according to the head position of the currently traversed user;

[0119] If the alarm scene currently traversed by the user is a car door alarm scene, it is determined based on the head position of the currently traversed user whether the head position of the currently traversed user falls into the alarm area corresponding to the car door alarm scene.

[0120] In one embodiment, before step 204 , the anti-head collision alarm provided by the present disclosure may further include: obtaining an alarm area corresponding to the alarm scene.

[0121] In one embodiment, the alarm area corresponding to the alarm scene includes:

[0122] Acquire a plurality of training images, wherein the training images are images of a human head at a boundary position of the alarm area;

[0123] In one embodiment, the boundaries of the alarm area are typically defined by a product manager.

[0124] In one embodiment, taking the front seat alarm scenario as an example, a product manager can designate a person to sit in the front seat to calibrate the alarm area. The person can sit in both front seats, or just one of them, and lean back to the edge of the alarm area. A non-depth image collector located in front of the front seat captures training images.

[0125] Recognize the human heads in the training image to obtain a plurality of human head bounding boxes in the training image; wherein one human head corresponds to one human head bounding box;

[0126] Determine the center point position of each of the head circumference frames to obtain the center point positions of multiple head circumference frames;

[0127] In one embodiment, the center point position is used to indicate the center coordinate point of the circumscribed frame of the head.

[0128] In one embodiment, the center point position of the head circumference frame may be obtained through the coordinates of the head circumference frame.

[0129] A closed area formed by connecting lines of multiple center points is determined as an alarm area.

[0130] In one embodiment, the size of the warning area refers to the size of the area enclosed by all head positions at the boundary positions of the warning area in all training images.

[0131] Step 205: If the judgment result is yes, output the alarm information of the user currently traversed; the alarm information includes the basic user information of the user.

[0132] In one embodiment, if the judgment result is yes, it means that the user is at risk of hitting the entertainment screen with his head, and an alarm should be issued.

[0133] In one embodiment, the alarm information may be a text alarm information displayed on the entertainment screen and / or the central control screen, or may be a voice alarm information.

[0134] In one embodiment, the content of the text alarm message or the voice alarm message may be “Little Ming in the front seat, be careful not to bump your head.”

[0135] In one embodiment, the alarm information may further include an alarm scenario.

[0136] In one embodiment, an alarm message may be output each time a user is traversed; the basic user information of the traversed users may also be cached until all users are traversed, and then the basic user information of the users who are at risk of colliding their heads with the entertainment screen is uniformly output.

[0137] In one embodiment, after outputting an alarm message, the alarm can be turned off and then turned on again after a preset duration. The preset duration can be 10 seconds or other durations, and is not limited in this disclosure. Turning off the alarm after outputting an alarm message not only allows users to perceive the alarm message but also saves computing resources.

[0138] The anti-head collision alarm method provided by the present disclosure includes: acquiring a vehicle interior image; wherein the vehicle interior image refers to a vehicle interior image collected by at least one non-depth image collector set in the vehicle cabin; identifying user information of at least one user in the vehicle interior image, and obtaining user information of each of the users; wherein the user information includes the user's body position, head position and user attributes; traversing the user information of each user in the vehicle interior image, and determining the alarm scene in which the currently traversed user is located based on the body position of the currently traversed user; judging whether the currently traversed user's head position falls into the alarm area corresponding to the alarm scene based on the currently traversed user's head position; wherein different alarm scenes correspond to different alarm areas; if the judgment result is yes, outputting the alarm information of the currently traversed user; the alarm information includes at least one of the currently traversed user's body position, head position or user attributes.

[0139] According to the disclosed solution, by identifying user information in the vehicle interior image, the user's body position, head position, and user attributes can be obtained. Furthermore, based on the user's body position, the alarm scenario the user is in can be determined. When the user's head position falls within the alarm area corresponding to the alarm scenario, an alarm message containing the user information can be output. This improves the user's awareness of alarm information.

[0140] In one embodiment, as shown in FIG4 , step 202 includes:

[0141] Step 401: identifying basic user information of each user in the vehicle interior image to obtain basic user information of each user;

[0142] In one embodiment, a trained face recognition network may be used to recognize the face of each user in the vehicle interior image to obtain the user name of each user.

[0143] In one embodiment, the training set of the face recognition network may include a large number of facial features, each labeled with a user name. The facial features may include the facial features of users who frequently use a vehicle. For example, for a family vehicle, the facial features collected may include the facial features of all family members; for a company vehicle, the facial features collected may include the facial features of all company employees.

[0144] In one embodiment, searching a user basic information database for user basic information that matches the user name;

[0145] In one embodiment, the basic user information includes at least one of the user name or user attributes; the user attributes include the user's age characteristics and gender characteristics, etc.; the user's age characteristics are used to indicate whether the user is an adult or a child; the user's gender characteristics are used to indicate whether the user is male or female.

[0146] In one embodiment, the user's age characteristics and gender characteristics can be pre-bound to the user's name and stored in the vehicle or cloud to form a basic user information database.

[0147] In one embodiment, a user attribute recognition network based on graph matching or torso length may be directly used to identify the user's age characteristics.

[0148] Step 402 , identifying the body and head of each user in the vehicle interior image, and obtaining a body bounding box of each user and a head bounding box of each head in the vehicle interior image;

[0149] Step 403: Determine the intersection-and-union (IoU) ratio of each human body bounding box and each head bounding box, and match the user corresponding to the human body bounding box with the head bounding box whose IoU ratio is not less than a preset IoU threshold.

[0150] In one embodiment, the intersection-over-union (IoU) of the human body's circumference frame and the head's circumference frame can reflect the degree of overlap between the human body's circumference frame and the head's circumference frame; therefore, a human body and head with an IoU greater than a preset IoU threshold are usually the human body and head of the same user.

[0151] In one embodiment, the intersection-and-union ratio between a human body circumference frame and each head circumference frame can be determined, and the human body and head corresponding to the human body circumference frame and the head circumference frame whose intersection-and-union ratio is greater than a preset intersection-and-union ratio are determined to be the human body and head of the same user.

[0152] Step 404 : determining the center point position of the user's body circumference frame as the user's body position, and determining the center point position of the head circumference frame of the head matching the user as the user's head position.

[0153] In one embodiment, step 205 includes:

[0154] If the judgment result is yes, determining the original alarm probability; the original alarm probability is the alarm probability determined according to the alarm scene currently traversed by the user;

[0155] In one embodiment, if the judgment result is yes, it means that the head position of the user currently traversing falls into the alarm area corresponding to the alarm scene.

[0156] In one embodiment, if the alarm scenario described for the currently traversed user's head position is a front seat alarm scenario, then if the judgment result is yes, it means that the currently traversed user's head position falls into the alarm area corresponding to the front seat alarm scenario.

[0157] In one embodiment, if the alarm scenario described for the currently traversed user's head position is a two-row seat alarm scenario, then if the judgment result is yes, it means that the currently traversed user's head position falls into the alarm area corresponding to the two-row seat alarm scenario.

[0158] In one embodiment, if the alarm scenario described for the currently traversed user's head position is an aisle area alarm scenario, then if the judgment result is yes, it means that the currently traversed user's head position falls into the alarm area corresponding to the aisle area alarm scenario.

[0159] In one embodiment, if the alarm scene described by the currently traversed user's head position is an alarm scene for the area under the aisle, then if the judgment result is yes, it means that the currently traversed user's head position falls into the alarm area corresponding to the alarm scene for the area under the aisle.

[0160] In one embodiment, if the alarm scene described by the currently traversed user's head position is a car door alarm scene, then if the judgment result is yes, it means that the currently traversed user's head position falls into the alarm area corresponding to the car door alarm scene.

[0161] Correcting the original alarm probability using the preset alarm sensitivity coefficient corresponding to the alarm area to obtain a target alarm probability corresponding to the alarm area;

[0162] In one embodiment, the preset alarm sensitivity coefficient is used to indicate the triggering difficulty corresponding to the alarm scenario currently traversed by the user;

[0163] In one embodiment, the preset alarm sensitivity coefficient can be set by the user; for example, through the human-computer interaction interface on the central control screen.

[0164] In one embodiment, taking the alarm scenario of the front seat alarm scenario as an example:

[0165] If the user often leans back from the front seat to talk with the user in the back seat, and at the same time does not expect the vehicle to send out alarm messages all the time, the preset alarm sensitivity coefficient corresponding to the front seat alarm can be set to a value between 0 and 1, such as 0.5, which will reduce the number of alarms corresponding to the front seat alarm scenario to a certain extent.

[0166] In one embodiment, if the user does not want to issue an alarm message, the preset alarm sensitivity coefficient of the corresponding alarm scenario can be set to 0.

[0167] In response to the target alarm probability being greater than a preset alarm probability threshold, an alarm message is output.

[0168] In one embodiment, before step 202, the anti-head collision alarm method includes:

[0169] receiving an alarm area adjustment instruction for adjusting an alarm area corresponding to an alarm scene currently traversed by the user;

[0170] In one embodiment, the alarm area adjustment instruction is issued by the user.

[0171] In one embodiment, the user's alarm area adjustment instruction may be received via the vehicle's central control screen.

[0172] In one embodiment, a voice alarm area adjustment instruction issued by the user may also be received via a microphone.

[0173] The size of the alarm area is scaled according to the alarm area scaling factor in the alarm area adjustment instruction.

[0174] In one embodiment, the alarm area corresponding to different alarm scenarios may be increased or decreased according to the user's expectation of the alarm sensitivity for different alarm scenarios.

[0175] In one embodiment, the alarm area scaling factor may be 150%, indicating that the alarm area is adjusted to 1.5 times the current alarm area; the alarm area scaling factor may also be 50%, indicating that the alarm area is adjusted to half the current alarm area; this is not limited in the present disclosure.

[0176] In one embodiment, outputting the alarm information of the currently traversed user includes:

[0177] If no closing instruction for closing the output of the alarm information corresponding to the alarm scene of the currently traversed user is received, the alarm information of the currently traversed user is output.

[0178] In one embodiment, the shutdown instruction is issued by a user.

[0179] In one embodiment, the user's shutdown command may be received via the vehicle's central control screen.

[0180] In one embodiment, a voice shutdown command issued by the user may also be received via a microphone.

[0181] In one embodiment, the closing instruction issued by the user may also be received via a mechanical button.

[0182] In one embodiment, in addition to the closing instruction, an opening instruction may also be received.

[0183] In one embodiment, the alarm information corresponding to the alarm scenario may be output or not output according to the user's alarm expectation for different alarm scenarios.

[0184] In one embodiment, if the user in the front seat often leans back to talk with the user in the second seat, in order not to affect the conversation experience, a shutdown command may be issued and the alarm message may not be output.

[0185] In one embodiment, taking the front seat scenario as an example, the user's body is located in the front seat, and their head is located in the alarm area corresponding to the front seat scenario. Therefore, to trigger an alarm for the front seat alarm scenario, it is sufficient to obtain a first vehicle interior image captured by a non-depth image collector located in front of the front seat.

[0186] Based on this, the vehicle interior image includes a first vehicle interior image acquired by a non-depth image collector arranged in front of the front seat. The alarm scene includes a front seat alarm scene;

[0187] Correspondingly, as shown in FIG5 , if the judgment result is yes, determining the original alarm probability includes:

[0188] Step 501: Obtain the head area of ​​the user currently traversing in the first vehicle interior image;

[0189] In one embodiment, the length and width of the head bounding box of the currently traversed user's head may be multiplied to obtain the head area of ​​the currently traversed user;

[0190] In one embodiment, the human head circumference frame is generally rectangular.

[0191] In one embodiment, since the image captured by the non-depth image collector is a planar image, the closer the human head is to the camera, the larger its area on the first vehicle interior image; conversely, the farther the human head is from the camera, the smaller its area on the first vehicle interior image.

[0192] Step 502: determining a first initial alarm probability based on a comparison between the head area of ​​the currently traversed user and a first preset head area threshold;

[0193] In one embodiment, only when the head position of the currently traversed user is within the alarm area corresponding to the front seat alarm scene, the user of the currently traversed user is at risk of colliding with the entertainment screen, and the first initial alarm probability is 1 at this time; when the head position of the currently traversed user is not within the first target area, the head of the currently traversed user is not at risk of colliding with the entertainment screen, and the first initial alarm probability is 0 at this time.

[0194] In one embodiment, because the first vehicle interior image is not captured by the depth image collector, the size of the user's head is highly correlated with the distance between the user's head and the obstacle. Specifically, the larger the user's head size, the smaller the distance between the user's head and the front-row image collector, and the larger the distance between the user's head and the obstacle. Conversely, the smaller the user's head size, the larger the distance between the user's head and the front-row image collector, and the smaller the distance between the user's head and the obstacle.

[0195] In one embodiment, the first preset head area threshold refers to the area value of the corresponding head circumscribed frame on the first vehicle interior image when the head is at the boundary of the alarm area corresponding to the front seat scene.

[0196] In one embodiment, the first preset head area threshold may also refer to the area value of the head circumscribed frame corresponding to the first vehicle interior image when the head contacts an obstacle.

[0197] In one embodiment, the area value of the external bounding box of the human head corresponding to the boundary of the alarm area corresponding to the front seat scene, and the area value of the external bounding box of the human head corresponding to the first vehicle interior image when the human head contacts an obstacle can also be used simultaneously to jointly determine the original alarm probability.

[0198] Step 503: Compare the user's head area with a standard head area to obtain a relative head area;

[0199] In one embodiment, the standard head area refers to the area value of the corresponding head external frame on the first vehicle interior image when the user sits in the front seat in a standard sitting posture (without stretching the head forward or backward).

[0200] In one embodiment, multiple first vehicle interior images of a user sitting in a standard sitting posture in the front seat may be collected, and then the area value of the external bounding box of the head in each first vehicle interior image may be calculated. Finally, the area values ​​of the multiple external bounding boxes of the head may be averaged to obtain the standard head area.

[0201] In one embodiment, the relative head area refers to the degree to which the user's head area is scaled relative to the standard head area. Therefore, the relative head area is highly correlated with the distance between the user's head and the head corresponding to the standard sitting posture, that is, the distance between the user's head and the obstacle.

[0202] Step 504: determining a first alarm probability correction value based on a comparison between the relative head area and a preset relative head area threshold;

[0203] In one embodiment, the larger the relative head area is, the lower the degree of scaling of the user's head area is, that is, the closer the distance between the head and the head corresponding to the standard sitting posture is; when the relative head area is 1, it indicates that the sitting posture of the user in the front seat is the standard sitting posture, and there is no situation of stretching the head backward; when the relative head area is less than 1, and the smaller the relative head area is, the greater the distance the head of the user in the front seat stretches backward; at this time, the alarm probability should be increased.

[0204] Step 505: Use the first alarm probability correction value to correct the first initial alarm probability to obtain an original alarm probability.

[0205] In one embodiment, taking the second-row seat alarm scenario as an example, the user's body is located in the second row of seats, and their head is located within the alarm area corresponding to the second-row seat alarm scenario. When the user in the second row seat leans forward, the first vehicle interior image is captured by the non-depth image collector located in front of the front seat. Therefore, both the second-row seat body information and the alarm area corresponding to the second-row seat alarm scenario are captured. However, if the user in the second row seat leans forward too far, the non-depth image collector located behind the obstacle will not be able to capture the user's head information. In this case, the alarm probability must be increased.

[0206] Therefore, to accurately warn about the second-row seat alarm scenario, it is necessary to obtain both a first vehicle interior image captured by a non-depth image collector set in front of the front seat and a second vehicle interior image captured by a non-depth image collector set behind the obstacle.

[0207] Based on this, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector arranged in front of the front seats and a second vehicle interior image captured by a non-depth image collector arranged behind the obstacle; the alarm scene includes a second-row seat alarm scene.

[0208] Correspondingly, as shown in FIG6 , if the judgment result is yes, determining the original alarm probability includes:

[0209] Step 601: Obtain the head area of ​​the user currently traversing in the first vehicle interior image;

[0210] In one embodiment, the currently traversed user refers to a user whose body position is in the second row of seats and whose head position is in the alarm area corresponding to the second row of seats alarm scenario.

[0211] In one embodiment, since the first vehicle interior image is captured by a non-depth image collector arranged in front of the front row of seats, both human body information of the second row of seats and alarm area information corresponding to the alarm scene of the second row of seats can be obtained.

[0212] Step 602: Determine a second initial alarm probability based on a comparison between the head area of ​​the currently traversed user and a second preset head area threshold;

[0213] In one embodiment, the second preset head area threshold refers to the area value of the corresponding head circumscribed frame on the first vehicle interior image when the head is at the boundary of the second-row seat alarm scene.

[0214] In one embodiment, the second preset head area threshold may also refer to the area value of the head circumscribed frame corresponding to the first vehicle interior image when the head contacts an obstacle.

[0215] In one embodiment, the area value of the external bounding box of the human head corresponding to the first vehicle interior image when the human head is at the boundary of the second row of seats alarm scene and the area value of the external bounding box of the human head corresponding to the first vehicle interior image when the human head contacts an obstacle can also be used simultaneously to jointly determine the second initial alarm probability.

[0216] In one embodiment, when the area of ​​the user's head is smaller than the area of ​​the second preset head external frame, it means that the head is close to the non-depth image collector set in front of the front seat and is also close to the obstacle, and the possibility of a head collision is greater. Therefore, the corresponding second initial alarm probability is also greater.

[0217] Step 603: determining a second alarm probability correction value based on whether a head matching the currently traversed user exists in the second vehicle interior image;

[0218] In one embodiment, when the head of a user in the second row of seats extends too far forward, the non-depth image collector set behind the obstacle cannot collect the user's head information. In this case, the second initial alarm probability must be increased.

[0219] In one embodiment, when there is no head matching the currently traversed user in the second vehicle interior image, it means that when the head of the user in the second row of seats extends forward too far, the non-depth image collector set behind the obstacle cannot collect the user's head information. At this time, the second initial alarm probability must be increased.

[0220] In one embodiment, when a human head matching the currently traversed user exists in the second vehicle interior image, it means that when the head of the user in the second row of seats is not extended forward far enough, the non-depth image collector set behind the obstacle can collect the user's head information. At this time, the second initial alarm probability can be reduced.

[0221] Step 604: Use the second alarm probability correction value to correct the second initial alarm probability to obtain an original alarm probability.

[0222] In one embodiment, taking the aisle area alarm scenario as an example, the user's body is located in the aisle area, and their head is located in the alarm area corresponding to the aisle area alarm scenario. When the user in the aisle area extends their head forward, the first vehicle interior image is captured by the non-depth image collector located in front of the front seat. Therefore, both the body information in the aisle area and the alarm area information corresponding to the aisle area alarm scenario can be acquired. However, if the user in the aisle area extends their head forward too far, the non-depth image collector located behind the obstacle will not be able to capture the user's head information. In this case, the alarm probability must be increased.

[0223] Therefore, to accurately alarm the aisle area alarm scene, it is necessary to obtain a first vehicle interior image captured by a non-depth image collector set in front of the front seat, and a second vehicle interior image captured by a non-depth image collector set behind the obstacle;

[0224] Based on this, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector arranged in front of the front seat and a second vehicle interior image captured by a non-depth image collector arranged behind the obstacle; the alarm scene includes an alarm scene in the aisle area.

[0225] Correspondingly, as shown in FIG7 , if the judgment result is yes, determining the original alarm probability includes:

[0226] Step 701: Obtain the head area of ​​the user currently traversing in the first vehicle interior image;

[0227] In one embodiment, the currently traversed user refers to a user whose body position is in the aisle area and whose head position is in the alarm area corresponding to the alarm scenario of the aisle area.

[0228] In one embodiment, since the first vehicle interior image is captured by a non-depth image collector arranged in front of the front seats, it is possible to obtain both human body information in the aisle area and alarm area information corresponding to the alarm scene in the aisle area.

[0229] Step 702: Determine a third initial alarm probability based on a comparison between the head area of ​​the currently traversed user and a third preset head area threshold;

[0230] In one embodiment, the third preset head area threshold refers to the area value of the corresponding human head circumscribed frame on the first vehicle interior image when the human head is at the boundary of the regional alarm scene in the aisle.

[0231] In one embodiment, the third preset head area threshold may also refer to the area value of the head circumscribed frame corresponding to the first vehicle interior image when the head contacts an obstacle.

[0232] In one embodiment, the area value of the external bounding box of the human head corresponding to the boundary of the regional alarm scene in the aisle and the area value of the external bounding box of the human head corresponding to the interior image of the first vehicle when the human head contacts an obstacle can also be used simultaneously to jointly determine the third initial alarm probability.

[0233] In one embodiment, when the area of ​​the user's head is smaller than the area of ​​the third preset head external frame, it means that the user's head is close to the non-depth image collector set in front of the front seat and is also close to the obstacle, and the possibility of a head collision is greater. Therefore, the corresponding third initial alarm probability is also greater.

[0234] Step 703: determining a third alarm probability correction value based on whether a head matching the currently traversed user exists in the second vehicle interior image;

[0235] In one embodiment, when the user's head in the aisle area extends too far forward, the non-depth image collector set behind the obstacle cannot collect the user's head information. In this case, the third initial alarm probability must be increased.

[0236] In one embodiment, when there is no human head matching the currently traversed user in the second vehicle interior image, it means that when the user's head in the aisle area extends forward too far, the non-depth image collector set behind the obstacle cannot collect the user's head information. At this time, the third initial alarm probability must be increased.

[0237] In one embodiment, when a human head matching the currently traversed user exists in the second vehicle interior image, it indicates that when the user's head in the aisle area does not extend forward far enough, the non-depth image collector behind the obstacle can collect the user's head information. At this time, the third initial alarm probability can be reduced.

[0238] Step 704: Use the third alarm probability correction value to correct the third initial alarm probability to obtain an original alarm probability.

[0239] In one embodiment, taking the aisle area alarm scenario as an example, a user's body is located in the aisle area, and their head is located in the alarm zone corresponding to the aisle area alarm scenario. When the user in the aisle area leans forward, the second vehicle interior image is captured by a non-depth image collector positioned behind the obstacle. Therefore, both the user's body information in the aisle area and the alarm zone information corresponding to the aisle area alarm scenario are captured.

[0240] Therefore, to accurately alarm the alarm scene in the area under the aisle, it is only necessary to obtain the second vehicle interior image captured by the non-depth image collector set behind the obstacle;

[0241] Based on this, the vehicle interior image includes a second vehicle interior image captured by a non-depth image collector arranged behind the obstacle; and the alarm scene includes an alarm scene in the area under the aisle.

[0242] Correspondingly, as shown in FIG8 , if the judgment result is yes, the original alarm probability is determined according to the alarm probability determination method corresponding to the alarm area, including:

[0243] Step 801: Obtain the head area of ​​the user currently traversing in the second vehicle interior image;

[0244] In one embodiment, the currently traversed user refers to a user whose body position is in the area below the aisle and whose head position is in the alarm area corresponding to the alarm scenario of the area below the aisle.

[0245] In one embodiment, since the second vehicle interior image is collected by a non-depth image collector arranged behind the obstacle, it is possible to obtain both human body information in the area under the aisle and alarm area information corresponding to the alarm scene in the area under the aisle.

[0246] Step 802: Determine a fourth initial alarm probability based on a comparison between the head area of ​​the currently traversed user and a fourth preset head area threshold;

[0247] In one embodiment, the fourth preset head area threshold refers to the area value of the corresponding human head circumscribed frame on the second vehicle interior image when the human head is at the boundary of the alarm scene in the area below the aisle.

[0248] In one embodiment, the fourth preset head area threshold may also refer to the area value of the head circumscribed frame corresponding to the second vehicle interior image when the head contacts an obstacle.

[0249] In one embodiment, the fourth initial alarm probability can be determined simultaneously by using the area value of the external bounding box of the human head corresponding to the second vehicle interior image when the human head is at the boundary of the alarm scene in the area below the aisle, and the area value of the external bounding box of the human head corresponding to the second vehicle interior image when the human head contacts an obstacle.

[0250] In one embodiment, when the area of ​​the user's head is smaller than the area of ​​the fourth preset head external frame, it means that the head is close to the non-depth image collector set behind the obstacle and is also close to the obstacle, and the possibility of a head collision is greater. Therefore, the corresponding fourth initial alarm probability is also greater.

[0251] Step 803: Obtain the grayscale value of each pixel in the second vehicle interior image;

[0252] In one embodiment, when a user's head is extended too far forward in the aisle area, it can easily block the non-depth image collector behind the obstacle, causing the grayscale values ​​of pixels in the second vehicle interior image to decrease. High levels of occlusion indicate that the user's head is very close to the obstacle, necessitating an increase in the fourth initial alarm probability.

[0253] Step 804, comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of first target pixels whose grayscale value is greater than the preset grayscale value;

[0254] In one embodiment, the number of first target pixel points whose grayscale values ​​are greater than the preset grayscale value is used to indicate the degree of occlusion of the second vehicle interior image.

[0255] In one embodiment, the preset grayscale value is a grayscale value that is relatively dark, such as a grayscale value below 120. A grayscale value of 0 represents black.

[0256] Step 805: determining a fourth alarm probability correction value based on a comparison between the number of the first target pixels and the first preset number of pixels;

[0257] Step 806: Use the fourth alarm probability correction value to correct the fourth initial alarm probability to obtain an original alarm probability.

[0258] In one embodiment, taking a car door alarm scenario as an example, the user's body is positioned near the car door, and their head is within the alarm zone corresponding to the car door alarm scenario. When the user makes a move, the second vehicle interior image is captured by a non-depth image collector positioned behind the obstacle. This allows the user to capture both the body information near the car door and the alarm zone corresponding to the car door alarm scenario.

[0259] Therefore, to accurately alarm the regional alarm scene in the aisle, it is only necessary to obtain the second vehicle interior image captured by the non-depth image collector set behind the obstacle;

[0260] Based on this, the vehicle interior image includes a second vehicle interior image captured by a non-depth image collector arranged behind the obstacle; and the alarm scene includes a door alarm scene.

[0261] Correspondingly, as shown in FIG9 , if the judgment result is yes, determining the original alarm probability includes:

[0262] Step 901, obtaining the grayscale value of each pixel in the second vehicle interior image;

[0263] In one embodiment, the grayscale value of a pixel in the second vehicle interior image is used to indicate the degree to which a non-depth image collector disposed behind an obstacle is blocked.

[0264] Step 902, comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of second target pixels whose grayscale value is smaller than the preset grayscale value;

[0265] In one embodiment, a greater number of second target pixel points whose grayscale values ​​are less than the preset grayscale value indicates a lower luminous flux of the second vehicle interior image, and a higher degree of occlusion of the non-depth image collector disposed behind the obstacle.

[0266] Step 903: determine whether the number of the second target pixel points is less than the second preset number of pixel points;

[0267] Step 904: If the judgment result is yes, obtaining the head confidence of the user currently traversing in the second vehicle interior image;

[0268] In one embodiment, the number of the second target pixel points is less than the second preset pixel point number, indicating that the non-depth image collector set behind the obstacle is not highly blocked, and further judgment can be made.

[0269] In one embodiment, the number of the second target pixels is not less than the second preset number of pixels, indicating that the non-depth image collector disposed behind the obstacle is highly blocked, and the original alarm probability is determined to be 0.

[0270] Step 905: determining whether the head confidence is not less than a preset head confidence threshold;

[0271] In one embodiment, the head confidence is used to indicate the credibility of the corresponding head.

[0272] Step 906: If the judgment result is yes, obtain the head area of ​​the user currently traversing in the second vehicle interior image;

[0273] Step 907: Determine whether the head area of ​​the user currently traversing in the second vehicle interior image is not less than a fifth preset head area threshold;

[0274] In one embodiment, the fifth preset head area threshold refers to the area value of the corresponding head circumscribed frame on the second vehicle interior image when the human body is located at the vehicle door and the human head contacts an obstacle.

[0275] Step 908: If the judgment result is yes, determine the original alarm probability as the preset alarm probability.

[0276] Example 2

[0277] FIG10 is a schematic structural diagram of a head collision prevention alarm device provided by an embodiment of the present disclosure. As shown in FIG10 , the head collision prevention alarm device 1000 includes:

[0278] The first acquisition unit 1001 is configured to acquire a vehicle interior image; wherein the vehicle interior image refers to a vehicle interior image acquired by at least one non-depth image collector disposed in the vehicle cabin;

[0279] a first recognition unit 1002 configured to recognize user information of at least one user in the vehicle interior image and obtain user information of each user; wherein the user information includes location information and basic user information of the user; the location information includes a body position and a head position; and the basic user information includes at least one of a user name and a user attribute;

[0280] The first determining unit 1003 is configured to traverse the position information of each user in the vehicle interior image, and determine the alarm scene in which the currently traversed user is located based on the body position of the currently traversed user;

[0281] The judging unit 1004 is configured to judge, based on the currently traversed head position of the user, whether the currently traversed head position of the user falls within the alarm area corresponding to the alarm scenario; wherein different alarm scenarios correspond to different alarm areas;

[0282] The first output unit 1005 is configured to output the alarm information of the currently traversed user if the judgment result is yes; the alarm information includes the basic user information of the currently traversed user.

[0283] In one embodiment, the first identification unit 1002 is specifically configured to:

[0284] Identifying basic user information of each user in the vehicle interior image to obtain basic user information of each user;

[0285] Identify the human body and head of each user in the vehicle interior image, and obtain a human body circumference frame of each user and a head circumference frame of each head in the vehicle interior image;

[0286] Determine an intersection-and-union (IoU) ratio between each of the human body bounding boxes and each of the head bounding boxes, and match the user corresponding to the human body bounding box with the head bounding box whose IoU ratio is not less than a preset IoU threshold;

[0287] The center point position of the human body circumference frame of the user is determined as the human body position of the user, and the center point position of the head circumference frame of the head matching the user is determined as the head position of the user.

[0288] In one embodiment, the first output unit 1005 is specifically configured to:

[0289] If the judgment result is yes, determining the original alarm probability; the original alarm probability is the alarm probability determined according to the alarm scene currently traversed by the user;

[0290] The original alarm probability is corrected using a preset alarm sensitivity coefficient corresponding to the alarm area to obtain a target alarm probability corresponding to the alarm area; the preset alarm sensitivity coefficient is used to indicate the triggering difficulty corresponding to the alarm scenario currently traversed by the user;

[0291] In response to the target alarm probability being greater than a preset alarm probability threshold, the alarm information of the currently traversed user is output.

[0292] In one embodiment, the anti-head collision alarm device 1000 further includes an alarm area adjustment unit, which is configured to:

[0293] receiving an alarm area adjustment instruction for adjusting an alarm area corresponding to an alarm scene currently traversed by the user;

[0294] The size of the alarm area is scaled according to the alarm area scaling factor in the alarm area adjustment instruction.

[0295] In one embodiment, the first determining unit 1003 is specifically configured to:

[0296] If no closing instruction for closing the output of the alarm information corresponding to the alarm scene of the currently traversed user is received, the alarm information of the currently traversed user is output.

[0297] In one embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector disposed in front of a front seat; and the alarm scene includes a front seat alarm scene.

[0298] Accordingly, the first output unit 1005 is specifically configured to:

[0299] Obtaining the head area of ​​the user currently traversing in the first vehicle interior image;

[0300] Determining a first initial alarm probability based on a comparison between the head area of ​​the currently traversed user and a first preset head area threshold;

[0301] Comparing the user's head area with a standard head area to obtain a relative head area;

[0302] determining a first alarm probability correction value based on a comparison between the relative head area and a preset relative head area threshold;

[0303] The first initial alarm probability is corrected using the first alarm probability correction value to obtain an original alarm probability.

[0304] In one embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector arranged in front of the front seats and a second vehicle interior image captured by a non-depth image collector arranged behind an obstacle; the alarm scene includes a second-row seat alarm scene.

[0305] Accordingly, the first output unit 1005 is specifically configured to:

[0306] Obtaining the head area of ​​the user currently traversing in the first vehicle interior image;

[0307] Determining a second initial alarm probability based on a comparison between the head area of ​​the currently traversed user and a second preset head area threshold;

[0308] determining a second alarm probability correction value based on whether there is a human head matching the currently traversed user in the second vehicle interior image;

[0309] The second initial alarm probability is corrected using the second alarm probability correction value to obtain an original alarm probability.

[0310] In one embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector arranged in front of the front seats and a second vehicle interior image captured by a non-depth image collector arranged behind an obstacle; the alarm scene includes an alarm scene in the aisle area.

[0311] Accordingly, the first output unit 1005 is specifically configured to:

[0312] Obtaining the head area of ​​the user currently traversing in the first vehicle interior image;

[0313] determining a third initial alarm probability based on a comparison between the head area of ​​the currently traversed user and a third preset head area threshold;

[0314] determining a third alarm probability correction value according to whether there is a human head matching the currently traversed user in the second vehicle interior image;

[0315] The third initial alarm probability is corrected using the third alarm probability correction value to obtain an original alarm probability.

[0316] In one embodiment, the vehicle interior image includes a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; and the alarm scene includes an alarm scene in an area under the aisle.

[0317] Accordingly, the first output unit 1005 is specifically configured to:

[0318] Obtaining the head area of ​​the user currently traversing in the second vehicle interior image;

[0319] determining a fourth initial alarm probability based on a comparison between the head area of ​​the currently traversed user and a fourth preset head area threshold;

[0320] Obtaining a grayscale value of each pixel in the second vehicle interior image;

[0321] Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of first target pixel points whose grayscale value is greater than the preset grayscale value;

[0322] Determining a fourth alarm probability correction value based on a comparison between the number of the first target pixel points and the first preset number of pixel points;

[0323] The fourth initial alarm probability is corrected using the fourth alarm probability correction value to obtain an original alarm probability.

[0324] In one embodiment, the vehicle interior image includes a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; and the alarm scene includes a door alarm scene.

[0325] Accordingly, the first output unit 1005 is specifically configured to:

[0326] Obtaining a grayscale value of each pixel in the second vehicle interior image;

[0327] Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of second target pixel points whose grayscale value is smaller than the preset grayscale value;

[0328] Determining whether the number of the second target pixel points is less than the second preset number of pixel points;

[0329] If the judgment result is yes, obtaining the head confidence of the user currently traversing in the second vehicle interior image;

[0330] Determining whether the head confidence is not less than a preset head confidence threshold;

[0331] If the judgment result is yes, obtaining the head area of ​​the user currently traversing in the second vehicle interior image;

[0332] determining whether an area of ​​a head of the user currently traversing in the second vehicle interior image is not less than a fifth preset head area threshold;

[0333] If the judgment result is yes, the original alarm probability is determined to be the preset alarm probability.

[0334] It should be noted that the above explanation of the method embodiment of the first embodiment is also applicable to the device of the second embodiment. The principle is the same and is not limited in the second embodiment.

[0335] Example 3

[0336] An anti-collision alarm method provided by the embodiment of the present disclosure,

[0337] As shown in FIG11 , the anti-collision alarm method provided by the embodiment of the present disclosure includes the following steps:

[0338] Step 111: Acquire a target image.

[0339] The target image may be an interior image of a vehicle in a vehicle scene, or an interior image of a bedroom scene, which is not specifically limited here.

[0340] The following embodiments are all described using the vehicle interior image in a vehicle scene as an example. The processes of the anti-collision alarm method in other scenes refer to the anti-collision alarm method in the vehicle scene:

[0341] The vehicle interior image is captured by at least one non-depth image collector located within the vehicle cabin. The non-depth image collector can be located in front of the front seats, such as in front of or behind the rearview mirror. The non-depth image collector located in front of the front seats is used to capture a first vehicle interior image of the front seats, the second row of seats, and the aisle area within the cabin.

[0342] The non-depth image collector can also be set at an obstacle, such as behind the entertainment screen, for example, twenty centimeters behind the entertainment screen; the non-depth image collector set behind the entertainment screen is used to collect a second vehicle interior image of the second row of seats, third row of seats and the area under the aisle inside the cabin.

[0343] Based on this, in one embodiment, a first vehicle interior image and / or a second vehicle interior image is obtained; the first vehicle interior image refers to a vehicle interior image captured by a non-depth image collector arranged in front of the front seat, and the second vehicle interior image refers to a vehicle interior image captured by a non-depth image collector arranged behind an obstacle.

[0344] Step 112: When the key position of the human body in the target image falls into the alarm area, output alarm information, which includes basic information of the user.

[0345] Identify the user in the vehicle interior image and obtain the user's user information. If multiple users are identified in the vehicle interior image, obtain the user information of each user.

[0346] Among them, user information includes key positions of the human body and basic user information. Key positions of the human body include: head, elbows, knees and other positions; basic user information includes at least one of the user name and user attributes, and the user attributes include at least one of the user age characteristics and gender characteristics; the user age characteristic is used to indicate whether the user is an adult or a child; the user gender characteristic is used to indicate whether the user is male or female.

[0347] In the following embodiments, the key position of the human body is described as the position of the head.

[0348] Determine whether the key position of the human body falls into the alarm area. If the key position of the human body falls into the alarm area, it means that the user is at risk of colliding with the key part of the human body on the entertainment screen, and output the alarm information of the user to warn the user.

[0349] In one embodiment, the warning message may be a text warning message displayed on the entertainment screen and / or the central control screen, or may be a voice warning message. The content of the text warning message or the voice warning message may be "Little Ming in the front seat, be careful not to hit your head."

[0350] It should be noted that an alarm message can be output each time a user is traversed; the basic user information of the users whose traversals have been completed can also be cached until all users have been traversed, and the basic user information of users whose key parts of the body are at risk of colliding with the entertainment screen can be uniformly output.

[0351] In one embodiment, after outputting an alarm message, the alarm can be turned off and then turned on again after a preset duration. The preset duration can be 10 seconds or other durations, and is not limited in this disclosure. Turning off the alarm after outputting an alarm message not only allows users to perceive the alarm message but also saves computing resources.

[0352] In the above-mentioned embodiments of the present disclosure, by identifying user information in a target image, the user's key body positions and basic user information can be obtained. When the user's key body positions fall within the alarm area, an alarm message containing the user's basic information can be output. This improves the user's perception of the alarm message.

[0353] In an optional specific embodiment, when the key position of the human body in the target image falls into the alarm area, before outputting the alarm information, the method further includes:

[0354] Determining the alarm scene where the user is located according to the human body position in the target image;

[0355] An alarm area corresponding to the alarm scenario is determined according to the alarm scenario and the basic information of the user.

[0356] A human body detection network and a human body key point recognition network are used to identify human bodies and heads in the vehicle interior image, and then the identified human bodies and heads are matched to obtain the human body position and head position of each user.

[0357] According to the coordinates of the user's body position, the alarm scene in which the user is located is determined, and the alarm area is determined based on the alarm scene and the user's basic information. That is, there is an association between the alarm scene, the alarm area and the user's basic information. Based on the alarm scene and the user's basic information, the alarm area corresponding to the alarm scene and the user's basic information can be determined.

[0358] In an optional specific embodiment, when the key position of the human body in the target image falls into the alarm area, before outputting the alarm information, the method further includes:

[0359] Determining the alarm scene where the user is located according to the human body position in the target image;

[0360] According to the alarm scenario, an alarm area corresponding to the alarm scenario is determined.

[0361] A human body detection network and a human body key point recognition network are used to identify human bodies and heads in the vehicle interior image, and then the identified human bodies and heads are matched to obtain the human body position and head position of each user.

[0362] The alarm scene in which the user is located is determined based on the coordinates of the user's body position. Different alarm scenes correspond to different alarm areas. Based on the correspondence between alarm scenes and alarm areas, the alarm area corresponding to the current alarm scene can be determined.

[0363] In one embodiment, the human detection network typically consists of a backbone network, a shoulder network, and a head network, and is used to detect the presence of users in the vehicle interior image and output the coordinates of the bounding box corresponding to each user, such as the coordinates of the upper left and lower right vertices of the bounding box.

[0364] The human body detection network can also output the confidence of the human body bounding box corresponding to each user. Based on this, before outputting the coordinates of the human body bounding box corresponding to each user, it is necessary to delete the human body bounding box whose confidence is less than the preset human body bounding box confidence from the human body bounding box output by the human body detection network.

[0365] In one embodiment, after obtaining the coordinates of the human body external bounding box, an image of each selected position of the human body external bounding box can be cropped from the vehicle interior image to obtain at least one user image; then, a human body key point recognition network is used to identify each of the user images to obtain the coordinates of the human body key points of each of the user images.

[0366] In one embodiment, as shown in FIG3 , key points of a human body refer to points that can represent key positions of a human body, such as points 1 to 24 in FIG3 , where points 1 to 4 represent key points of a human head.

[0367] In an optional specific embodiment, the step of identifying the user in the vehicle interior image and obtaining the user information of the user specifically includes:

[0368] Identifying each user in the vehicle interior image and obtaining basic user information of each user;

[0369] Identify the body and head of each user in the vehicle interior image, and obtain a body circumference frame of each user and a head circumference frame of each head in the vehicle interior image;

[0370] Determine an intersection-and-union (IoU) ratio between each of the human body bounding boxes and each of the head bounding boxes, and match the user corresponding to the human body bounding box with the head bounding box whose IoU ratio is not less than a preset IoU threshold;

[0371] The center point position of the human body circumference frame of the user is determined as the human body position of the user, and the center point position of the head circumference frame of the head matching the user is determined as the head position of the user.

[0372] In one embodiment, the intersection-over-union (IoU) of the human body's circumference frame and the head's circumference frame can reflect the degree of overlap between the human body's circumference frame and the head's circumference frame; therefore, a human body and head with an IoU greater than a preset IoU threshold are usually the human body and head of the same user.

[0373] In one embodiment, the intersection-and-union ratio between a human body circumference frame and each head circumference frame can be determined, and the human body and head corresponding to the human body circumference frame and the head circumference frame whose intersection-and-union ratio is greater than a preset intersection-and-union ratio are determined to be the human body and head of the same user.

[0374] In an optional specific embodiment, when the key position of the human body in the target image falls into the alarm area, outputting the alarm information includes:

[0375] When the key position of the human body falls into the alarm area corresponding to the alarm scene, determining the original alarm probability corresponding to the alarm scene;

[0376] Determining a target alarm probability corresponding to the alarm area according to a preset alarm sensitivity coefficient corresponding to the alarm area and the original alarm probability;

[0377] In response to the target alarm probability being greater than a preset alarm probability threshold, an alarm message is output.

[0378] Different alarm scenarios correspond to different original alarm probabilities. If the head position falls into the alarm area corresponding to the alarm scenario, the original alarm probability corresponding to the alarm scenario is determined.

[0379] The preset alarm sensitivity coefficient is used to indicate the triggering difficulty corresponding to the alarm scenario currently traversed by the user. The preset alarm sensitivity coefficient can be set by the user; for example, through the human-computer interaction interface on the central control screen.

[0380] In one embodiment, taking the alarm scenario of the front seat alarm scenario as an example:

[0381] If the user often leans back from the front seat to talk with the user in the back seat, and at the same time does not expect the vehicle to send out alarm messages all the time, the preset alarm sensitivity coefficient corresponding to the front seat alarm can be set to a value between 0 and 1, such as 0.5, which will reduce the number of alarms corresponding to the front seat alarm scenario to a certain extent.

[0382] In one embodiment, if the user does not want to issue an alarm message, the preset alarm sensitivity coefficient of the corresponding alarm scenario can be set to 0.

[0383] In an optional specific embodiment, when the key position of the human body in the target image falls into the alarm area, before the step of outputting the alarm information, the method includes:

[0384] receiving an alarm area adjustment instruction for adjusting the alarm area;

[0385] The size of the alarm area is scaled according to the alarm area scaling factor in the alarm area adjustment instruction.

[0386] The alarm area adjustment instruction is issued by the user, and the user's alarm area adjustment instruction can be received through the vehicle's central control screen, or the user's voice alarm area adjustment instruction can be received through the microphone.

[0387] In one embodiment, the alarm area corresponding to different alarm scenarios can be enlarged or reduced based on the user's desired alarm sensitivity for different alarm scenarios. For example, the alarm area scaling factor can be 150%, indicating that the alarm area is adjusted to 1.5 times the current alarm area; the alarm area scaling factor can also be 50%, indicating that the alarm area is adjusted to half the current alarm area; this is not limited in this disclosure.

[0388] In an optional specific embodiment, the step 112 of outputting the alarm information includes:

[0389] If no shutdown instruction for shutting down the output of the alarm information is received, the alarm information is output.

[0390] The closing command is issued by the user and can be received through the vehicle's central control screen, a microphone, or a mechanical button. In addition to the closing command, an opening command can also be received.

[0391] In one embodiment, the alarm information corresponding to the alarm scenario may be output or not output according to the user's alarm expectation for different alarm scenarios.

[0392] In one embodiment, if the user in the front seat often leans back to talk with the user in the second seat, in order not to affect the conversation experience, a shutdown command may be issued and the alarm message may not be output.

[0393] In one embodiment, taking the front seat scenario as an example, the user's body is located in the front seat, and their head is located in the alarm area corresponding to the front seat scenario. Therefore, to trigger an alarm for the front seat alarm scenario, it is sufficient to obtain a first vehicle interior image captured by a non-depth image collector located in front of the front seat.

[0394] In an optional specific embodiment, when the key position of the human body falls into the alarm area corresponding to the alarm scene, the step of determining the original alarm probability corresponding to the alarm scene specifically includes:

[0395] In the case where the alarm scene includes a first alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, then determining the original alarm probability corresponding to the alarm scene according to the area of ​​the key position of the human body of the user in the target image; or

[0396] In the case where the alarm scene includes a second alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, the original alarm probability corresponding to the alarm scene is determined based on the area of ​​the key position of the human body of the user in the target image and the grayscale value of each pixel in the target image.

[0397] In one embodiment, when the target image is an image of the interior of a vehicle, the alarm scene includes at least one of a front seat alarm scene, a second row seat alarm scene, an upper aisle area alarm scene, a lower aisle area alarm scene, or a door alarm scene.

[0398] In one embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector disposed in front of a front seat; the alarm scene includes a front seat alarm scene; and the step of determining an original alarm probability corresponding to the alarm scene based on the area of ​​a key position of the user's body in the target image specifically includes:

[0399] Obtaining key body area of ​​the user (e.g., head area) in the first vehicle interior image;

[0400] Determining a first initial alarm probability based on a comparison between an area of ​​a key human body position (e.g., a head area) and a first preset threshold area of ​​a key human body position (e.g., a head area threshold);

[0401] Comparing the key area of ​​the user's body (e.g., head area) with the key area of ​​a standard body (e.g., standard head area) to obtain a relative key area of ​​the body (e.g., relative head area);

[0402] Determining a first alarm probability correction value based on a comparison between the relative human body key position area (e.g., relative head area) and a preset relative human body key position area threshold (e.g., preset relative head area threshold);

[0403] An original alarm probability corresponding to the front seat alarm scenario is obtained according to the first alarm probability correction value and the first initial alarm probability.

[0404] In one embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector disposed in front of a front row of seats and a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; the alarm scenario includes a second-row seat alarm scenario; and the step of determining an original alarm probability corresponding to the alarm scenario based on the area of ​​a key position of the user's body in the target image specifically includes:

[0405] Obtaining key body area of ​​the user (e.g., head area) in the first vehicle interior image;

[0406] Determining a second initial alarm probability based on a comparison between an area of ​​a key human body position (e.g., a head area) and a second preset threshold area of ​​a key human body position (e.g., a head area threshold);

[0407] determining a second alarm probability correction value based on whether a key human position (e.g., a head) matching the user exists in the second vehicle interior image;

[0408] The original alarm probability corresponding to the second row seat alarm scenario is obtained according to the second alarm probability correction value and the second initial alarm probability.

[0409] In one embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector disposed in front of a front seat and a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; the alarm scenario includes an aisle area alarm scenario; and the step of determining an original alarm probability corresponding to the alarm scenario based on the key area of ​​the user's body in the target image specifically includes:

[0410] Obtaining key body area of ​​the user (e.g., head area) in the first vehicle interior image;

[0411] Determining a third initial alarm probability based on a comparison between an area of ​​a key human body position (e.g., a head area) of the user and a third preset threshold area of ​​a key human body position (e.g., a head area threshold);

[0412] determining a third alarm probability correction value based on whether a key human position (e.g., a head) matching the user exists in the second vehicle interior image;

[0413] The original alarm probability corresponding to the area alarm scenario in the aisle is obtained according to the third alarm probability correction value and the third initial alarm probability.

[0414] In one embodiment, the vehicle interior image includes a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; the alarm scene includes an alarm scene in an area under an aisle; and the step of determining an original alarm probability corresponding to the alarm scene based on the area of ​​the key position of the user's body in the target image specifically includes:

[0415] Obtaining key body area (e.g., head area) of the user in the second vehicle interior image;

[0416] Determining a fourth initial alarm probability based on a comparison between an area of ​​a key human body position (e.g., a head area) of the user and a fourth preset threshold area of ​​a key human body position (e.g., a head area threshold);

[0417] Obtaining a grayscale value of each pixel in the second vehicle interior image;

[0418] Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of first target pixel points whose grayscale value is greater than the preset grayscale value;

[0419] Determining a fourth alarm probability correction value based on a comparison between the number of the first target pixel points and the first preset number of pixel points;

[0420] The original alarm probability corresponding to the alarm scene in the area under the aisle is obtained according to the fourth alarm probability correction value and the fourth initial alarm probability.

[0421] In one embodiment, the vehicle interior image includes a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; the alarm scene includes a door alarm scene; and the step of determining an original alarm probability corresponding to the alarm scene based on the area of ​​the key position of the user's body in the target image specifically includes:

[0422] Obtaining a grayscale value of each pixel in the second vehicle interior image;

[0423] Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of second target pixel points whose grayscale value is smaller than the preset grayscale value;

[0424] When the number of the second target pixel points is less than the second preset number of pixel points, obtaining the confidence level of the key position of the human body of the user currently traversed in the second vehicle interior image (e.g., the confidence level of the human head);

[0425] When the key human position confidence (e.g., head confidence) is not less than a preset key human position confidence threshold (e.g., head confidence threshold), obtaining the key human position area (e.g., head area) of the user in the second vehicle interior image;

[0426] When the area of ​​the user's body key position (such as the head area) in the second vehicle interior image is not less than the fifth preset body key position area threshold (such as the fifth preset head area threshold), the original alarm probability corresponding to the door alarm scene is determined to be the preset alarm probability.

[0427] The embodiment content of the method in the above-mentioned embodiment 3 corresponds to the embodiment content of the method in the embodiment 1. For details not disclosed in the embodiment of the method in the embodiment 3, reference can be made to the embodiment content of the method in the embodiment 1, and they will not be repeated in this embodiment 3.

[0428] Example 4

[0429] Corresponding to the anti-collision alarm method of the third embodiment, the present invention further provides an anti-collision alarm device. Since the device embodiment of the present invention corresponds to the method embodiment of the third embodiment, details not disclosed in the device embodiment can be referred to the method embodiment of the third embodiment, and will not be further described in the present invention.

[0430] FIG12 is a schematic structural diagram of an anti-collision alarm device provided by an embodiment of the present disclosure. As shown in FIG12 , the anti-collision alarm device 1200 includes:

[0431] A second acquiring unit 121 is configured to acquire a target image;

[0432] The second output unit 122 is configured to output an alarm message when a key position of a human body in the target image falls into an alarm area. The alarm message includes basic user information.

[0433] In one embodiment, the apparatus further comprises:

[0434] a third determining unit, configured to determine an alarm scene in which the user is located based on a human body position in the target image;

[0435] The fourth determining unit is configured to determine an alarm area corresponding to the alarm scenario according to the alarm scenario and the basic information of the user.

[0436] In one embodiment, the apparatus further comprises:

[0437] a fifth determining unit, configured to determine an alarm scene in which the user is located based on a human body position in the target image;

[0438] A sixth determining unit is configured to determine an alarm area corresponding to the alarm scenario according to the alarm scenario.

[0439] In one embodiment, the second output unit 122 is specifically configured to:

[0440] When the key position of the human body falls into the alarm area corresponding to the alarm scene, determining the original alarm probability corresponding to the alarm scene;

[0441] Determining a target alarm probability corresponding to the alarm area according to a preset alarm sensitivity coefficient corresponding to the alarm area and the original alarm probability;

[0442] In response to the target alarm probability being greater than a preset alarm probability threshold, an alarm message is output.

[0443] In one embodiment, when the key position of the human body falls into the alarm area corresponding to the alarm scene, the second output unit 122 is specifically used to:

[0444] In the case where the alarm scene includes a first alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, then determining the original alarm probability corresponding to the first alarm scene according to the area of ​​the key position of the human body of the user in the target image; or

[0445] In the case that the alarm scene includes a second alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, the original alarm probability corresponding to the second alarm scene is determined based on the area of ​​the key position of the human body of the user in the target image and the grayscale value of each pixel in the target image.

[0446] In an optional specific embodiment, when the target image is an image of the interior of a vehicle, the alarm scene includes at least one of: a front seat alarm scene, a second row seat alarm scene, an upper aisle area alarm scene, a lower aisle area alarm scene, or a door alarm scene.

[0447] In an optional specific embodiment, the vehicle interior image includes a first vehicle interior image acquired by a non-depth image collector disposed in front of a front seat; the alarm scene includes a front seat alarm scene;

[0448] When determining the original alarm probability corresponding to the alarm scene based on the key area of ​​the user's body in the target image, the second output unit 122 is specifically configured to:

[0449] Acquire key areas of the user's body in the first vehicle interior image;

[0450] Determining a first initial alarm probability based on a comparison between an area of ​​a key position of the user's body and a first preset threshold value of an area of ​​a key position of the body;

[0451] Comparing the key position area of ​​the user's body with the key position area of ​​a standard body to obtain a relative key position area of ​​the body;

[0452] Determining a first alarm probability correction value based on a comparison between the area relative to the key position of the human body and a preset threshold value of the area relative to the key position of the human body;

[0453] An original alarm probability corresponding to the front seat alarm scenario is obtained according to the first alarm probability correction value and the first initial alarm probability.

[0454] In an optional specific embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector arranged in front of the front seat and a second vehicle interior image captured by a non-depth image collector arranged behind the obstacle; the alarm scene includes a second-row seat alarm scene;

[0455] When determining the original alarm probability corresponding to the alarm scene based on the key area of ​​the user's body in the target image, the second output unit 122 is specifically configured to:

[0456] Acquire key areas of the user's body in the first vehicle interior image;

[0457] Determining a second initial alarm probability based on a comparison between an area of ​​a key position of the user's body and a second preset threshold value of an area of ​​a key position of the body;

[0458] determining a second alarm probability correction value based on whether a key human body position matching the user exists in the second vehicle interior image;

[0459] The original alarm probability corresponding to the second row seat alarm scenario is obtained according to the second alarm probability correction value and the second initial alarm probability.

[0460] In an optional specific embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector arranged in front of the front seat and a second vehicle interior image captured by a non-depth image collector arranged behind the obstacle; the alarm scene includes an alarm scene in the aisle area;

[0461] When determining the original alarm probability corresponding to the alarm scene based on the key area of ​​the user's body in the target image, the second output unit 122 is specifically configured to:

[0462] Acquire key areas of the user's body in the first vehicle interior image;

[0463] Determining a third initial alarm probability based on a comparison between the area of ​​the user's body key position and a third preset body key position area threshold;

[0464] determining a third alarm probability correction value based on whether a key human position matching the user is present in the second vehicle interior image;

[0465] The original alarm probability corresponding to the area alarm scenario in the aisle is obtained according to the third alarm probability correction value and the third initial alarm probability.

[0466] In an optional specific embodiment, the vehicle interior image includes a second vehicle interior image acquired by a non-depth image collector disposed behind the obstacle; the alarm scene includes an alarm scene in the area under the aisle;

[0467] When determining the original alarm probability corresponding to the alarm scene based on the key area of ​​the user's body in the target image, the second output unit 122 is specifically configured to:

[0468] Obtaining key areas of the user's body in the second vehicle interior image;

[0469] Determining a fourth initial alarm probability based on a comparison between an area of ​​a key position of the user's body and a fourth preset threshold value of an area of ​​a key position of the body;

[0470] Obtaining a grayscale value of each pixel in the second vehicle interior image;

[0471] Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of first target pixel points whose grayscale value is greater than the preset grayscale value;

[0472] Determining a fourth alarm probability correction value based on a comparison between the number of the first target pixel points and the first preset number of pixel points;

[0473] The original alarm probability corresponding to the alarm scene in the area under the aisle is obtained according to the fourth alarm probability correction value and the fourth initial alarm probability.

[0474] In an optional specific embodiment, the vehicle interior image includes a second vehicle interior image acquired by a non-depth image collector disposed behind an obstacle; the alarm scene includes a door alarm scene;

[0475] When determining the original alarm probability corresponding to the alarm scene based on the key area of ​​the user's body in the target image, the second output unit 122 is specifically configured to:

[0476] Obtaining a grayscale value of each pixel in the second vehicle interior image;

[0477] Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of second target pixel points whose grayscale value is smaller than the preset grayscale value;

[0478] When the number of the second target pixel points is less than the second preset number of pixel points, obtaining the confidence level of the key position of the user's body currently traversed in the second vehicle interior image;

[0479] When the confidence level of the key human body position is not less than a preset key human body position confidence level threshold, obtaining the key human body position area of ​​the user in the second vehicle interior image;

[0480] When the area of ​​the user's body key position in the second vehicle interior image is not less than a fifth preset body key position area threshold, the original alarm probability corresponding to the door alarm scene is determined to be the preset alarm probability.

[0481] In an optional specific embodiment, the device further includes:

[0482] a first receiving unit, configured to receive an alarm area adjustment instruction for adjusting the alarm area;

[0483] The first scaling unit is configured to scale the size of the alarm area according to the alarm area scaling coefficient in the alarm area adjustment instruction.

[0484] In an optional specific embodiment, when outputting the alarm information, the second output unit 122 is specifically used to:

[0485] If no shutdown instruction for shutting down the output of the alarm information is received, the alarm information is output.

[0486] It should be noted that the above explanation of the method embodiment of the third embodiment is also applicable to the device of the fourth embodiment. The principles are the same and are no longer limited in the fourth embodiment.

[0487] Example 5

[0488] An anti-collision alarm method provided by an embodiment of the present disclosure can be applied to a terminal equipped with an IR camera, such as a vehicle equipped with an IR camera. The executor of the method can be the controller of the vehicle's anti-collision alarm module or the vehicle's entire vehicle controller.

[0489] As shown in FIG13 , the anti-collision alarm method provided by the embodiment of the present disclosure includes the following steps:

[0490] Step 131: Acquire a target image.

[0491] The target image may be an interior image of a vehicle in a vehicle scene, or an interior image of a bedroom scene, which is not specifically limited here.

[0492] The following embodiments are all described using the vehicle interior image in a vehicle scene as an example. The processes of the anti-collision alarm method in other scenes refer to the anti-collision alarm method in the vehicle scene:

[0493] The vehicle interior image is captured by at least one non-depth image collector located within the vehicle cabin. The non-depth image collector can be located in front of the front seats, such as in front of or behind the rearview mirror. The non-depth image collector located in front of the front seats is used to capture a first vehicle interior image of the front seats, the second row of seats, and the aisle area within the cabin.

[0494] The non-depth image collector can also be set at an obstacle, such as behind the entertainment screen, for example, twenty centimeters behind the entertainment screen; the non-depth image collector set behind the entertainment screen is used to collect a second vehicle interior image of the second row of seats, third row of seats and the area under the aisle inside the cabin.

[0495] Based on this, in one embodiment, a first vehicle interior image and / or a second vehicle interior image is obtained; the first vehicle interior image refers to a vehicle interior image captured by a non-depth image collector arranged in front of the front seat, and the second vehicle interior image refers to a vehicle interior image captured by a non-depth image collector arranged behind an obstacle.

[0496] Step 132: Determine the alarm scene where the user is located based on the position of the human body in the target image.

[0497] Identify the user in the vehicle interior image and obtain the user's user information. If multiple users are identified in the vehicle interior image, obtain the user information of each user.

[0498] A human body detection network and a human body key point recognition network are used to identify human bodies and heads in the vehicle interior image, and then the identified human bodies and heads are matched to obtain the human body position and head position of each user.

[0499] The user information includes key positions of the human body and the position of the human body. The key positions of the human body include: the head, elbows, knees and other positions.

[0500] In the following embodiments, the key position of the human body is described as the position of the head.

[0501] The coordinates of the user's body position are obtained, and based on the coordinates of the body position, the alarm scene in which the user is located is determined.

[0502] Step 133: Outputting an alarm message when the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene.

[0503] Different alarm scenarios correspond to different alarm areas. Based on the correspondence between alarm scenarios and alarm areas, the alarm area corresponding to the current alarm scenario can be determined. If the head position falls within the alarm area corresponding to the alarm scenario, an alarm message for the user is output. This alarm message can be generated based on the alarm area corresponding to the alarm scenario, or the alarm message can include the alarm scenario and its corresponding alarm area.

[0504] In the above-mentioned embodiments of the present disclosure, by identifying user information in a target image, the user's key body positions and body location can be determined. Different alarm scenarios correspond to different alarm zones. When the user's key body position falls within the alarm zone corresponding to an alarm scenario, user alarm information can be generated and output based on the alarm scenario and its corresponding alarm zone. This improves the accuracy of collision avoidance alarms.

[0505] In an optional specific embodiment, in step 133, when the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene, before outputting the alarm information, the method further includes:

[0506] Determining an alarm area corresponding to the alarm scenario according to the alarm scenario and the basic information of the user;

[0507] The alarm information includes at least one of the user basic information, the alarm scenario and the alarm area.

[0508] The user information also includes basic user information, which includes at least one of the user's name and user attributes. The user attributes include at least one of the user's age characteristics and gender characteristics. The user's age characteristics are used to indicate whether the user is an adult or a child; the user's gender characteristics are used to indicate whether the user is male or female.

[0509] According to the coordinates of the user's body position, the alarm scene in which the user is located is determined, and the alarm area is determined based on the alarm scene and the user's basic information. That is, there is an association between the alarm scene, the alarm area and the user's basic information. Based on the alarm scene and the user's basic information, the alarm area corresponding to the alarm scene and the user's basic information can be determined.

[0510] In one embodiment, the human detection network typically consists of a backbone network, a shoulder network, and a head network, and is used to detect the presence of users in the vehicle interior image and output the coordinates of the bounding box corresponding to each user, such as the coordinates of the upper left and lower right vertices of the bounding box.

[0511] The human body detection network can also output the confidence of the human body bounding box corresponding to each user. Based on this, before outputting the coordinates of the human body bounding box corresponding to each user, it is necessary to delete the human body bounding box whose confidence is less than the preset human body bounding box confidence from the human body bounding box output by the human body detection network.

[0512] In one embodiment, after obtaining the coordinates of the human body external bounding box, an image of each selected position of the human body external bounding box can be cropped from the vehicle interior image to obtain at least one user image; then, a human body key point recognition network is used to identify each of the user images to obtain the coordinates of the human body key points of each of the user images.

[0513] In one embodiment, as shown in FIG3 , key points of a human body refer to points that can represent key positions of a human body, such as points 1 to 24 in FIG3 , where points 1 to 4 represent key points of a human head.

[0514] In an optional specific embodiment, step 112 of identifying a user in the vehicle interior image and obtaining user information of the user specifically includes:

[0515] Identifying each user in the vehicle interior image and obtaining basic user information of each user;

[0516] Identify the human body and head of each user in the vehicle interior image, and obtain a human body circumference frame of each user and a head circumference frame of each head in the vehicle interior image;

[0517] Determine an intersection-and-union (IoU) ratio between each of the human body bounding boxes and each of the head bounding boxes, and match the user corresponding to the human body bounding box with the head bounding box whose IoU ratio is not less than a preset IoU threshold;

[0518] The center point position of the human body circumference frame of the user is determined as the human body position of the user, and the center point position of the head circumference frame of the head matching the user is determined as the head position of the user.

[0519] In one embodiment, the intersection-over-union (IoU) of the human body's circumference frame and the head's circumference frame can reflect the degree of overlap between the human body's circumference frame and the head's circumference frame; therefore, a human body and head with an IoU greater than a preset IoU threshold are usually the human body and head of the same user.

[0520] In one embodiment, the intersection-and-union ratio between a human body circumference frame and each head circumference frame can be determined, and the human body and head corresponding to the human body circumference frame and the head circumference frame whose intersection-and-union ratio is greater than a preset intersection-and-union ratio are determined to be the human body and head of the same user.

[0521] In an optional specific embodiment, when the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene, outputting the alarm information includes:

[0522] When the key position of the human body falls into the alarm area corresponding to the alarm scene, determining the original alarm probability corresponding to the alarm scene;

[0523] Determining a target alarm probability corresponding to the alarm area according to a preset alarm sensitivity coefficient corresponding to the alarm area and the original alarm probability;

[0524] In response to the target alarm probability being greater than a preset alarm probability threshold, an alarm message is output.

[0525] Different alarm scenarios correspond to different original alarm probabilities. If the head position falls into the alarm area corresponding to the alarm scenario, the original alarm probability corresponding to the alarm scenario is determined.

[0526] The preset alarm sensitivity coefficient is used to indicate the triggering difficulty corresponding to the alarm scenario currently traversed by the user. The preset alarm sensitivity coefficient can be set by the user; for example, through the human-computer interaction interface on the central control screen.

[0527] In one embodiment, taking the alarm scenario of the front seat alarm scenario as an example:

[0528] If the user often leans back from the front seat to talk with the user in the back seat, and at the same time does not expect the vehicle to send out alarm messages all the time, the preset alarm sensitivity coefficient corresponding to the front seat alarm can be set to a value between 0 and 1, such as 0.5, which will reduce the number of alarms corresponding to the front seat alarm scenario to a certain extent.

[0529] In one embodiment, if the user does not want to issue an alarm message, the preset alarm sensitivity coefficient of the corresponding alarm scenario can be set to 0.

[0530] In an optional specific embodiment, when the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene, before the step of outputting the alarm information, the method includes:

[0531] receiving an alarm area adjustment instruction for adjusting the alarm area;

[0532] The size of the alarm area is scaled according to the alarm area scaling factor in the alarm area adjustment instruction.

[0533] The alarm area adjustment instruction is issued by the user, and the user's alarm area adjustment instruction can be received through the vehicle's central control screen, or the user's voice alarm area adjustment instruction can be received through the microphone.

[0534] In one embodiment, the alarm area corresponding to different alarm scenarios can be enlarged or reduced based on the user's desired alarm sensitivity for different alarm scenarios. For example, the alarm area scaling factor can be 150%, indicating that the alarm area is adjusted to 1.5 times the current alarm area; the alarm area scaling factor can also be 50%, indicating that the alarm area is adjusted to half the current alarm area; this is not limited in this disclosure.

[0535] In an optional specific embodiment, the step of outputting the alarm information includes:

[0536] If no shutdown instruction for shutting down the output of the alarm information is received, the alarm information is output.

[0537] The closing command is issued by the user and can be received through the vehicle's central control screen, a microphone, or a mechanical button. In addition to the closing command, an opening command can also be received.

[0538] In one embodiment, the alarm information corresponding to the alarm scenario may be output or not output according to the user's alarm expectation for different alarm scenarios.

[0539] In one embodiment, if the user in the front seat often leans back to talk with the user in the second seat, in order not to affect the conversation experience, a shutdown command may be issued and the alarm message may not be output.

[0540] In one embodiment, taking the front seat scenario as an example, the user's body is located in the front seat, and their head is located in the alarm area corresponding to the front seat scenario. Therefore, to trigger an alarm for the front seat alarm scenario, it is sufficient to obtain a first vehicle interior image captured by a non-depth image collector located in front of the front seat.

[0541] In an optional specific embodiment, when the key position of the human body falls into the alarm area corresponding to the alarm scene, the step of determining the original alarm probability corresponding to the alarm scene specifically includes:

[0542] In the case where the alarm scene includes a first alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, then determining the original alarm probability corresponding to the alarm scene according to the area of ​​the key position of the human body of the user in the target image; or

[0543] In the case where the alarm scene includes a second alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, the original alarm probability corresponding to the alarm scene is determined based on the area of ​​the key position of the human body of the user in the target image and the grayscale value of each pixel in the target image.

[0544] In one embodiment, when the target image is an image of the interior of a vehicle, the alarm scene includes at least one of a front seat alarm scene, a second row seat alarm scene, an upper aisle area alarm scene, a lower aisle area alarm scene, or a door alarm scene.

[0545] In one embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector disposed in front of a front seat; the alarm scene includes a front seat alarm scene; and the step of determining an original alarm probability corresponding to the alarm scene based on the area of ​​a key position of the user's body in the target image specifically includes:

[0546] Obtaining key body area of ​​the user (e.g., head area) in the first vehicle interior image;

[0547] Determining a first initial alarm probability based on a comparison between an area of ​​a key human body position (e.g., a head area) and a first preset threshold area of ​​a key human body position (e.g., a head area threshold);

[0548] Comparing the key area of ​​the user's body (e.g., head area) with the key area of ​​a standard body (e.g., standard head area) to obtain a relative key area of ​​the body (e.g., relative head area);

[0549] Determining a first alarm probability correction value based on a comparison between the relative human body key position area (e.g., relative head area) and a preset relative human body key position area threshold (e.g., preset relative head area threshold);

[0550] An original alarm probability corresponding to the front seat alarm scenario is obtained according to the first alarm probability correction value and the first initial alarm probability.

[0551] In one embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector disposed in front of a front row of seats and a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; the alarm scenario includes a second-row seat alarm scenario; and the step of determining an original alarm probability corresponding to the alarm scenario based on the area of ​​a key position of the user's body in the target image specifically includes:

[0552] Obtaining key body area of ​​the user (e.g., head area) in the first vehicle interior image;

[0553] Determining a second initial alarm probability based on a comparison between an area of ​​a key human body position (e.g., a head area) and a second preset threshold area of ​​a key human body position (e.g., a head area threshold);

[0554] determining a second alarm probability correction value based on whether a key human position (e.g., a head) matching the user exists in the second vehicle interior image;

[0555] The original alarm probability corresponding to the second row seat alarm scenario is obtained according to the second alarm probability correction value and the second initial alarm probability.

[0556] In one embodiment, the vehicle interior image includes a first vehicle interior image captured by a non-depth image collector disposed in front of a front seat and a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; the alarm scenario includes an aisle area alarm scenario; and the step of determining an original alarm probability corresponding to the alarm scenario based on the key area of ​​the user's body in the target image specifically includes:

[0557] Obtaining key body area of ​​the user (e.g., head area) in the first vehicle interior image;

[0558] Determining a third initial alarm probability based on a comparison between an area of ​​a key human body position (e.g., a head area) of the user and a third preset threshold area of ​​a key human body position (e.g., a head area threshold);

[0559] determining a third alarm probability correction value based on whether a key human position (e.g., a head) matching the user exists in the second vehicle interior image;

[0560] The original alarm probability corresponding to the area alarm scenario in the aisle is obtained according to the third alarm probability correction value and the third initial alarm probability.

[0561] In one embodiment, the vehicle interior image includes a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; the alarm scene includes an alarm scene in an area under an aisle; and the step of determining an original alarm probability corresponding to the alarm scene based on the area of ​​the key position of the user's body in the target image specifically includes:

[0562] Obtaining key body area (e.g., head area) of the user in the second vehicle interior image;

[0563] Determining a fourth initial alarm probability based on a comparison between an area of ​​a key human body position (e.g., a head area) of the user and a fourth preset threshold area of ​​a key human body position (e.g., a head area threshold);

[0564] Obtaining a grayscale value of each pixel in the second vehicle interior image;

[0565] Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of first target pixel points whose grayscale value is greater than the preset grayscale value;

[0566] Determining a fourth alarm probability correction value based on a comparison between the number of the first target pixel points and the first preset number of pixel points;

[0567] The original alarm probability corresponding to the alarm scene in the area under the aisle is obtained according to the fourth alarm probability correction value and the fourth initial alarm probability.

[0568] In one embodiment, the vehicle interior image includes a second vehicle interior image captured by a non-depth image collector disposed behind an obstacle; the alarm scene includes a door alarm scene; and the step of determining an original alarm probability corresponding to the alarm scene based on the area of ​​the key position of the user's body in the target image specifically includes:

[0569] Obtaining a grayscale value of each pixel in the second vehicle interior image;

[0570] Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of second target pixel points whose grayscale value is smaller than the preset grayscale value;

[0571] When the number of the second target pixel points is less than the second preset number of pixel points, obtaining the confidence level of the key position of the human body of the user currently traversed in the second vehicle interior image (e.g., the confidence level of the human head);

[0572] When the key human position confidence (e.g., head confidence) is not less than a preset key human position confidence threshold (e.g., head confidence threshold), obtaining the key human position area (e.g., head area) of the user in the second vehicle interior image;

[0573] When the area of ​​the user's body key position (such as the head area) in the second vehicle interior image is not less than the fifth preset body key position area threshold (such as the fifth preset head area threshold), the original alarm probability corresponding to the door alarm scene is determined to be the preset alarm probability.

[0574] The embodiment content of the method in the above-mentioned embodiment 5 corresponds to the embodiment content of the method in the embodiment 1. For details not disclosed in the embodiment of the method in the embodiment 5, reference can be made to the embodiment content of the method in the embodiment 1, and they will not be repeated in this embodiment 5.

[0575] Example 6

[0576] Corresponding to the anti-collision alarm method of the fifth embodiment, the present invention further provides an anti-collision alarm device. Since the device embodiment of the present invention corresponds to the method embodiment of the fifth embodiment, details not disclosed in the device embodiment can be referred to the method embodiment of the fifth embodiment, and will not be further described in the present invention.

[0577] FIG14 is a schematic structural diagram of an anti-collision alarm device provided by an embodiment of the present disclosure. As shown in FIG14 , the anti-collision alarm device 1400 includes:

[0578] The third acquisition unit 141 is used to acquire a target image;

[0579] A second determining unit 142 is configured to determine the alarm scene in which the user is located based on the position of the human body in the target image;

[0580] The third output unit 143 is configured to output alarm information when the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene.

[0581] In one embodiment, the apparatus further comprises:

[0582] a seventh determining unit, configured to determine an alarm area corresponding to the alarm scenario according to the alarm scenario and the basic information of the user;

[0583] The alarm information includes at least one of the user basic information, the alarm scenario and the alarm area.

[0584] In one embodiment, the third output unit 143 is specifically configured to:

[0585] When the key position of the human body falls into the alarm area corresponding to the alarm scene, determining the original alarm probability corresponding to the alarm scene;

[0586] Determining a target alarm probability corresponding to the alarm area according to a preset alarm sensitivity coefficient corresponding to the alarm area and the original alarm probability;

[0587] In response to the target alarm probability being greater than a preset alarm probability threshold, an alarm message is output.

[0588] In one embodiment, when the key position of the human body falls into the alarm area corresponding to the alarm scene, the third output unit 144 is specifically used to:

[0589] In the case where the alarm scene includes a first alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, then determining the original alarm probability corresponding to the first alarm scene according to the area of ​​the key position of the human body of the user in the target image; or

[0590] In the case that the alarm scene includes a second alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, the original alarm probability corresponding to the second alarm scene is determined based on the area of ​​the key position of the human body of the user in the target image and the grayscale value of each pixel in the target image.

[0591] In one embodiment, when the target image is an image of the interior of a vehicle, the alarm scene includes at least one of a front seat alarm scene, a second row seat alarm scene, an upper aisle area alarm scene, a lower aisle area alarm scene, or a door alarm scene.

[0592] In one embodiment, the apparatus comprises:

[0593] a second receiving unit, configured to receive an alarm area adjustment instruction for adjusting the alarm area;

[0594] The second scaling unit is configured to scale the size of the alarm area according to the alarm area scaling coefficient in the alarm area adjustment instruction.

[0595] In one embodiment, when outputting the alarm information, the third output unit 143 is specifically used to:

[0596] If no shutdown instruction for shutting down the output of the alarm information is received, the alarm information is output.

[0597] It should be noted that the above explanation of the method embodiment of the fifth embodiment is also applicable to the device of the sixth embodiment. The principles are the same and are no longer limited in the sixth embodiment.

[0598] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a non-transitory computer-readable storage medium storing computer instructions, and a vehicle.

[0599] Specifically, an embodiment of the present disclosure provides an electronic device, including:

[0600] at least one processor; and

[0601] a memory communicatively connected to at least one processor; wherein,

[0602] The memory stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor so that the at least one processor can perform the steps of the method of the aforementioned embodiment one, embodiment three, or embodiment five.

[0603] An embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the steps of the method of the aforementioned embodiment 1, embodiment 3, or embodiment 5.

[0604] An embodiment of the present disclosure provides a vehicle, wherein the vehicle includes the apparatus of the aforementioned second embodiment, fourth embodiment, or sixth embodiment, or the aforementioned electronic device.

[0605] FIG15 shows a schematic block diagram of an example electronic device 1100 that can be used to implement an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, vehicle-mounted devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0606] As shown in Figure 15, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1102 or a computer program loaded from a storage unit 1108 into a RAM (Random Access Memory) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An I / O (Input / Output) interface 1105 is also connected to the bus 1104.

[0607] Various components in device 1100 are connected to an I / O interface 1105, including an input unit 1104, such as a keyboard and mouse; an output unit 1107, such as various types of displays and speakers; a storage unit 1108, such as a magnetic disk and optical disk; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows device 1100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0608] The computing unit 1101 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), various specialized AI (Artificial Intelligence) computing chips, various computing units that run machine learning model algorithms, a DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as the head collision prevention alarm method. For example, in some embodiments, the head collision prevention alarm method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the aforementioned anti-head collision alarm method in any other appropriate manner (for example, by means of firmware).

[0609] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System on Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0610] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0611] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0612] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0613] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A collision avoidance alarm method, comprising: Acquire the target image; When a key position of a human body in the target image falls into an alarm area, an alarm message is output, where the alarm message includes basic information of the user.

2. The method according to claim 1, wherein In the case where the key position of the human body in the target image falls into the alarm area, before outputting the alarm information, the method further includes: Determining the alarm scene where the user is located according to the human body position in the target image; An alarm area corresponding to the alarm scenario is determined according to the alarm scenario and the basic information of the user.

3. The method according to claim 1 or 2, wherein: In the case where the key position of the human body in the target image falls into the alarm area, before outputting the alarm information, the method further includes: Determining the alarm scene where the user is located according to the human body position in the target image; According to the alarm scenario, an alarm area corresponding to the alarm scenario is determined.

4. The method according to claim 2 or 3, wherein: When the key position of the human body in the target image falls into the alarm area, outputting alarm information includes: When the key position of the human body falls into the alarm area corresponding to the alarm scene, determining the original alarm probability corresponding to the alarm scene; Determining a target alarm probability corresponding to the alarm area according to a preset alarm sensitivity coefficient corresponding to the alarm area and the original alarm probability; In response to the target alarm probability being greater than a preset alarm probability threshold, an alarm message is output.

5. The method according to claim 4, wherein When the key position of the human body falls into the alarm area corresponding to the alarm scene, determining the original alarm probability corresponding to the alarm scene includes: In the case where the alarm scene includes a first alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, then determining the original alarm probability corresponding to the first alarm scene according to the area of ​​the key position of the human body of the user in the target image; or In the case that the alarm scene includes a second alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, the original alarm probability corresponding to the second alarm scene is determined based on the area of ​​the key position of the human body of the user in the target image and the grayscale value of each pixel in the target image.

6. The method according to any one of claims 2 to 5, wherein: When the target image is an image of the interior of a vehicle, the alarm scene includes at least one of a front seat alarm scene, a second row seat alarm scene, an upper aisle area alarm scene, a lower aisle area alarm scene, or a door alarm scene.

7. The method according to claim 6, wherein: The vehicle interior image includes a first vehicle interior image acquired by a non-depth image collector arranged in front of the front seat; the alarm scene includes a front seat alarm scene; The determining, based on the key area of ​​the user's body in the target image, the original alarm probability corresponding to the alarm scene includes: Acquire key areas of the user's body in the first vehicle interior image; Determining a first initial alarm probability based on a comparison between an area of ​​a key position of the user's body and a first preset threshold value of an area of ​​a key position of the body; Comparing the key position area of ​​the user's body with the key position area of ​​a standard body to obtain a relative key position area of ​​the body; Determining a first alarm probability correction value based on a comparison between the area relative to the key position of the human body and a preset threshold value of the area relative to the key position of the human body; An original alarm probability corresponding to the front seat alarm scenario is obtained according to the first alarm probability correction value and the first initial alarm probability.

8. The method according to claim 6 or 7, wherein: The vehicle interior image includes a first vehicle interior image acquired by a non-depth image collector arranged in front of the front seat and a second vehicle interior image acquired by a non-depth image collector arranged behind the obstacle; the alarm scene includes a second-row seat alarm scene; The determining, based on the key area of ​​the user's body in the target image, the original alarm probability corresponding to the alarm scene includes: Acquire key areas of the user's body in the first vehicle interior image; Determining a second initial alarm probability based on a comparison between an area of ​​a key position of the user's body and a second preset threshold value of an area of ​​a key position of the body; determining a second alarm probability correction value based on whether a key human body position matching the user exists in the second vehicle interior image; The original alarm probability corresponding to the second row seat alarm scenario is obtained according to the second alarm probability correction value and the second initial alarm probability.

9. The method according to any one of claims 6 to 8, wherein: The vehicle interior image includes a first vehicle interior image acquired by a non-depth image collector arranged in front of the front seat and a second vehicle interior image acquired by a non-depth image collector arranged behind the obstacle; the alarm scene includes an alarm scene in the aisle area; The determining, based on the key area of ​​the user's body in the target image, the original alarm probability corresponding to the alarm scene includes: Acquire key areas of the user's body in the first vehicle interior image; Determining a third initial alarm probability based on a comparison between the area of ​​the user's body key position and a third preset body key position area threshold; determining a third alarm probability correction value based on whether a key human position matching the user is present in the second vehicle interior image; The original alarm probability corresponding to the area alarm scenario in the aisle is obtained according to the third alarm probability correction value and the third initial alarm probability.

10. The method according to any one of claims 6 to 9, wherein: The vehicle interior image includes a second vehicle interior image acquired by a non-depth image collector disposed behind the obstacle; the alarm scene includes an alarm scene in the area under the aisle; The determining, based on the key area of ​​the user's body in the target image, the original alarm probability corresponding to the alarm scene includes: Obtaining key areas of the user's body in the second vehicle interior image; Determining a fourth initial alarm probability based on a comparison between an area of ​​a key position of the user's body and a fourth preset threshold value of an area of ​​a key position of the body; Obtaining a grayscale value of each pixel in the second vehicle interior image; Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of first target pixel points whose grayscale value is greater than the preset grayscale value; Determining a fourth alarm probability correction value based on a comparison between the number of the first target pixel points and the first preset number of pixel points; The original alarm probability corresponding to the alarm scene in the area under the aisle is obtained according to the fourth alarm probability correction value and the fourth initial alarm probability.

11. The method according to any one of claims 6 to 10, wherein: The vehicle interior image includes a second vehicle interior image acquired by a non-depth image collector disposed behind an obstacle; the alarm scene includes a vehicle door alarm scene; The determining, based on the key area of ​​the user's body in the target image, the original alarm probability corresponding to the alarm scene includes: Obtaining a grayscale value of each pixel in the second vehicle interior image; Comparing the grayscale value of each pixel with a preset grayscale value, and determining the number of second target pixel points whose grayscale value is smaller than the preset grayscale value; When the number of the second target pixel points is less than the second preset number of pixel points, obtaining the confidence level of the key position of the user's body currently traversed in the second vehicle interior image; When the confidence level of the key human body position is not less than a preset key human body position confidence level threshold, obtaining the key human body position area of ​​the user in the second vehicle interior image; When the area of ​​the user's body key position in the second vehicle interior image is not less than a fifth preset body key position area threshold, the original alarm probability corresponding to the door alarm scene is determined to be the preset alarm probability.

12. The method according to any one of claims 1 to 11, wherein: In the case where the key position of the human body in the target image falls into the alarm area, before outputting the alarm information, the method includes: receiving an alarm area adjustment instruction for adjusting the alarm area; The size of the alarm area is scaled according to the alarm area scaling factor in the alarm area adjustment instruction.

13. The method according to any one of claims 1 to 12, wherein: The output alarm information includes: If no shutdown instruction for shutting down the output of the alarm information is received, the alarm information is output.

14. An anti-collision alarm method, comprising: Acquire the target image; Determining the alarm scene where the user is located according to the human body position in the target image; When the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene, an alarm message is output.

15. The method according to claim 14, wherein When the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene, before outputting the alarm information, the method further includes: Determining an alarm area corresponding to the alarm scenario according to the alarm scenario and the basic information of the user; The alarm information includes at least one of the user basic information, the alarm scenario and the alarm area.

16. The method according to claim 14 or 15, wherein: When the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene, outputting the alarm information includes: When the key position of the human body falls into the alarm area corresponding to the alarm scene, determining the original alarm probability corresponding to the alarm scene; Determining a target alarm probability corresponding to the alarm area according to a preset alarm sensitivity coefficient corresponding to the alarm area and the original alarm probability; In response to the target alarm probability being greater than a preset alarm probability threshold, an alarm message is output.

17. The method according to claim 16, wherein When the key position of the human body falls into the alarm area corresponding to the alarm scene, determining the original alarm probability corresponding to the alarm scene includes: In the case where the alarm scene includes a first alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, then determining the original alarm probability corresponding to the first alarm scene according to the area of ​​the key position of the human body of the user in the target image; or In the case that the alarm scene includes a second alarm scene, if the key position of the human body falls into the alarm area corresponding to the alarm scene, the original alarm probability corresponding to the second alarm scene is determined based on the area of ​​the key position of the human body of the user in the target image and the grayscale value of each pixel in the target image.

18. The method according to claim 17, wherein When the target image is an image of the interior of a vehicle, the alarm scene includes at least one of a front seat alarm scene, a second row seat alarm scene, an upper aisle area alarm scene, a lower aisle area alarm scene, or a door alarm scene.

19. The method according to any one of claims 14 to 18, wherein: In the case where the key position of the human body in the target image falls into the alarm area corresponding to the alarm scene, before outputting the alarm information, the method includes: receiving an alarm area adjustment instruction for adjusting the alarm area; The size of the alarm area is scaled according to the alarm area scaling factor in the alarm area adjustment instruction.

20. The method according to any one of claims 14 to 19, wherein The output alarm information includes: If no shutdown instruction for shutting down the output of the alarm information is received, the alarm information is output.

21. A head collision prevention alarm method, comprising: Acquire a vehicle interior image; wherein the vehicle interior image refers to a vehicle interior image acquired by at least one non-depth image collector disposed in the vehicle cabin; Identifying user information of at least one user in the vehicle interior image to obtain user information of each user; wherein the user information includes location information and basic user information of the user; the location information includes a body position and a head position, and the basic user information includes at least one of a user name and a user attribute; Traversing the position information of each user in the vehicle interior image, and determining the alarm scene of the currently traversed user according to the body position of the currently traversed user; According to the currently traversed head position of the user, determining whether the currently traversed head position of the user falls into the alarm area corresponding to the alarm scene; wherein different alarm scenes correspond to different alarm areas; If the judgment result is yes, the alarm information of the user currently traversed is output; the alarm information includes the basic user information of the user currently traversed.

22. An anti-head collision alarm device, comprising: A first acquisition unit is configured to acquire a vehicle interior image; wherein the vehicle interior image refers to a vehicle interior image acquired by at least one non-depth image acquirer disposed in the vehicle cabin; a first recognition unit, configured to recognize user information of at least one user in the vehicle interior image and obtain user information of each user; wherein the user information includes location information and basic user information of the user; the location information includes a body position and a head position; and the basic user information includes at least one of a user name and a user attribute; a first determining unit, configured to traverse the position information of each user in the vehicle interior image, and determine an alarm scene in which the currently traversed user is located based on the body position of the currently traversed user; a judgment unit, configured to judge, based on the currently traversed head position of the user, whether the currently traversed head position of the user falls into an alarm area corresponding to the alarm scenario; wherein different alarm scenarios correspond to different alarm areas; The first output unit is configured to output the alarm information of the currently traversed user if the judgment result is yes; the alarm information includes at least one of the body position, head position or user attribute of the currently traversed user.

23. An anti-collision alarm device, comprising: A second acquisition unit, configured to acquire a target image; The second output unit is configured to output alarm information when a key position of a human body in the target image falls into an alarm area, wherein the alarm information includes basic user information.

24. An anti-collision alarm device, comprising: A third acquisition unit is used to acquire a target image; a second determining unit, configured to determine an alarm scene in which the user is located based on a human body position in the target image; The third output unit is configured to output alarm information when a key position of a human body in the target image falls into an alarm area corresponding to the alarm scene.

25. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 21.

26. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method according to any one of claims 1 to 21.

27. A vehicle comprising the apparatus according to any one of claims 22 to 24 or the electronic device according to claim 25.

Images