Information prompting method and device, vehicle, equipment and storage medium

Abstract

The invention discloses an information prompting method. The method comprises the following steps: extracting received image information to obtain target image information; determining an early warning monitoring area based on the state of the target monitoring object, wherein the early warning monitoring area is an area where the target monitoring object is located or an area covered by the target monitoring object in a movement process; according to the target image information, whether a target object is in an early warning monitoring area is detected through a preset condition, and a detection result is obtained; according to the detection result, whether the target image information meets a preset prompt condition or not is judged, information prompt is triggered according to the judgment result, and the preset prompt condition comprises an image area threshold value. The collision risk corresponding to the passenger can be detected based on the form of the obstacle, information prompting is carried out in time when the passenger has the collision risk, and the safety risk problem that no prompting is provided when the passenger has the collision risk is solved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technologies, and particularly to an information prompting method, apparatus, vehicle, device, and storage medium. Background Art

[0002] Currently, with the continuous development of intelligent vehicles, some intelligent devices may be installed in the vehicle cabin for passengers' entertainment. Although these intelligent devices bring convenience to users, they may also pose obstacles when installed in the rear row of the cabin. When users fail to notice the obstacles, there is an unnecessary risk of bumping. Moreover, with the diversification of intelligent devices, the forms of intelligent devices are becoming more and more diverse. Different obstacle forms will correspondingly bring different degrees of potential bumping risks to passengers. In scenarios such as when passengers change seats or get off the vehicle, they may accidentally bump into the intelligent devices without noticing, resulting in passenger injuries. Based on this problem, a method that can detect the corresponding bumping risks of passengers based on the form of obstacles and promptly give information prompts when passengers are at risk of bumping becomes necessary. Summary of the Invention

[0003] To solve the above problems, the present disclosure provides an information prompting method, apparatus, and vehicle, as well as an information prompting method, apparatus, vehicle, device, and storage medium. This method can detect the corresponding bumping risks of passengers based on the form of obstacles and promptly give information prompts when passengers are at risk of bumping, solving the safety risk problem of no prompt when passengers are at risk of bumping.

[0004] According to a first aspect of the present disclosure, there is provided an information prompting method, including:

[0005] Performing extraction processing on the received image information to obtain target image information;

[0006] Determining a warning monitoring area based on the state of a target monitoring object, where the warning monitoring area is the area where the target monitoring object is located or the area covered by the target monitoring object during movement;

[0007] According to the target image information, detecting whether a target object is in the warning monitoring area through a preset condition to obtain a detection result;

[0008] Based on the detection result, determining whether the target image information meets a preset prompt condition, and triggering an information prompt according to the determination result, where the preset prompt condition includes an image area threshold.

[0009] As an optional implementation manner of an embodiment of the present disclosure, the target image information includes valid image data information of a target area, and extracting and processing the received image information to obtain the target image information, where the target image information includes valid image data information of a target area, and extracting and processing the received image information to obtain the target image information includes:

[0010] Extracting depth data information of the image information, where the depth data information includes a horizontal axis depth value, a vertical axis depth value, and a vertical axis depth value;

[0011] Setting the vertical axis depth value exceeding a preset vertical axis depth value threshold to zero, and at the same time setting the horizontal axis depth value and the vertical axis depth value corresponding to the vertical axis depth value to zero to obtain a target area depth value;

[0012] Traversing the target area depth value, filtering out depth values with the vertical axis depth value greater than a preset distance, and removing isolated point depth values to obtain a valid target depth value.

[0013] As an optional implementation manner of an embodiment of the present disclosure, the state of the target monitoring object includes that the target monitoring object is in a moving state; determining a warning monitoring area based on the state of the target monitoring object includes:

[0014] When the state of the target monitoring object is that the target monitoring object is in a moving state, determining the warning monitoring area as a first preset distance range outside the fan-shaped area swept by the target monitoring object during movement, and setting this area as the first warning monitoring area.

[0015] As an optional implementation manner of an embodiment of the present disclosure, the state of the target monitoring object further includes that the target monitoring object is in a stationary state; determining a warning monitoring area based on the state of the target monitoring object includes:

[0016] When the screen state is that the target monitoring object is in a stationary state, determining the warning monitoring area as an area at a second preset distance from the target monitoring object in the vertical distance as the second warning monitoring area.

[0017] As an optional implementation manner of an embodiment of the present disclosure, detecting whether a target object is in a warning monitoring area according to the target image information through a preset condition to obtain a detection result includes:

[0018] When the state of the target monitoring object is that the target monitoring object is in a moving state;

[0019] Converting the abscissa dimension in the depth data information to the screen rotation axis dimension to obtain new depth data information;

[0020] Calculate the straight-line distance between the target image and the rotating shaft according to the new depth data information;

[0021] Take the minimum value of the straight-line distances between the target image and the rotating shaft, compare the minimum value with the preset straight-line distance threshold, and when the minimum value is less than the preset straight-line distance threshold, the detection result is that the target image is in the first warning monitoring area.

[0022] As an optional implementation manner of the embodiment of the present disclosure, the image area threshold includes a first preset area threshold. Based on the detection result, determine whether the target image information meets the preset prompt condition, and trigger an information prompt according to the determination result, including:

[0023] When the detection result is that the target image is in the first warning monitoring area, calculate the area of the target image according to the effective target depth value;

[0024] Compare the area of the target image with the first preset area threshold. If the area of the target image is greater than the first preset area threshold, trigger an information prompt.

[0025] As an optional implementation manner of the embodiment of the present disclosure, detect whether the target object is in the warning monitoring area through preset conditions to obtain a detection result, including:

[0026] When the vertical distance between the target image and the target monitoring object is less than a second preset distance, the detection result is that the target image is in the second warning monitoring area.

[0027] As an optional implementation manner of the embodiment of the present disclosure, the extraction and processing of the received image information to obtain the target image information further includes: extracting the image data information of the image information; performing human detection processing on the image data information to determine whether there is target human information in the image data information to obtain a determination result,

[0028] The determination of whether the target image information meets the preset prompt condition based on the detection result and the triggering of the information prompt according to the determination result include:

[0029] When the detection result is that the target image is in the second warning monitoring area, calculate the area of the target image according to the effective target depth value;

[0030] Compare the area of the target image with the second preset area threshold. If the area of the target image is greater than the second preset area threshold and the determination result is that there is target human information in the image data information, trigger an information prompt.

[0031] According to a second aspect of the present disclosure, the present disclosure provides an information prompting device, including:

[0032] A processing module, configured to perform extraction processing on the received image information to obtain target image information, where the target image information includes effective image data information of a target area;

[0033] A determination module, configured to determine a warning monitoring area based on the state of a target monitoring object, where the warning monitoring area is the area where the target monitoring object is located or the area covered by the target monitoring object during movement;

[0034] A detection module, configured to detect whether a target object is in the warning monitoring area according to the target image information through a preset condition, and obtain a detection result;

[0035] A control module, configured to determine whether the target image information meets a preset prompting condition according to the detection result, and trigger information prompting according to the determination result, where the preset prompting condition includes an image area threshold.

[0036] According to a third aspect of the present disclosure, the present disclosure provides a readable storage medium, on which a computer program is stored, characterized in that when the computing program is executed by a processor, it implements the information prompting method according to any one of the embodiments in the first aspect.

[0037] According to a fourth aspect of the present disclosure, the present disclosure provides an electronic device, including:

[0038] At least one processor; and

[0039] A memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the steps of the information prompting method according to any one of the embodiments in the first aspect.

[0041] According to a fifth aspect of the present disclosure, the present disclosure provides a vehicle, including: the information prompting device according to the second aspect described above.

[0042] The technical solutions provided by the embodiments of the present disclosure have the following advantages compared with the prior art: The present disclosure provides an information prompting method, which includes: extracting and processing the received image information to obtain target image information, where the target image information includes effective image data information of a target area; determining a warning monitoring area based on the state of a target monitoring object, where the warning monitoring area is the area where the target monitoring object is located or the area covered by the target monitoring object during movement; according to the target image information, detecting whether a target object is in the warning monitoring area through a preset condition to obtain a detection result; through the detection result, judging whether the target image information meets a preset prompting condition, and triggering an information prompt according to the judgment result, where the preset prompting condition includes an image area threshold. The present disclosure can determine a warning monitoring area based on the state of a target monitoring object, corresponding different warning monitoring areas to monitoring objects in different states, and according to the target image information, detecting whether the target image is in the warning monitoring area. If the target passenger is already in the warning monitoring area, further judge whether it meets the preset prompting condition. If the preset prompting condition is met, trigger an information prompt. This method can detect targets in the warning monitoring areas corresponding to monitoring objects in different states, and when the target meets the preset prompting condition, trigger an information prompt, solving the safety risk problem of no prompt when there is a risk of passengers bumping.

[0043] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understandable through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure and, together with the specification, are used to explain the principles of the present disclosure.

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0046] Figure 1 is a flowchart of an information prompting method provided in the first aspect of the embodiments of the present disclosure;

[0047] Figure 2 is another implementation flowchart of an information prompting method provided in the first aspect of the embodiments of the present disclosure;

[0048] Figure 3It is another schematic flowchart of an information prompting method provided in the first aspect of the embodiments of the present disclosure;

[0049] Figure 4 It is a schematic diagram of the distribution of the early warning monitoring area under different target monitoring object states provided in the first aspect of the embodiments of the present disclosure;

[0050] Figure 5 It is a schematic structural diagram of an information prompting device provided in the second aspect of the embodiments of the present disclosure;

[0051] Figure 6 It is a schematic structural diagram of an electronic device provided in the fourth aspect of the embodiments of the present disclosure. Detailed implementation manners

[0052] In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments may be combined with each other.

[0053] Many specific details are set forth in the following description in order to fully understand the present disclosure, but the present disclosure may be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all the embodiments.

[0054] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner. In addition, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" refers to two or more.

[0055] It should be noted that, in this article, the term "comprising" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0056] The embodiments of the present application provide an information prompting method, which is specifically applied to vehicles. Figure 1A flowchart of an information prompting method provided in the first aspect of the embodiments of the present disclosure. This control method can be executed by a control device or equipment for information prompting. The device or equipment can be configured in a server, a processor, or a main control chip. Exemplarily, it can be deployed on the in-vehicle unit side, the domain controller side, etc. of a vehicle. Refer to Figure 1 As shown, the information prompting method includes the following steps S101 - S104:

[0057] S101. Extract and process the received image information to obtain target image information, where the target image information includes valid image data information of a target area.

[0058] In some embodiments, the received image can be used to collect image information inside the vehicle through an image acquisition device provided inside the vehicle. Among them, the provided image acquisition device can be a pre - installed camera, an image sensor, etc. inside the vehicle. Optionally, the types of the camera and the image sensor can be any cameras capable of acquiring image information, such as a monocular IR camera, a binocular IR camera, or a TOF camera. The camera and the image sensor are used to collect images inside the vehicle compartment, and the processor receives the image information collected by the camera and the image sensor, performs extraction and processing to obtain target image information. The extraction includes extracting depth data information of the image information and extracting image data information, etc. Data processing is performed on the extracted depth data and the extracted image data information of the image information, filtering invalid information data, etc., to obtain target image information.

[0059] S102. Determine a warning monitoring area based on the state of a target monitoring object, where the warning monitoring area is the area where the target monitoring object is located or the area covered by the target monitoring object during movement.

[0060] In some embodiments, the warning monitoring area is determined based on the state of the target monitoring object. The target monitoring object can be a set - up flip - up display screen, a rotatable audio device, or other movable intelligent devices for passenger entertainment, without limitation here. In this technical solution, the target monitoring object is taken as an example of a flip - up display screen. For example, Figure 4 A distribution schematic diagram of the warning monitoring area under different states of the target monitoring object provided in the first aspect of the embodiments of the present disclosure. Based on the fact that the target monitoring object is in the process of movement, it is determined that the warning monitoring area is the area covered during movement, such as Figure 4 the warning monitoring area A shown; when the target monitoring object is in a stationary state, the warning monitoring area is the area where the target monitoring object is located, such as Figure 4 the warning monitoring area B shown.

[0061] S103. According to the target image information, detect whether a target object is in the warning monitoring area through a preset condition to obtain a detection result.

[0062] In some embodiments, based on the target image information obtained according to the above steps, a preset condition is used to detect whether the target object is in the early warning monitoring area, that is, whether the position of the passenger is in the dangerous area. The target image information includes depth data information and image data information. During the movement of the screen, a first preset distance outside the fan-shaped area swept by the screen during the movement is defined as the first early warning monitoring area. According to the depth data information in the target image information, the straight-line distance between the target passenger and Figure 4 the central axis shown in the figure is compared with the preset distance condition to obtain the detection result; when the screen is in a stationary state, a range perpendicular to the screen and at a second preset distance is defined as the second early warning monitoring area. According to the depth data information in the target image information, the straight-line distance between the target passenger and Figure 4 the monitoring object in the figure is compared with the preset distance condition to obtain the detection result. The above detection result includes that the target passenger is / is not in the first early warning monitoring area / the second early warning monitoring area.

[0063] S103. Based on the detection result, determine whether the target image information meets the preset prompt condition, and trigger an information prompt according to the determination result, where the preset prompt condition includes an image area threshold.

[0064] In some embodiments, determining whether the target image information meets the preset prompt condition based on the detection result specifically means that when the detection result obtained by detecting the target passenger in the target image information is that the passenger is in the first early warning monitoring area or in the second early warning monitoring area, a further determination is triggered. According to the depth information data and / or image information data in the target image information, it is determined whether the image information of the target passenger meets the preset prompt condition. The preset prompt condition is one or more of a preset distance threshold, an image area threshold, and a face detection result. When the distance threshold, image area threshold, and face detection result of the target image information meet the preset distance threshold and face detection result, an information prompt is triggered. When the target image information already meets the above prompt condition, the target passenger already has a risk of bumping, and an information prompt is given in a timely manner to avoid unnecessary bumping danger.

[0065] Exemplarily, when the above-mentioned passenger is at risk of bumping, it may be a scenario where a rear passenger exchanges seats with other passengers or fails to notice the state of the monitoring object when getting off the vehicle, etc., all scenarios where activities need to be carried out and there may be a risk of injury.

[0066] Specifically, the information prompt includes, but is not limited to, voice prompts, light prompts and other prompt methods that may attract the attention of passengers. This technical solution is not limited here. Taking voice prompts as an example, an alarm sound can be set to prompt passengers that they are in a dangerous state; custom voices such as "Please be careful of hitting your head" or "You have entered a dangerous area and there is a risk of bumping" can also be set to prompt passengers, attracting the attention of passengers and avoiding the problem of bumping. Taking light prompts as an example, a safety prompt light can be set to control the atmosphere light to flash red or yellow lights. At the same time, the flashing frequency of the lights can be set to attract the attention of passengers to the safety problem of bumping.

[0067] The information prompt method provided by the embodiment of the present application is specifically as follows: extracting and processing the received image information to obtain target image information; determining a warning monitoring area based on the state of the target monitoring object, where the warning monitoring area is the area where the target monitoring object is located or the area covered by the target monitoring object during movement; according to the target image information, detecting whether the target object is in the warning monitoring area through a preset condition to obtain a detection result; through the detection result, judging whether the target image information meets a preset prompt condition, and triggering an information prompt according to the judgment result, where the preset prompt condition includes an image area threshold. The embodiment of the present application can determine a warning monitoring area based on the state of the target monitoring object, corresponding different warning monitoring areas to monitoring objects in different states, and detecting whether the target image is in the warning monitoring area according to the target image information. If the target passenger is already in the warning monitoring area, it is further judged whether it meets the preset prompt condition. If the preset prompt condition is met, an information prompt is triggered. This method can perform target detection on the warning monitoring areas corresponding to monitoring objects in different states, and when the target meets the preset prompt condition, trigger an information prompt, solving the safety risk problem of no prompt when passengers are at risk of bumping, and providing a safer riding experience and service.

[0068] As an extension and refinement of the above embodiment, refer to Figure 2 As shown, another implementation flowchart of an information prompt method provided by the first aspect of the embodiment of the present disclosure, the prompt method includes the following steps S201-S205

[0069] S201. Extract and process the received image to obtain depth data information.

[0070] In some embodiments, consistent with the above method, the received image is the image information of the cabin collected by an image acquisition device provided inside the vehicle. Among them, the provided image acquisition device can be a pre - installed camera, image sensor, etc. inside the vehicle. Optionally, the types of the camera and image sensor can be any cameras that can obtain image information, such as a monocular IR camera, a binocular IR camera, or a TOF camera. The camera and image sensor are used to collect images of the vehicle cabin, and the processor receives the image information collected by the camera and image sensor and extracts the depth data of the image information. The depth data information can be extracted by methods such as stereo vision method (by using binocular cameras to shoot the same scene and calculating the depth information of objects using the parallax between the two images), structured light method (using a projector to project a specific light pattern and calculating the depth information of objects by shooting the light pattern reflected by the object with a camera), optical flow method (inferring the depth information of objects by calculating the pixel displacement between adjacent frames), and other methods that can achieve the extraction of depth data information.

[0071] Specifically, taking the TOF camera as an example, the image information collected by the TOF camera in each frame includes two parts: depth data information and image data information. Among them, the resolution of the image data information is 640x480, and the depth data information includes the depth of each pixel point in the image data in the x, y, and z directions relative to the camera. Therefore, the depth data information is a matrix with a dimension of 640x480x3. After obtaining a frame of data from the TOF camera and extracting the depth data information, the above - mentioned depth data information includes the depth values in the x, y, and z directions, that is, the horizontal axis depth value, the vertical axis depth value, and the vertical axis depth value. Taking the direction of the rear row seats relative to the rear of the vehicle as the vertical axis direction, after extracting the depth image information of the image, in order to further reduce the influence of incorrect depth values on the algorithm effect, taking the three - row seats as an example, measure the actual distance of the three - row area as the preset vertical axis depth value. Since there is no phenomenon of exceeding the ranging in the values of the horizontal and vertical axes, while setting the vertical axis depth value exceeding the preset vertical axis depth value threshold to zero, set the corresponding horizontal axis depth value and vertical axis depth value of the vertical axis depth value to zero, so that the depth value validity of the three - dimensional matrix at the same pixel point is consistent, and obtain the depth value of the target area.

[0072] Exemplarily, after obtaining the depth value of the target area, traverse the depth value of the target area, and filter out the depth values whose vertical axis depth value is greater than a preset distance. Optionally, after in-cabin testing, when the object is within 380 mm from the TOF camera, it is defined as a nearby object. Filter out all pixel points in the matrix whose vertical axis depth value in the z direction (i.e., the vertical axis direction) is less than 380 mm to obtain the depth value of the nearby object, and eliminate the depth value of isolated points. An isolated point is a non-connected point. Taking the current pixel point as the center, if the depth value at this point differs from the depth value of any one of the surrounding 8 pixel points by less than 20 mm, then this point is considered to be connected. The unconnected pixel points are isolated points and are regarded as invalid data and are not included in the statistics for elimination processing. After processing, an effective target depth value is obtained.

[0073] S202. Determine a first warning monitoring area based on the motion state of the target monitoring object.

[0074] Exemplarily, the state of the target monitoring object includes that the target monitoring object is in a motion state. When the state of the target monitoring object is that the target monitoring object is in a motion state, as Figure 4 shown, determine the warning monitoring area as the first preset distance range outside the fan-shaped area swept by the target monitoring object during the motion process, and set this area as the first warning monitoring area. Specifically, the first preset distance is 200 mm, and a 200 mm range outside the fan-shaped area swept by the screen during the motion is delimited as the first warning monitoring area, as Figure 4 shown as monitoring area A.

[0075] S203. According to the target image information, detect whether the target object is in the first warning monitoring area through preset conditions to obtain a detection result.

[0076] In some embodiments, according to the effective target depth value obtained in the above steps, perform preset condition detection to determine whether the target object is in the first warning monitoring area. When the state of the target monitoring object is that the target monitoring object is in a motion state, convert the abscissa dimension in the depth data information to the screen rotation axis dimension to obtain new depth data information; calculate the straight-line distance between the target image and the rotation axis according to the new depth data information; take the minimum value of the straight-line distance between the target image and the rotation axis, and compare the minimum value with the preset straight-line distance threshold. When the minimum value is less than the preset straight-line distance threshold, the detection result is that the target image is in the first warning monitoring area. Specifically, in the calculation process, first convert the depth matrix x, y, z under the camera view to the depth matrix xT, yT, zT with the screen rotation axis as the view. Since the rotation axis is in the x direction, we can calculate the straight-line distance D of all pixel points in the image relative to the rotation axis according to formula 1:

[0077] D = √(〖z_T〗^2+〖y_T〗^2)

[0078] After obtaining the straight-line distance D of all pixels, we find the minimum straight-line distance Dmin and set a preset straight-line distance threshold Dth. When the minimum straight-line distance Dmin is less than the preset straight-line distance threshold Dth, it means that the detection result is that the target passenger has entered the first warning monitoring area, that is, has entered monitoring area A.

[0079] S204. When the detection result is that the target image is in the first warning monitoring area, calculate the area of the target image according to the effective target depth value.

[0080] Exemplarily, when the detection result is that the target image is in the first warning monitoring area, calculate the area of the target image according to the effective target depth value. The area of the above target image is the area of the nearby object. According to the depth value coordinates (x, y, z) corresponding to the nearby object, use the area calculation formula to calculate the area S1 of the nearby object, that is, the area S1 of the target image.

[0081] S205. Compare the area of the target image with a first preset area threshold. If the area of the target image is greater than the first preset area threshold, trigger an information prompt.

[0082] Exemplarily, after obtaining the area S1 of the target image, compare S1 with a first preset area threshold Sth1. The first preset area threshold Sth1 can be set to any value between 1000 and 1500 cm 2 For example, taking the first preset area threshold as 1000 cm 2 as an example, if the calculated area of the target image is 1200 cm 2 , that is, greater than the first preset area threshold of 1000 cm 2 , then when the passenger is in a moving state on the screen, there is a risk of bumping and meets the conditions for information prompting. Trigger an information prompt. The information prompt includes but is not limited to voice prompts, light prompts and other prompting methods that may attract the attention of passengers. This technical solution is not limited here. Taking the voice prompt as an example, an alarm sound can be set to prompt the passenger that they are in a dangerous state; a custom voice such as "Please be careful of hitting your head" or "You have entered a dangerous area and there is a risk of bumping" can also be set to prompt the passenger to attract their attention and avoid the problem of bumping. Taking the light prompt as an example, a safety prompt light can be set to control the atmosphere light to flash red or yellow lights. At the same time, the flashing frequency of the lights can be set to attract the passenger's attention to the safety problem of bumping.

[0083] As an extension and refinement of the above embodiments, refer to Figure 3As shown in the figure, it is another implementation process schematic diagram of an information prompt method provided in the first aspect of the embodiments of the present disclosure. The prompt method includes the following steps S301-S305:

[0084] S301. Perform extraction processing on the received image to obtain depth data information and image data information.

[0085] In some embodiments, the extraction method of the depth data information has been described and exemplified in S201. The processor receives the image information collected by the shooting camera and the image sensor, and extracts the depth data and image data in the image information respectively. The depth data is a matrix with a dimension of 640x480x3, and the resolution of the image data is 640×480. Then, the depth data information and the image data information are processed, such as filtering the coordinates of points that do not meet the ranging range and removing the coordinates of isolated points as described above.

[0086] S302. Perform human body detection processing on the image data information to determine whether there is target human body information in the image data information, and obtain a judgment result.

[0087] Exemplarily, input the image data information into the human body detection model for inference to obtain the target image information of the preset area. The human body detection model can be selected as a model based on a deep convolutional network (Convolutional Neural Network, CNN). By stacking multiple convolutional layers and fully connected layers to extract human body features, or three-dimensional face modeling and local mean normalization can also be used to learn human body features, which can quickly identify human faces, etc. In this technical solution, no specific limitation is imposed on the human body detection model, and any model that can implement the human body detection function can be applied to the technical solution of the present disclosure. Based on the human body features, human body key points, human body position and other information inferred from the image in the above human body detection model, determine whether there is target human body information in the image data information, and obtain a judgment result. The judgment result is that there is target human body information or there is no target human body information in the image data information.

[0088] S303. Determine the second warning monitoring area based on the stationary state of the target monitoring object.

[0089] In some embodiments, according to the effective target depth value obtained in the above steps, perform preset condition detection to determine whether the target object is in the second warning monitoring area. When the screen state is that the target monitoring object is in a stationary state, determine that the warning monitoring area is the area at a second preset distance from the target monitoring object in the vertical distance as the second warning monitoring area. Specifically, the second preset distance can be set to 350mm, and the area at a vertical distance of 350mm from the screen is delimited as the area where there may be bumps, as shown in the monitoring area B in Figure 4 as shown.

[0090] S304. Detect whether the target object is in the second warning monitoring area according to the target image information to obtain a detection result.

[0091] In some embodiments, according to the effective target depth value obtained in the above steps, perform a preset condition to detect whether the target object is in the second warning monitoring area. When the state of the target monitoring object is that the target monitoring object is in a stationary state, and when the vertical distance between the target image and the target monitoring object is less than a second preset distance, the detection result is that the target image is in the second warning monitoring area. Specifically, when the second preset distance is set to 350 mm, that is, when the depth value in the longitudinal axis direction of the target image is less than 350 mm from the vertical distance of the monitoring object, the detection result is that the target image is in the second warning monitoring area.

[0092] S305. When the detection result is that the target image is in the second warning monitoring area, calculate the area of the target image according to the effective target depth value.

[0093] Exemplarily, when the detection result is that the target image is in the second warning monitoring area, calculate the area of the target image according to the effective target depth value. The area of the above target image is the area of the nearby object. According to the depth value coordinates (x, y, z) corresponding to the nearby object, use the area calculation formula to calculate the area S2 of the nearby object, that is, the area S2 of the target image.

[0094] S306. Compare the area of the target image with a second preset area threshold. If the area of the target image is greater than the second preset area threshold and the judgment result is that there is target human body information in the image data information, trigger an information prompt.

[0095] Exemplarily, after obtaining the area S2 of the target image, compare S2 with a second preset area threshold Sth2. The second preset area threshold Sth2 can be set to any value between 1250 and 1750 cm 2 because when the screen is in a stationary state, without the straight-line distance of the pixel points relative to the rotating shaft as an auxiliary judgment, in order to prevent misdetection, set Sth2 > Sth1. Taking the second preset area threshold as 1500 cm 2 as an example, if the calculated area of the target image is 1800 cm 2 , that is, greater than the second preset area threshold of 1500 cm 2, meanwhile, the judgment result obtained in the above-mentioned human body detection and processing process is that there is target human body information in the image data information. In order to prevent false alarms caused by frame-by-frame errors that may occur when only using the area as the judgment condition when the monitored object is in a static state. Therefore, we fuse the image data information in the TOF data for human body detection and judgment. When the object area is greater than the preset area threshold and target human body information is detected, it means that there is a risk of bumping when the passenger is in a static state on the screen and meets the conditions for information prompt, triggering information prompt. The information prompt includes but is not limited to prompt methods that may attract the attention of passengers such as voice prompt, light prompt, etc. This technical solution is not limited here. Taking voice prompt as an example, an alarm sound can be set to prompt the passenger that they are in a dangerous state; custom voices such as "Please be careful of hitting your head" or "You have entered a dangerous area and there is a risk of bumping" can also be set to prompt the passenger and attract their attention to avoid the problem of bumping. Taking light prompt as an example, a safety prompt light can be set to control the atmosphere light to flash red or yellow lights, and the flashing frequency of the light can also be set to attract the attention of passengers to the safety problem of bumping.

[0096] Based on the same inventive concept, as an implementation of the above method, an embodiment of the present application further provides an information prompt device. This embodiment corresponds to the foregoing method embodiment. For the convenience of reading, details of the foregoing method embodiment will not be repeated one by one in this embodiment. However, it should be clear that the target detection device in this embodiment can correspondingly implement all the contents of the foregoing method embodiment.

[0097] An embodiment of the present application provides an information prompt device, Figure 5 which is a structural schematic diagram of an information prompt device provided in the second aspect of the present disclosure embodiment. As Figure 5 shown, the information prompt device 500 includes:

[0098] A processing module 501, configured to extract and process the received image information to obtain target image information, where the target image information includes valid image data information of a target area;

[0099] A determination module 502, configured to determine a warning monitoring area based on the state of the target monitored object, where the warning monitoring area is the area where the target monitored object is located or the area covered by the target monitored object during movement;

[0100] A detection module 503, configured to detect whether a target object is in the warning monitoring area according to the target image information through a preset condition, and obtain a detection result;

[0101] The control module 504 is configured to determine whether the target image information meets the preset prompt conditions based on the detection result, and trigger an information prompt according to the determination result, where the preset prompt conditions include an image area threshold.

[0102] As an optional implementation manner of an embodiment of the present application, the processing module is specifically configured to receive the image information collected by the shooting camera and the image sensor, respectively extract the depth data and the image data in the image information, and process the depth data information and the image data information, such as filtering the coordinates of points that do not meet the ranging range and removing the coordinates of isolated points as described above.

[0103] As an optional implementation manner of an embodiment of the present application, the determining module is specifically configured to determine a corresponding first early warning monitoring area or a second early warning monitoring area based on the moving or stationary state of the target monitoring object.

[0104] As an optional implementation manner of an embodiment of the present application, the detecting module is specifically configured to detect whether the target object is in the first early warning monitoring area or in the second early warning monitoring area according to the target image information through preset conditions, and obtain a detection result that the target detection object is in / not in the first early warning monitoring area / the second early warning monitoring area.

[0105] As an optional implementation manner of an embodiment of the present application, the control module is specifically configured to, based on the detection result that the target detection object is in the first early warning monitoring area / the second early warning monitoring area, correspondingly determine whether the target image information meets the preset prompt conditions, and trigger an information prompt according to the determination result, where the preset prompt conditions include an image area threshold and a human body detection result.

[0106] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computing device is enabled to implement the information prompt method provided in the above embodiment.

[0107] Based on the same inventive concept, an embodiment of the present disclosure further provides an electronic device. Figure 6FIG. 0 is a schematic structural diagram of an electronic device provided in the fourth aspect of the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0108] As Figure 6 shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 602 or a computer program loaded from a storage unit 608 into a RAM (Random Access Memory) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An I / O (Input / Output) interface 605 is also connected to the bus 604.

[0109] A plurality of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0110] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as the information prompting method. For example, in some embodiments, the information prompting method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the methods described above can be executed. Alternatively, in other embodiments, the computing unit 601 can be configured to execute the foregoing communication method in any other suitable manner (e.g., by means of firmware).

[0111] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, FPGA (Field Programmable Gate Array), ASIC (Application-Specific Integrated Circuit), ASSP (Application Specific Standard Product), SOC (System On Chip), CPLD (Complex Programmable Logic Device), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0112] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program codes can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0113] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only-Memory), or a flash memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0114] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or an LCD (Liquid Crystal Display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0115] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0116] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server can also be a server of a distributed system, or a server combined with blockchain.

[0117] It should be noted that artificial intelligence is a discipline that studies how to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), and it has both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0118] Based on the same inventive concept, an embodiment of the present application further provides a vehicle, which includes the information prompting device provided in the above embodiment or the electronic device provided in the above embodiment.

[0119] It should be understood that various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitation is made herein.

[0120] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. An information prompting method, characterized in that, Including: Performing extraction processing on the received image information to obtain target image information; Determining a warning monitoring area based on the state of the target monitoring object, where the warning monitoring area is the area where the target monitoring object is located or the area covered by the target monitoring object during movement; According to the target image information, detecting whether the target object is in the warning monitoring area through preset conditions to obtain a detection result; Based on the detection result, determining whether the target image information meets the preset prompt conditions, and triggering an information prompt according to the determination result, where the preset prompt conditions include an image area threshold.

2. The information prompting method according to claim 1, wherein The target image information includes effective image data information of the target area. The performing extraction processing on the received image information to obtain target image information includes: Extracting the depth data information of the image information, where the depth data information includes a horizontal axis depth value, a vertical axis depth value, and a vertical axis depth value; Setting the vertical axis depth value exceeding the preset vertical axis depth value threshold to zero, and at the same time setting the corresponding horizontal axis depth value and vertical axis depth value of the vertical axis depth value to zero to obtain the target area depth value; Traversing the target area depth value, filtering out the depth values with the vertical axis depth value greater than the preset distance, and removing the isolated point depth values to obtain the effective target depth value.

3. The prompting method according to claim 2, wherein The state of the target monitoring object includes that the target monitoring object is in a moving state; The determining a warning monitoring area based on the state of the target monitoring object includes: When the state of the target monitoring object is that the target monitoring object is in a moving state, determining the warning monitoring area as the first preset distance range outside the fan-shaped area swept by the target monitoring object during movement, and setting this area as the first warning monitoring area.

4. The prompting method according to claim 2, wherein The state of the target monitoring object further includes that the target monitoring object is in a stationary state; Determining a warning monitoring area based on the state of the target monitoring object includes: When the screen state is that the target monitoring object is in a stationary state, determining the warning monitoring area as the area at a second preset distance perpendicular to the target monitoring object as the second warning monitoring area.

5. The prompting method according to claim 3, wherein The detecting whether the target object is in the warning monitoring area through preset conditions according to the target image information to obtain a detection result includes: When the state of the target monitoring object is that the target monitoring object is in a moving state; Converting the abscissa dimension in the depth data information to the screen rotation axis dimension to obtain new depth data information; Calculating the straight-line distance between the target image and the rotation axis according to the new depth data information; Taking the minimum value of the straight-line distance between the target image and the rotation axis, comparing the minimum value with the preset straight-line distance threshold, and when the minimum value is less than the preset straight-line distance threshold, the detection result is that the target image is in the first warning monitoring area.

6. The prompting method according to claim 5, wherein The image area threshold includes a first preset area threshold. The determining whether the target image information meets the preset prompt conditions based on the detection result and triggering an information prompt according to the determination result includes: When the detection result is that the target image is in the first warning monitoring area, calculate the area of the target image according to the effective target depth value; Compare the area of the target image with a first preset area threshold. If the area of the target image is greater than the first preset area threshold, trigger an information prompt.

7. The prompting method according to claim 4, wherein According to the target image information, detect whether the target object is in the warning monitoring area through preset conditions to obtain a detection result, including: When the vertical distance between the target image and the target monitoring object is less than a second preset distance, the detection result is that the target image is in the second warning monitoring area.

8. The prompting method according to claim 7, wherein, The extraction and processing of the received image information to obtain target image information further includes: extracting the image data information of the image information; performing human body detection processing on the image data information to determine whether there is target human body information in the image data information to obtain a judgment result. The determination of whether the target image information meets the preset prompt conditions based on the detection result and triggering an information prompt according to the judgment result includes: When the detection result is that the target image is in the second warning monitoring area, calculate the area of the target image according to the effective target depth value; Compare the area of the target image with a second preset area threshold. If the area of the target image is greater than the second preset area threshold and the judgment result is that there is target human body information in the image data information, trigger an information prompt.

9. An information prompting device, characterized in that, Including: A processing module for extracting and processing the received image information to obtain target image information, where the target image information includes effective image data information of a target area; A determination module for determining a warning monitoring area based on the state of a target monitoring object, where the warning monitoring area is the area where the target monitoring object is located or the area covered by the target monitoring object during movement; A detection module for detecting whether a target object is in the warning monitoring area through preset conditions according to the target image information to obtain a detection result; A control module for determining whether the target image information meets the preset prompt conditions based on the detection result and triggering an information prompt according to the judgment result, where the preset prompt conditions include an image area threshold.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, the steps of the method according to any one of claims 1-8 are implemented.

11. An electronic device, characterized in that, Including: A memory having a computer program stored thereon; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.

12. A vehicle, characterized in that, Including the anti-collision head warning device according to claim 9.

Images