Method, device and equipment for monitoring life bodies remaining in a vehicle and storage medium

Abstract

The application discloses a kind of in-vehicle life retention life monitoring method, device, equipment and storage medium, in-vehicle life retention life monitoring method includes: obtaining the in-vehicle image of vehicle in power-off and vehicle door lock state;Target detection is carried out based on in-vehicle image, and target detection result is obtained;In response to target detection result including unknown classification area, warning information is sent to the terminal device of target object;Wherein, warning information is used to prompt target object to determine whether there is in-vehicle life retention life, and warning information at least includes the image data of image in unknown classification area in in-vehicle image.By the above-mentioned mode, when in-vehicle life retention life cannot be determined by target detection, whether there is in-vehicle life retention life is further determined by target object, and the accuracy of monitoring in-vehicle life retention life is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobile safety protection, and in particular to a monitoring method, device and equipment for a life body remaining in a vehicle and a storage medium. BACKGROUND

[0002] With the continuous popularity of automobiles, there are more and more safety accidents in vehicles. For example, when a driver leaves a vehicle, the driver forgets that there are other life bodies, such as children and pets, remaining in the vehicle, which leads to the life bodies remaining in the closed vehicle. With the passage of time, gases, such as carbon dioxide, which have adverse effects on the life bodies, gather in the vehicle, or the temperature in the vehicle rises, which makes the environment in the vehicle bad and may threaten the life safety of the life bodies remaining in the vehicle.

[0003] If the remaining life bodies in the vehicle can be accurately monitored after the driver leaves the vehicle, safety protection measures can be timely intervened, and the possibility of safety accidents of the remaining life bodies can be reduced. Therefore, how to improve the accuracy of monitoring the remaining life bodies in the vehicle becomes a technical problem to be solved. SUMMARY

[0004] The present application mainly solves the technical problem of providing a monitoring method, device, equipment and computer readable storage medium for a life body remaining in a vehicle, which can improve the accuracy of monitoring the remaining life bodies in the vehicle.

[0005] To solve the above technical problem, one technical solution adopted by the present application is to provide a monitoring method for a life body remaining in a vehicle, the method comprising: acquiring a vehicle interior image in a power-off and door-locked state of the vehicle; performing target detection based on the vehicle interior image to obtain a target detection result; and in response to the target detection result including an unknown classification region, sending warning information to a terminal device of a target object, wherein the warning information is used to prompt the target object to determine whether there is a remaining life body in the vehicle, and the warning information at least includes image data of the unknown classification region in the vehicle interior image.

[0006] In the case where the target detection result includes the unknown classification region, before the warning information is sent to the terminal device of the target object, the method further comprises: determining whether there is a moving object in the unknown classification region; and sending the warning information to the terminal device of the target object, comprising: in response to the presence of the moving object in the unknown classification region, sending the warning information to the terminal device.

[0007] In the case where the target detection result includes the unknown classification region, before the warning information is sent to the terminal device of the target object, the method further comprises: determining whether there is a moving object in the unknown classification region; and sending the warning information to the terminal device of the target object, comprising: in response to the presence of the moving object in the unknown classification region, sending the warning information to the terminal device.

[0008] The method further comprises: in response to determining that the trapped living body exists in the vehicle, determining whether the trapped living body is in an active state; and in response to the trapped living body being in the active state, interacting with the trapped living body based on attribute parameters of the trapped living body, wherein the attribute parameters comprise at least one of a species and an age.

[0009] The attribute parameters comprise at least one of a species and an age.

[0010] The attribute parameters comprise at least one of a species and an age.

[0011] The attribute parameters comprise at least one of a species and an age.

[0012] The method further comprises: in response to determining that the trapped living body exists in the vehicle, performing at least one of the following operations: in response to the temperature in the vehicle being greater than a preset temperature threshold, controlling the air conditioner to start; in response to the light intensity in the vehicle being less than a preset light intensity threshold, controlling the in-vehicle lighting device to turn on; controlling the rear window to be lowered to a preset opening degree threshold; sending positioning information to a terminal device; and controlling the in-vehicle fragrance device to start.

[0013] The determining that the trapped living body exists in the vehicle comprises: determining that the trapped living body exists based on the target detection result; or receiving feedback information sent by the terminal device, the feedback information indicating that the trapped living body exists in the vehicle.

[0014] The method further comprises: receiving a remote control instruction sent by the terminal device, the remote control instruction being used to instruct to perform at least one of the following operations: controlling the air conditioner to start, turning on the in-vehicle lighting device, and controlling the rear window to be lowered to a preset opening degree threshold.

[0015] To solve the above technical problems, another technical solution adopted by the present application is to provide a monitoring device for a life body remaining in a vehicle, which comprises: an acquisition module configured to acquire an image in the vehicle in a state that the vehicle is powered off and the vehicle door is locked; a target detection module configured to perform target detection based on the image in the vehicle to obtain a target detection result; and a sending module configured to send warning information to a terminal device of a target object in response to the target detection result including an unknown classification area, wherein the warning information is used to prompt the target object to determine whether there is a remaining life body in the vehicle, and the warning information at least includes image data of the unknown classification area in the image in the vehicle.

[0016] The monitoring device for a life body remaining in a vehicle further comprises a first judgment module. In the case that the target detection result includes the unknown classification area, the first judgment module is configured to determine whether there is a moving object in the unknown classification area before the sending module sends the warning information to the terminal device of the target object. The sending module is configured to send the warning information to the terminal device in response to the existence of the moving object in the unknown classification area.

[0017] The first judgment module is configured to determine state parameters of the unknown classification area in adjacent two frames of the image in the vehicle, the state parameters including at least one of position information and size information of the unknown classification area, and determine whether there is a moving object in the unknown classification area based on the state parameters of the unknown classification area in the adjacent two frames of the image in the vehicle.

[0018] The monitoring device for a life body remaining in a vehicle further comprises a second judgment module and an interaction module. The second judgment module is configured to determine whether the remaining life body is in an active state in response to the determination that there is a remaining life body in the vehicle. The interaction module is configured to interact with the remaining life body based on attribute parameters of the remaining life body in response to the remaining life body being in the active state, wherein the attribute parameters include at least one of a species and an age.

[0019] The interaction module is configured to play first multimedia data in response to the species of the remaining life body being non-human, or the species of the remaining life body being human and the age being less than or equal to a preset age threshold, wherein the first multimedia data is used to calm the emotion of the remaining life body.

[0020] The interaction module is configured to play a preset question in response to the species of the remaining life body being human and the age being greater than the preset age threshold, wherein the preset question is used to verify the intelligence level of the remaining life body, and interact with the remaining life body based on the answer content of the remaining life body to the preset question.

[0021] The interaction module is configured to play second multimedia data in response to the answer content being consistent with a preset answer of a preset question, wherein the second multimedia data is configured to guide the trapped living body to unlock the vehicle door; and play first multimedia data in response to the answer content not being received or the answer content being inconsistent with the preset answer of the preset question.

[0022] The monitoring device for the trapped living body in the vehicle further includes a control module. The control module is configured to perform at least one of the following operations in response to determining that the trapped living body exists in the vehicle: controlling the air conditioner to start in response to the temperature in the vehicle being greater than a preset temperature threshold; controlling a light device in the vehicle to turn on in response to the light intensity in the vehicle being less than a preset light intensity threshold; controlling a rear window to be lowered to a preset opening threshold; sending positioning information to a terminal device; and controlling a fragrance device in the vehicle to start.

[0023] The monitoring device for the trapped living body in the vehicle further includes a determination module configured to determine that the trapped living body exists based on the target detection result. Alternatively, the monitoring device for the trapped living body in the vehicle further includes a receiving module configured to receive feedback information sent by the terminal device, the feedback information indicating that the trapped living body exists in the vehicle.

[0024] The receiving module is further configured to receive a remote control instruction sent by the terminal device, the remote control instruction being configured to instruct the control module to perform at least one of the following operations: controlling the air conditioner to start, turning on the light device in the vehicle, and controlling the rear window to be lowered to the preset opening threshold.

[0025] To solve the above technical problems, another technical solution adopted by the present application is to provide an electronic device, including a storage and a processor coupled with each other, the storage storing program instructions; the processor is configured to execute the program instructions stored in the storage to implement the above-mentioned monitoring method for the trapped living body in the vehicle.

[0026] To solve the above technical problems, another technical solution adopted by the present application is to provide a computer readable storage medium for storing program instructions, the program instructions being executable by a processor to implement the above-mentioned monitoring method for the trapped living body in the vehicle.

[0027] The above scheme obtains the in-vehicle image in the power-off and door-locked state of the vehicle, and performs target detection based on the in-vehicle image to obtain a target detection result. When the target detection result includes an unknown classification area, i.e., the target detection cannot determine that there is a trapped living body in the vehicle, warning information is sent to the terminal device of the target object, and the warning information at least includes image data in the unknown classification area in the in-vehicle image, so that the target object determines whether there is a trapped living body in the vehicle. This way can further determine whether there is a trapped living body in the vehicle through the target object when the target detection cannot determine that there is a trapped living body in the vehicle, thereby improving the accuracy of monitoring the trapped living body in the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a flowchart of an embodiment of the in-vehicle trapped living body monitoring method provided by the present application;

[0029] Figure 2 is a flowchart of another embodiment of the in-vehicle trapped living body monitoring method provided by the present application;

[0030] Figure 3 is a schematic diagram of the principle of target trajectory tracking provided by the present application;

[0031] Figure 4 is a schematic diagram of an embodiment of the in-vehicle trapped living body monitoring system provided by the present application;

[0032] Figure 5 is a schematic diagram of an embodiment of the in-vehicle trapped living body monitoring device provided by the present application;

[0033] Figure 6 is a schematic diagram of an embodiment of the electronic device provided by the present application;

[0034] Figure 7 is a schematic diagram of an embodiment of the computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0035] To make the purpose, technical solutions and effects of the present application clearer and more explicit, the following embodiments are described in detail with reference to the accompanying drawings.

[0036] In the following description, specific details are presented to facilitate a thorough understanding of the present application, but are not intended to limit the present application.

[0037] The term “and / or” herein is merely used to associate associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, “multiple” herein means two or more. In addition, the term “at least one” herein means any one of multiple or any combination of at least two of multiple, for example, at least one of A, B and C includes any one or more elements selected from the set of A, B and C. “Several” means at least one. The terms “first”, “second”, etc. in the description and claims herein and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0038] Please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the method for monitoring the life bodies remaining in the vehicle provided by the present application. The method can be executed by a vehicle controller, which can be a vehicle controller or other controller, etc. The present embodiment does not make specific limitations on this. It should be noted that the method of the present application is not limited to the order of the flowchart shown in Figure 1 . As shown in Figure 1 , the method comprises the following steps:

[0039] S11: Obtain an in-vehicle image when the vehicle is powered off and the doors are locked.

[0040] When it is detected that the vehicle is powered off and the doors are locked, it is determined that the driver has left the vehicle.

[0041] In an embodiment, an image detection device is installed in the vehicle. When it is detected that the vehicle is powered off and the doors are locked, the image detection device is controlled to be turned on, and an in-vehicle image collected by the image detection device is obtained.

[0042] S12: Perform target detection based on the in-vehicle image to obtain a target detection result.

[0043] Specifically, the in-vehicle image is input into a target detection model, and the target detection model is used to perform target detection on the in-vehicle image to obtain a target detection result. The target detection result can include a plurality of target detection frame regions, and classification information, position information, etc. of each target detection frame region.

[0044] In an example, the target detection model can be trained by a plurality of sample images, so that the target detection model can identify and classify in-vehicle static resident objects. The in-vehicle static resident objects can include seats, pillows, etc. In this example, when other targets appear in the vehicle in addition to the aforementioned in-vehicle static resident objects, the target detection model cannot classify them.

[0045] In another example, the target detection model can be trained by a plurality of sample images, so that the target detection model can recognize and classify common living beings and the aforementioned in-vehicle static resident objects. Illustratively, the common living beings can be pets, children, adults, etc. In this example, when a living being exists in the vehicle and is partially blocked by an in-vehicle static resident object, for example, a living being located in the rear row of the vehicle is partially blocked by the front row seat, the target detection model can not be able to recognize and classify the living being.

[0046] In this embodiment, the target detection model can be R-CNN (Region based Convolutional Neural Network), Faster R-CNN, YOLO (You Only Look Once), etc., which is not specifically limited.

[0047] S13: In response to the target detection result including the unknown classification region, sending warning information to the terminal device of the target object.

[0048] The warning information is used to prompt the target object to determine whether there is a stranded living being in the vehicle, and the warning information at least includes image data of the unknown classification region in the in-vehicle image. It should be noted that the unknown classification region in this embodiment refers to a region in which the target detection cannot classify the target in the detection frame.

[0049] In an embodiment, in the case where the target detection result includes the unknown classification region, the warning information is directly sent to the terminal device of the target object, so that the target object determines whether there is a stranded living being in the vehicle based on the warning information.

[0050] In another embodiment, in the case where the target detection result includes the unknown classification region, it is first determined whether there is a moving object in the unknown classification region. When there is a moving object in the unknown classification region, it is determined that there is an abnormality in the vehicle. At this time, the warning information is sent to the terminal device of the target object.

[0051] Illustratively, the target object can be at least one of a car owner, a person having a blood relationship with the car owner, a person having a friend relationship with the car owner, and a relevant department responsible for life safety or emergency affairs. The terminal device of the target object can be a mobile phone, a computer, a tablet, a wearable smart device, etc., which is not specifically limited in this embodiment.

[0052] In this embodiment, the in-vehicle image of the vehicle in the power-off and door-locked state is obtained, and target detection is performed based on the in-vehicle image to obtain a target detection result. When the target detection result includes an unknown classification region, i.e., the presence of a trapped living body in the vehicle cannot be determined through target detection, warning information is sent to the terminal device of the target object, and the warning information at least includes image data of the unknown classification region in the in-vehicle image, so that the target object can determine whether there is a trapped living body in the vehicle. In this way, when the presence of a trapped living body in the vehicle cannot be determined through target detection, the target object can further determine whether there is a trapped living body in the vehicle, thereby improving the accuracy of monitoring the presence of a trapped living body in the vehicle.

[0053] Please refer to Figure 2 , Figure 2 is a flowchart of another embodiment of the in-vehicle trapped living body monitoring method provided by the present application, which can be executed by the vehicle controller described above. As shown in Figure 2 , the method comprises the following steps:

[0054] S21: Obtain an in-vehicle image of the vehicle in the power-off and door-locked state.

[0055] S22: Perform target detection based on the in-vehicle image to obtain a target detection result.

[0056] The related content of steps S21 and S22 can be referred to steps S11 and S12 respectively, which will not be described here.

[0057] S23: In response to the target detection result including an unknown classification region, determine whether there is a moving object in the unknown classification region.

[0058] In the case where the target detection result includes an unknown classification region, if there is a moving object in the unknown classification region, it is determined that there is an abnormality in the vehicle; and in the case where the target detection result includes an unknown classification region, if there is no moving object in the unknown classification region, it is determined that there is no abnormality in the vehicle.

[0059] In this embodiment, determining whether there is a moving object in the unknown classification region comprises: determining state parameters of the unknown classification region in adjacent two frames of in-vehicle images; and determining whether there is a moving object in the unknown classification region based on the state parameters of the unknown classification region in the adjacent two frames of in-vehicle images. The state parameters of the unknown classification region include at least one of position information and size information of the unknown classification region. For example, the position information of the unknown classification region includes horizontal pixel coordinates, vertical pixel coordinates, horizontal pixel coordinate change rate and vertical pixel coordinate change rate of the center of the target in the unknown classification region. The size information of the unknown classification region includes area, area change rate and aspect ratio of the unknown classification region.

[0060] In an embodiment, the target tracking algorithm is used to track the trajectory of the target in the unknown classification region to determine the state parameters of the unknown classification region in two adjacent frames of in-vehicle images.

[0061] Specifically, refer to Figure 3 , Figure 3 is a schematic diagram of the principle of the target trajectory tracking provided in the present application. As Figure 3 shown, the unknown classification region obtained by target detection is input into the trajectory tracking framework, a linear constant speed model is used to estimate the inter-frame displacement of the target in each unknown classification region, and the state of each target is modeled as: wherein u, v, s, r, and respectively represent the horizontal pixel coordinate, the vertical pixel coordinate, the area of the unknown classification region, the aspect ratio of the unknown classification region, the horizontal pixel coordinate change rate, the vertical pixel coordinate change rate, and the area change rate of the unknown classification region. On the one hand, the Kalman filter is used to predict the predicted unknown classification region (filter prediction box) in the next frame of in-vehicle images based on the linear constant speed model and the state parameters of the unknown classification region in the current frame of in-vehicle images. On the other hand, the measured unknown classification region (target detection box) in the next frame of in-vehicle images is detected by using the target detection algorithm, the IOU (Intersection over Union) distance between the predicted unknown classification region and the measured unknown classification region is calculated, a cost matrix is obtained, and the Hungarian matching is performed on the target in the current frame of in-vehicle images and the target in the next frame of in-vehicle images based on the cost matrix. If the matching is successful, the Kalman filtering algorithm is used to update the predicted unknown classification region and the measured unknown classification region to obtain the optimal estimation of the unknown classification region in the next frame of in-vehicle images, thereby obtaining the state parameters of the unknown classification region in the next frame of in-vehicle images. If the matching is not successful, the target may enter or leave the image, and the target trajectory and the trajectory identifier corresponding to the trajectory need to be created accordingly, or the target trajectory and the trajectory identifier need to be deleted.

[0062] In an embodiment, when the state parameters of the unknown classification region in the two adjacent in-vehicle images satisfy a preset condition, it is determined that there is a living object in the unknown classification region; when the state parameters of the unknown classification region in the two adjacent in-vehicle images do not satisfy the preset condition, it is determined that there is no living object in the unknown classification region. The preset condition includes that the absolute value of the difference between the state parameters of the two adjacent in-vehicle images is greater than a preset difference threshold. For example, the absolute value of the difference between the horizontal pixel coordinates of the target center of the current in-vehicle image and the horizontal pixel coordinates of the target center of the next in-vehicle image is greater than the preset difference threshold. It should be noted that when the state parameters include multiple state parameters, the preset difference threshold corresponding to each state parameter is set according to actual needs, and the preset difference thresholds corresponding to each state parameter can be different.

[0063] S24: In response to the existence of the living object in the unknown classification region, sending warning information to the terminal device of the target object.

[0064] The warning information is used to prompt the target object to determine whether there is a trapped living body in the vehicle, and the warning information at least includes image data of the unknown classification region in the in-vehicle image.

[0065] Further, after sending the warning information to the terminal device of the target object, it further includes receiving feedback information sent by the terminal device, and the feedback information indicates that there is a trapped living body in the vehicle. That is, when the terminal device of the target object receives the warning information, and the target object determines that there is a trapped living body in the vehicle based on the warning information, the terminal device can send feedback information to the vehicle controller, and the vehicle controller receives the feedback information and determines that there is a trapped living body in the vehicle.

[0066] In the embodiment, on the one hand, the target detection result can be obtained by performing target detection on the in-vehicle image, and when the trapped living body does not have shielding or the trapped living body has shielding but does not affect the identification and classification of the trapped living body, the existence of the trapped living body in the vehicle can be determined based on the target detection result. On the other hand, when the target detection result includes an unknown classification region and there is an abnormal living object in the unknown classification region, since it is impossible to determine whether there is a trapped living body in the vehicle through target detection, the target object can further determine whether there is an abnormal object in the vehicle by sending warning information to the terminal device of the target object, thereby improving the accuracy of monitoring the existence of the trapped living body in the vehicle.

[0067] Optionally, in the embodiment, when it is determined that there is a trapped living body in the vehicle, the monitoring method of the trapped living body further includes: judging whether the trapped living body is in an active state; and when the trapped living body is in the active state, interacting with the trapped living body based on attribute parameters of the trapped living body. The attribute parameters include at least one of species and age.

[0068] In an embodiment, determining whether the trapped living body is in an active state comprises: determining a state parameter of the trapped living body region in the two adjacent vehicle images; and determining whether the trapped living body is in an active state based on the state parameter of the trapped living body region in the two adjacent vehicle images. The state parameter of the trapped living body region comprises at least one of position information and size information of the trapped living body region. Details can be referred to the foregoing step S23, and thus are not described herein.

[0069] In an embodiment, interacting with the trapped living body based on the attribute parameter of the trapped living body comprises: playing first multimedia data in response to the trapped living body being a non-human, or the trapped living body being a human and having an age less than or equal to a preset age threshold. The first multimedia data is used to calm the trapped living body.

[0070] When the trapped living body is a human and has an age less than or equal to a preset age threshold, it is considered that the trapped living body is young and has no self-help ability. When the trapped living body is a human and has an age greater than the preset age threshold, it is considered that the trapped living body is old and may have self-help ability. The preset age threshold is set according to actual conditions. The age of the trapped living body can be detected by the foregoing target detection algorithm, or when the trapped living body is determined remotely by the target object, the target object can also feed back the age information of the trapped living body to the vehicle controller. The first multimedia data can be pre-set music, animation or voice, which is not limited in the embodiment.

[0071] In another embodiment, interacting with the trapped living body based on the attribute parameter of the trapped living body comprises: playing a preset question in response to the trapped living body being a human and having an age greater than a preset age threshold; and interacting with the trapped living body based on the answer content of the trapped living body to the preset question. The preset question is used to verify the intelligence level of the trapped living body. The preset question is set according to actual needs.

[0072] Specifically, interacting with the trapped living body based on the answer content of the trapped living body to the preset question comprises: when the answer content of the trapped living body is consistent with a preset answer to the preset question, it is determined that the trapped living body has a high intelligence level and can perform self-help through voice interaction guidance, at this time, second multimedia data can be played, and the second multimedia data is used to guide the trapped living body to unlock the vehicle door; and when the answer content of the trapped living body is not received or the answer content of the trapped living body is inconsistent with the preset answer to the preset question, it is determined that the trapped living body has a low intelligence level and cannot perform self-help through voice interaction guidance, at this time, first multimedia data can be played to calm the trapped living body.

[0073] Optionally, before playing the second multimedia data, the embodiment further includes: playing third multimedia data, the third multimedia data being used to guide the stranded living body to approach the image detection device to take an image. After the image of the stranded living body is acquired, the image of the stranded living body is stored as a record in the storage unit, so as to facilitate subsequent target object to call and view.

[0074] In the embodiment, when it is determined that the stranded living body exists in the vehicle and the stranded living body is in a moving state, it is considered that the stranded living body may be in a state of anxiety, fear, or panic. At this time, the first multimedia data is played to calm the stranded living body, so that the stranded living body is quiet, the risk of collision caused by the stranded living body due to random movement is reduced, and the safety of the stranded living body in the vehicle is further protected. The second multimedia data is played to guide the stranded living body to unlock the vehicle door to save itself, and the safety of the stranded living body is further ensured.

[0075] Please refer to Figure 4 , Figure 4 is a framework schematic diagram of an embodiment of the vehicle stranded living body monitoring system provided in the application. As shown in Figure 4 , the vehicle stranded living body monitoring system includes an image detection device 501, a temperature sensor 502, an illumination intensity sensor 503, a positioning device 504, an air conditioner 505, a vehicle window 506, a door lock 507, a vehicle light device 508, a fragrance device 509, an alarm module 510, a language interaction module 511, a vehicle controller 512, and a power supply module 513. The image detection device 501 is configured to detect an image in the vehicle. The temperature sensor 502 is configured to detect a temperature in the vehicle. The illumination intensity sensor 503 is configured to detect an illumination intensity in the vehicle. The positioning device 504 is configured to acquire positioning information of a vehicle in which the stranded living body is located. The alarm module 510 is configured to send alert information and the positioning information of the vehicle to a terminal device of a target object when it is determined that the stranded living body exists in the vehicle. The language interaction module 511 is configured to perform voice interaction with the stranded living body. The power supply module 513 is configured to supply power to each device or module in the monitoring system. For example, the power supply module 513 can be a solar photovoltaic panel or a storage battery.

[0076] Optionally, when it is determined that the stranded living body exists in the vehicle, the vehicle stranded living body monitoring method further includes at least one of the following operations: in response to the temperature in the vehicle being greater than a preset temperature threshold, controlling the air conditioner to start; in response to the illumination intensity in the vehicle being less than a preset illumination intensity threshold, controlling the vehicle light device to turn on; controlling the rear vehicle window to be lowered to a preset opening threshold; sending the positioning information to the terminal device of the target object; and controlling the vehicle fragrance device to start.

[0077] When it is determined that the trapped living body exists in the vehicle, if the temperature in the vehicle is greater than a preset temperature threshold, the life safety of the trapped living body can be threatened, at this time, the air conditioner can be controlled to start to reduce the temperature in the vehicle. The preset temperature threshold can be set according to actual needs.

[0078] When it is determined that the trapped living body exists in the vehicle, considering that when the vehicle is parked in a basement or other places with low light intensity, the vehicle can be in a dark state, at this time, the trapped living body can be nervous and scared. If the light intensity in the vehicle is less than a preset light intensity threshold, the vehicle light is controlled to be turned on, on the one hand, the trapped living body can be relieved from nervousness and fear, on the other hand, the vehicle light can attract the attention of surrounding people after being turned on, when the trapped living body exists in life safety, the surrounding people can observe the situation of the trapped living body and rescue in time. The preset light intensity threshold can be set according to actual needs.

[0079] When it is determined that the trapped living body exists in the vehicle, the rear window is controlled to be lowered to a preset opening threshold, on the one hand, the vehicle can be ventilated to improve the air quality in the vehicle, on the other hand, the child can be prevented from being lost. The preset opening threshold is set according to actual conditions.

[0080] When it is determined that the trapped living body exists in the vehicle, the positioning information is sent to the terminal device of the target object, which can facilitate the target object to quickly find and rescue the trapped living body.

[0081] When it is determined that the trapped living body exists in the vehicle, the vehicle fragrance device is controlled to start, which can remove the peculiar smell in the vehicle, kill bacteria and improve the air environment in the vehicle.

[0082] Optionally, in the embodiment, when the target object determines that the trapped living body exists in the vehicle based on the foregoing warning information, remote rescue measures can be taken to ensure the life safety of the trapped living body in the vehicle. At this time, the monitoring method of the trapped living body further includes: receiving a remote control instruction sent by the terminal device, the remote control instruction being used to instruct to perform at least one of the following operations: controlling the air conditioner to start, turning on the vehicle light device, and controlling the rear window to be lowered to a preset opening threshold.

[0083] Please refer to Figure 5 , Figure 5Fig. 1 is a schematic diagram of an embodiment of the in-vehicle trapped living body monitoring device provided in the present application. In the embodiment, the in-vehicle trapped living body monitoring device 50 comprises an acquisition module 51, a target detection module 52 and a sending module 53. The acquisition module 51 is configured to acquire an in-vehicle image of a vehicle in a power-off and door-locked state. The target detection module 52 is configured to perform target detection based on the in-vehicle image to obtain a target detection result. The sending module 53 is configured to send warning information to a terminal device of a target object in response to the target detection result including an unknown classification region. The warning information is used to prompt the target object to determine whether there is a trapped living body in the vehicle, and the warning information at least includes image data of an image located in the unknown classification region in the in-vehicle image.

[0084] Optionally, the in-vehicle trapped living body monitoring device 50 further comprises a first judgment module 54. In the case that the target detection result includes the unknown classification region, before the sending module 53 sends the warning information to the terminal device of the target object, the first judgment module 54 is configured to determine whether there is a moving object in the unknown classification region. The sending module 53 is configured to send the warning information to the terminal device in response to the existence of the moving object in the unknown classification region.

[0085] Optionally, the first judgment module 54 is configured to determine a state parameter of the unknown classification region in adjacent two frames of in-vehicle images, the state parameter including at least one of position information and size information of the unknown classification region; and determine whether there is a moving object in the unknown classification region based on the state parameter of the unknown classification region in the adjacent two frames of in-vehicle images.

[0086] Optionally, the in-vehicle trapped living body monitoring device 50 further comprises a second judgment module 55 and an interaction module 56. The second judgment module 55 is configured to determine whether a trapped living body is in an active state in response to the determination that there is a trapped living body in the vehicle. The interaction module 56 is configured to interact with the trapped living body based on an attribute parameter of the trapped living body in response to the trapped living body being in the active state, wherein the attribute parameter includes at least one of a species and an age.

[0087] Optionally, the interaction module 56 is configured to play first multimedia data in response to the species of the trapped living body being non-human, or the species of the trapped living body being human and the age being less than or equal to a preset age threshold, wherein the first multimedia data is used to calm the emotion of the trapped living body.

[0088] Optionally, the interaction module 56 is configured to play a preset question in response to the species of the trapped living body being human and the age being greater than the preset age threshold, wherein the preset question is used to verify the intelligence level of the trapped living body; and interact with the trapped living body based on the answer content of the trapped living body to the preset question.

[0089] Optionally, the interaction module 56 is configured to, in response to the answer content being consistent with the preset answer of the preset question, play second multimedia data, wherein the second multimedia data is used to guide the trapped living body to unlock the vehicle door; and in response to the answer content not being received or the answer content being inconsistent with the preset answer of the preset question, play first multimedia data.

[0090] Optionally, the in-vehicle trapped living body monitoring apparatus 50 further comprises a control module 57. The control module 57 is configured to, in response to determining that there is a trapped living body in the vehicle, perform at least one of the following operations: in response to the temperature in the vehicle being greater than a preset temperature threshold, controlling the air conditioner to start; in response to the light intensity in the vehicle being less than a preset light intensity threshold, controlling the in-vehicle lighting device to turn on; controlling the rear window to be lowered to a preset opening degree threshold; sending positioning information to a terminal device; and controlling the in-vehicle fragrance device to start.

[0091] Optionally, the in-vehicle trapped living body monitoring apparatus 50 further comprises a determination module 58, which is configured to determine that there is a trapped living body based on the target detection result. Alternatively, the in-vehicle trapped living body monitoring apparatus 50 further comprises a receiving module 59, which is configured to receive feedback information sent by a terminal device, the feedback information indicating that there is a trapped living body in the vehicle.

[0092] Optionally, the receiving module 59 is further configured to receive a remote control instruction sent by a terminal device, the remote control instruction being used to instruct the control module 57 to perform at least one of the following operations: controlling the air conditioner to start, turning on the in-vehicle lighting device, and controlling the rear window to be lowered to a preset opening degree threshold.

[0093] It should be noted that the apparatus of the present embodiment can perform the steps in the above method, and the details of the related content are described in the above method section, which will not be repeated here.

[0094] Please refer to Figure 6 , Figure 6 is a frame schematic diagram of an embodiment of an electronic device provided by the present application. In the present embodiment, the electronic device 60 comprises a memory 61 and a processor 62.

[0095] The processor 62 can also be referred to as a CPU (Central Processing Unit). The processor 62 can be an integrated circuit chip with processing capability. The processor 62 can also be a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general purpose processor can be a microprocessor or the processor 62 can also be any conventional processor 62 and the like.

[0096] The memory 61 in the electronic device 60 is configured to store program instructions required by the processor 62 to run.

[0097] The processor 62 is configured to execute the program instructions to implement the method for monitoring the in-vehicle trapped living body in the present application.

[0098] Referring to Figure 7 , Figure 7 is a schematic diagram of the framework of an embodiment of the computer-readable storage medium provided in the present application. The computer-readable storage medium 70 of the embodiment of the present application stores program instructions 71, which, when executed, implement the method for monitoring the in-vehicle trapped living body provided in the present application. The program instructions 71 can form a program file and be stored in the computer-readable storage medium 70 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of the method of each embodiment of the present application. The aforementioned computer-readable storage medium 70 includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, or a tablet.

[0099] The above scheme acquires the in-vehicle image of the vehicle in the power-off and door-locked state, and performs target detection based on the in-vehicle image to obtain a target detection result. When the target detection result includes an unknown classification area, i.e., the in-vehicle trapped living body cannot be determined through target detection, the warning information is sent to the terminal device of the target object, and the warning information at least includes the image data of the unknown classification area in the in-vehicle image, so as to determine whether the in-vehicle trapped living body exists. The target object can further determine whether the in-vehicle trapped living body exists when the in-vehicle trapped living body cannot be determined through target detection, thereby improving the accuracy of monitoring the in-vehicle trapped living body.

[0100] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiment descriptions, and the specific implementation can refer to the descriptions of the above method embodiments. For brevity, they will not be described here.

[0101] The above description of various embodiments tends to emphasize the differences between various embodiments, and the same or similar parts can be mutually referred to. For brevity, they will not be described here.

[0102] In several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other manners. For example, the division of the apparatus embodiments is merely an example, and the division of the modules or units can be different, for example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0103] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0104] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0105] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0106] The above description is merely an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the contents of the present application specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for monitoring living beings remaining inside a vehicle, characterized in that, The method includes: Acquire an image of the vehicle's interior when the power is off and the doors are locked; The trained target detection model is used to perform target detection on the in-vehicle images to obtain target detection results; wherein, the target detection model is used to identify and classify static permanent objects in the vehicle, or, the target detection model is used to identify and classify common living organisms and static permanent objects in the vehicle. In response to the target detection result including an unknown classification region, a warning message is sent to the terminal device of the target object; wherein, the unknown classification region refers to the region where the target detection cannot classify the target within the detection frame; the warning message is used to prompt the target object to determine whether there is a lingering living being inside the vehicle, and the warning message includes at least image data of the interior image located in the unknown classification region; The system receives feedback information sent by the terminal device, which indicates that a living organism remains in the vehicle.

2. The method according to claim 1, characterized in that, If the target detection result includes the unknown classification region, the method further includes the following steps before sending a warning message to the terminal device of the target object: Determine whether there are any moving objects within the unknown classification area; Sending warning information to the target object's terminal device includes: In response to the presence of the active object within the unknown classification area, the warning information is sent to the terminal device.

3. The method according to claim 2, characterized in that, Determining whether there are moving objects within the unknown classification area includes: Determine the state parameters of the unknown classification region in two adjacent frames of the in-vehicle image, wherein the state parameters include at least one of the location information and size information of the unknown classification region; Based on the state parameters of the unknown classification region in two adjacent frames of the in-vehicle image, it is determined whether there is a moving object in the unknown classification region.

4. The method according to claim 1, characterized in that, The method further includes: In response to determining that a resident living being exists in the vehicle, it is determined whether the resident living being is in an active state; In response to the presence of the stranded life form being active, the system interacts with the stranded life form based on its attribute parameters; wherein the attribute parameters include at least one of species and age.

5. The method according to claim 4, characterized in that, The interaction with the stranded life form based on its attribute parameters includes: In response to the fact that the species of the stranded life form is non-human, or that the species of the stranded life form is human and its age is less than or equal to a preset age threshold, first multimedia data is played, wherein the first multimedia data is used to soothe the emotions of the stranded life form.

6. The method according to claim 4, characterized in that, The interaction with the stranded life form based on its attribute parameters includes: In response to the fact that the species of the stranded life form is human and its age is greater than a preset age threshold, a preset question is played, wherein the preset question is used to verify the intelligence level of the stranded life form; Based on the answers given by the stranded life form to the preset questions, interact with the stranded life form.

7. The method according to claim 6, characterized in that, The interaction with the stranded life form based on its answers to the preset questions includes: In response to the answer content being consistent with the preset answer to the preset question, second multimedia data is played, wherein the second multimedia data is used to guide the stranded life form to unlock the car door; In response to the failure to receive the response, or the response not being consistent with the preset answer to the preset question, the first multimedia data is played.

8. The method according to claim 1, characterized in that, The method further includes: In response to determining that a living being remains in the vehicle, perform at least one of the following operations: The air conditioning is activated in response to the interior temperature exceeding a preset temperature threshold. In response to the interior light intensity being lower than a preset light intensity threshold, the interior lighting equipment is turned on. Control the rear windows to descend to a preset opening threshold; Send location information to the terminal device; Control the activation of the in-car fragrance system.

9. The method according to claim 1, characterized in that, Determining that a living being remains inside the vehicle also includes: Based on the target detection results, the presence of the lingering life form is confirmed.

10. The method according to claim 1, characterized in that, The method further includes: Receive a remote control command sent by the terminal device, the remote control command being used to instruct at least one of the following operations: Control the air conditioning to start, turn on the interior lights, and control the rear windows to lower to a preset opening threshold.

11. A monitoring device for a living being remaining inside a vehicle, characterized in that, The device includes: The acquisition module is used to acquire images of the vehicle interior when the power is off and the doors are locked. The object detection module is used to perform object detection on the in-vehicle image using a trained object detection model to obtain object detection results; wherein, the object detection model is used to identify and classify static permanent objects in the vehicle, or, the object detection model is used to identify and classify common living organisms and static permanent objects in the vehicle. The sending module is configured to send a warning message to the terminal device of the target object in response to the target detection result including an unknown classification region; wherein, the unknown classification region refers to the region where the target detection cannot classify the target within the detection frame; the warning message is used to prompt the target object to determine whether there is a lingering living being inside the vehicle, and the warning message includes at least image data of the interior image located in the unknown classification region; A receiving module is used to receive feedback information sent by the terminal device, the feedback information indicating that there is a lingering living organism in the vehicle.

12. An electronic device, characterized in that, Including interconnected memory and processor, The memory stores program instructions; The processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions that can be executed by a processor to implement the method of any one of claims 1-10.

Images