Vehicle bottom foreign matter monitoring method and device, electronic equipment and medium

Abstract

The invention relates to the technical field of automobile safety, and provides a vehicle bottom foreign matter monitoring method and device, electronic equipment and a medium. The method comprises the steps that under the condition that a target vehicle is in a parking state, a first closed curve and a second closed curve surrounding the current target vehicle are determined, and a corresponding external area, a first warning area and a second warning area are determined; continuously acquiring an environment information flow corresponding to the ground area through an environment sensor of the target vehicle; under the condition that the moving object entering the first warning area from the external area appears in the environment information flow, determining a movement track of the moving object based on the environment information flow; and determining a vehicle bottom foreign matter report based on the motion trail. As long as the environment information flow comprises the information corresponding to the first warning area, the motion trail of the moving object can be analyzed, so that the environment sensor is installed on the vehicle body and detects the surrounding environment, and vehicle bottom foreign matter monitoring can be reliably, stably and efficiently achieved.

Description

Technical Field

[0001] The present application relates to the field of automobile safety technology, and in particular to a method, device, electronic equipment and medium for monitoring foreign objects under a vehicle. Background Art

[0002] When a vehicle is parked, stray cats, dogs, and children may stay under the vehicle. If the user does not carefully check the bottom of the vehicle before starting the vehicle, accidents may occur. If you want to provide users with the service of checking for foreign objects under the vehicle, you need to install a camera or sensor under the vehicle to detect whether there are foreign objects under the vehicle through the data of the camera and sensor.

[0003] This solution requires additional cameras or sensors, and because the installation position is under the vehicle, the vehicle is easily disturbed by road conditions during driving, such as being damaged by stones bounced off the road, or being affected by mud splashed onto the surface, resulting in performance degradation, or posture changes due to bumps, etc. The overall stability is poor. Since it is installed under the vehicle, it is inconvenient for users to check by themselves, and the subsequent maintenance cost is high.

[0004] Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method, device, electronic device and medium for monitoring foreign objects under a vehicle to solve the problem that there is no efficient and stable solution for monitoring foreign objects under a vehicle in the prior art.

[0006] A first aspect of an embodiment of the present application provides a method for monitoring foreign matter under a vehicle, comprising:

[0007] When the target vehicle is in a parked state, a first closed curve and a second closed curve surrounding the current target vehicle are determined, and based on the first closed curve and the second closed curve, corresponding external areas, first warning areas and second warning areas are determined; in any direction of the target vehicle, a first distance between the first closed curve and the target vehicle is greater than a second distance between the second closed curve and the target vehicle, the external area is a ground area outside the first closed curve, the first warning area is a ground area between the first closed curve and the second closed curve, the second warning area is a ground area within the second closed curve, and the second warning area includes a ground area projected by the target vehicle;

[0008] Through the environmental sensors of the target vehicle, the corresponding environmental information flow of the ground area is continuously obtained;

[0009] analyzing the environmental information flow, and determining the movement trajectory of the active object based on the environmental information flow when an active object enters the first warning area from the external area appears in the environmental information flow;

[0010] Based on the positional relationship between the moving object in the motion trajectory and the second warning area, a report of a foreign object under the vehicle is determined.

[0011] A second aspect of the embodiment of the present application provides a vehicle bottom foreign body monitoring device, comprising:

[0012] The area determination module is used to determine the first closed curve and the second closed curve surrounding the current target vehicle when the target vehicle is in a parked state, and determine the corresponding external area, the first warning area and the second warning area based on the first closed curve and the second closed curve; in any direction of the target vehicle, the first distance between the first closed curve and the target vehicle is greater than the second distance between the second closed curve and the target vehicle, the external area is the ground area outside the first closed curve, the first warning area is the ground area between the first closed curve and the second closed curve, and the second warning area is the ground area within the second closed curve, including the ground area projected by the target vehicle;

[0013] An information flow acquisition module is used to continuously acquire the environmental information flow corresponding to the ground area through the environmental sensor of the target vehicle;

[0014] A first analysis module is used to analyze the environmental information flow, and when an active object enters the first warning area from the external area in the environmental information flow, determine the movement trajectory of the active object based on the environmental information flow;

[0015] The second analysis module is used to determine the under-vehicle foreign object report based on the positional relationship between the moving object and the second warning area in the motion trajectory.

[0016] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0017] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0018] Compared with the prior art, the beneficial effects of the embodiment of the present application include at least: the embodiment of the present application divides the target vehicle into areas based on the parking state: including the second warning area projected by the target vehicle, the first warning area outside the second warning area, and the external area outside the first warning area, and then continuously obtains the environmental information flow of the ground area through the environmental sensor set on the target vehicle, determines the motion trajectory by analyzing the environmental information flow, and then determines the foreign object report under the vehicle. The second warning area includes the area corresponding to the projection of the target vehicle. Once an active object enters the second warning area, it is considered to have entered the bottom of the target vehicle. Therefore, by monitoring whether the motion trajectory of the active object enters the second warning area, it is determined whether it has entered the bottom of the vehicle, and the generation of the foreign object report under the vehicle can provide users with a reference for the status of foreign objects under the vehicle. Since the movement of the active object has a direction, as long as the environmental information flow obtained by the environmental sensor includes information corresponding to the first warning area, the specific motion trajectory of the active object can be analyzed. Therefore, the environmental sensor can use a sensor installed on the vehicle body to detect the surrounding environment, and there is no need to deliberately set a sensor under the vehicle. It can reliably, stably and efficiently realize the monitoring of foreign objects under the vehicle, and is convenient for later maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 It is a schematic diagram of an application scenario of an embodiment of the present application;

[0021] Figure 2 It is a flow chart of a method for monitoring foreign matter under a vehicle provided in an embodiment of the present application;

[0022] Figure 3 It is a regional schematic diagram for dividing a ground area provided in an embodiment of the present application;

[0023] Figure 4 This is another regional schematic diagram for dividing a ground area provided in an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of a change in the motion trajectory of a moving object provided in an embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of a vehicle bottom foreign body monitoring device provided in an embodiment of the present application;

[0026] Figure 7It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0028] When a vehicle is parked, stray cats, dogs, and children may stay under the vehicle. If the user does not carefully check the bottom of the vehicle before starting the vehicle, accidents may occur. If you want to provide users with the service of checking for foreign objects under the vehicle, you need to install a camera or sensor under the vehicle to detect whether there are foreign objects under the vehicle through the data of the camera and sensor.

[0029] This solution requires additional cameras or sensors, and because the installation position is under the vehicle, the vehicle is easily disturbed by road conditions during driving, such as being damaged by stones bounced off the road, performance degradation caused by mud splashed onto the surface, and posture changes caused by bumps. The overall stability is poor. Because it is installed under the vehicle, it is inconvenient for users to check by themselves, and the subsequent maintenance cost is high.

[0030] In this embodiment, by monitoring the movement trajectory of active objects in the ground area corresponding to the target vehicle, it is analyzed whether there are active objects entering or leaving the bottom of the vehicle. Since movement has a direction, as long as the environmental information flow obtained by the environmental sensor includes the information of the first warning area, the analysis of the movement trajectory of the active object can be completed. When the active object is in the second warning area, it is considered that the active object is under the vehicle and is a foreign object under the vehicle. Based on the movement trajectory, the position relationship between the active object and the second warning area is obtained to determine the foreign object report under the vehicle. At this time, the environmental sensor is not required to be installed under the vehicle, and a sensor installed on the vehicle body to collect the surrounding environment of the vehicle can be used. This embodiment provides an effective and reliable monitoring solution, and the position setting of the sensor has high stability and low maintenance cost.

[0031] A method, device, electronic device and medium for monitoring foreign objects under a vehicle according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0032] Figure 1 1 is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a server 104 and a network 105.

[0033] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be hardware or software. When the first terminal device 101, the second terminal device 102, and the third terminal device 103 are hardware, they can be various electronic devices with display screens and support communication with the server 104, including but not limited to vehicle systems, smart phones, tablet computers, laptop portable computers, and desktop computers; when the first terminal device 101, the second terminal device 102, and the third terminal device 103 are software, they can be installed in the above electronic devices. The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be implemented as multiple software or software modules, or as a single software or software module, which is not limited in the embodiment of the present application. Further, various applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0034] The server 104 may be a server that provides various services, for example, a background server that receives a request sent by a terminal device that establishes a communication connection with the server, and the background server may receive and analyze the request sent by the terminal device, and generate a processing result. The server 104 may be a single server, or a server cluster composed of several servers, or a cloud computing service center, which is not limited in the present embodiment of the application.

[0035] It should be noted that the server 104 can be hardware or software. When the server 104 is hardware, it can be various electronic devices that provide various services for the first terminal device 101, the second terminal device 102, and the third terminal device 103. When the server 104 is software, it can be multiple software or software modules that provide various services for the first terminal device 101, the second terminal device 102, and the third terminal device 103, or it can be a single software or software module that provides various services for the first terminal device 101, the second terminal device 102, and the third terminal device 103, and the embodiment of the present application does not limit this.

[0036] The network 105 can be a wired network connected by coaxial cable, twisted pair and optical fiber, or it can be a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), infrared, etc., which is not limited in the embodiments of the present application.

[0037] It should be noted that the specific types, quantities and combinations of the first terminal device 101, the second terminal device 102, the third terminal device 103, the server 104 and the network 105 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present application do not limit this.

[0038] Figure 2 It is a flow chart of a method for monitoring foreign objects under a vehicle provided in an embodiment of the present application. Figure 2 The method for monitoring foreign objects under the vehicle can be Figure 1 The first terminal device, the second terminal device, the third terminal device or the server executes. Figure 2 As shown, the vehicle bottom foreign body monitoring method includes:

[0039] S201: When the target vehicle is in a parked state, determine a first closed curve and a second closed curve surrounding the current target vehicle, and determine corresponding external areas, first warning areas, and second warning areas based on the first closed curve and the second closed curve; in any direction of the target vehicle, a first distance between the first closed curve and the target vehicle is greater than a second distance between the second closed curve and the target vehicle, the external area is a ground area outside the first closed curve, the first warning area is a ground area between the first closed curve and the second closed curve, and the second warning area is a ground area within the second closed curve, including a ground area projected by the target vehicle;

[0040] S202: Continuously acquiring an environmental information flow corresponding to the ground area through an environmental sensor of the target vehicle;

[0041] S203: analyzing the environmental information flow, and in the case where an active object entering the first warning area from the external area appears in the environmental information flow, determining a movement trajectory of the active object based on the environmental information flow;

[0042] S204: Determine a report of a foreign object under the vehicle based on a positional relationship between the moving object and the second warning area in the motion trajectory.

[0043] When the target vehicle is in a parked state, steps S201-S204 of the method of this embodiment are executed to generate a report of foreign objects under the vehicle, which provides a reference for the safe start of the subsequent target vehicle. Therefore, in steps S201-S204, when the current target vehicle is in a parked state, the corresponding external area, the first warning area, and the second warning area are determined, and the environmental information flow is obtained through the environmental sensor. Based on the division of the external area, the first warning area, and the second warning area, the motion trajectory of the active object in the environmental information flow is analyzed to determine the positional relationship between the active object and the second warning area. Since the second warning area includes the ground area projected by the target vehicle, when the active object enters the second warning area, it is considered that the active area enters the bottom of the target vehicle, that is, the active object at this time is a foreign object under the vehicle. In fact, since there is a direction of movement, it is only necessary to analyze the position change of the active object and the first warning area to know whether the active object has entered the second warning area. The environmental information flow only needs to include the information of the first warning area. Therefore, the environmental sensor of the target vehicle is not required to be installed under the vehicle. The method of this embodiment can be implemented using conventional sensors installed on the vehicle body to collect the surrounding environment of the vehicle, thereby realizing foreign object monitoring under the vehicle in a stable, efficient and low-maintenance manner.

[0044] Specifically, the state information of the target vehicle is used to analyze whether the current target vehicle is in a parked state. If the current target vehicle is in a parked state, step S201 is executed to determine the first closed curve and the second closed curve, and then determine the external area, the first warning area, and the second warning area. It can be understood that the first closed curve and the second closed curve are both closed curves surrounding the target vehicle, and the second closed curve is located in the ground area contained by the first closed curve. Based on the first closed curve and the second closed curve, the ground area is divided into three parts: the second warning area within the second closed curve, the first warning area between the first closed curve and the second closed curve, and the external area outside the first closed curve, wherein the second closed curve is closer to the ground projection of the target vehicle, and the second warning area includes the ground area of ​​the target vehicle projection. Therefore, once an object is in the second warning area, it can be regarded as a foreign object under the vehicle. Figure 3 It is a schematic diagram of a region for dividing a ground area, wherein the second closed curve L2 surrounds the target vehicle V, and the first closed curve L1 surrounds the second closed curve L2. It is obvious that in any direction of the target vehicle V, the first distance between the first closed curve L1 and the target vehicle V is greater than the second distance between the target vehicle V of the second closed curve L2, and thus:

[0045] The second warning area D2 is a ground area including the projection of the target vehicle. Usually, the shape of the second warning area D2 is a convex polygon whose boundary is the second closed curve L2. Once an active object is in the second warning area D2, it can be regarded as being under the vehicle and being a foreign object under the vehicle.

[0046] The first warning area D1 is an annular area outside the second warning area D2, the outer boundary of the annular area is the first closed curve L1, and the inner boundary is the second closed curve L2; the area is close to the target vehicle, but does not include the ground area projected by the target vehicle; once an active object is in the first warning area D1, that is, the active object is close to the target vehicle, it may enter the second warning area D2 and become a foreign object under the vehicle, so the movement trajectory of the active object is tracked, that is, step S203 is executed to obtain analysis information of the foreign object under the vehicle in a timely manner;

[0047] The external area D0 is the ground area outside the first closed curve L1, or can be considered as other areas in the entire ground area except the second warning area D2 and the first warning area D1; this area is relatively far from the target vehicle and must not include the ground area projected by the target vehicle; the external area D0 is actually a relatively safe and unrelated area because it is far from the target vehicle. The shape of the external area D0 is determined according to the maximum acquisition range of the environmental sensor, and the farthest boundary of the area corresponding to the information that can be obtained in the environmental information flow is used as the external boundary of the external area D0. In fact, whether there is an active object in the external area D0 is not within the scope of concern of this embodiment. The active object only becomes an active object that needs to be concerned in this embodiment when it breaks through the first closed curve L1 and enters the first warning area D1 from the external area D0.

[0048] Similarly, whether an active object enters the second warning area D2 is not actually a conclusion drawn based on the information collected by the environmental sensor on the second warning area, but a conclusion drawn based on whether the active object breaks through the second closed curve L2 and leaves the first warning area D1. Therefore, it is not required that the environmental information flow must include all the information of the second warning area D2, and accurate analysis can be performed based on the information of the first warning area D1.

[0049] Understandably, Figure 3 This is just a specific example. The specific shapes of the first closed curve L1 and the second closed curve L2 are determined by a preset distance relationship, which includes the distance between each curve and the target vehicle, for example Figure 3 The first closed curve L1 is 1 meter away from the front, back, left, and right of the target vehicle, and the second closed curve L2 is 0.2 meter away from the front, back, left, and right of the target vehicle. The value of the distance can be set according to the sensor settings of the actual vehicle and user needs, and is not limited here.

[0050] Furthermore, the target vehicle is not necessarily in an open space when parked, and the target vehicle may be parked very close to a wall, e.g. Figure 4In one example, the rear of the target vehicle is close to the wall W, which cuts off the first closed curve L1 determined according to the preset distance relationship. At this time, when the area is divided according to the original first closed curve L1, the area behind the wall is invalid and does not need to be analyzed. Therefore, the first closed curve L1 and the second closed curve L2 can be updated according to the actual situation during parking, avoiding the analysis of invalid areas and improving monitoring efficiency.

[0051] Therefore, in some specific embodiments, when the target vehicle is in a parked state, the process of determining the first closed curve and the second closed curve surrounding the current target vehicle may include:

[0052] When the target vehicle is in a parked state, determining a first closed curve and a second closed curve surrounding the current target vehicle based on a preset distance relationship of the target vehicle;

[0053] After continuously acquiring the environmental information flow corresponding to the ground area through the environmental sensors of the target vehicle, it also includes:

[0054] The environmental information flow is analyzed, and when a fixed obstacle exists in the environmental information flow, the first closed curve and the second closed curve are updated based on the positions of the fixed obstacle and the target vehicle, and the external area, the first warning area, and the second warning area are updated.

[0055] It can be understood that fixed obstacles usually refer to fixed buildings, such as walls, columns, isolation fences, etc., which remain in the same position during the parking process of the target vehicle and hinder the running trajectory of the moving object; fixed obstacles can also include other objects whose positions may change, such as other cars parked next to the target vehicle. In this case, the action of updating the first closed curve and the second closed curve needs to be performed at a certain frequency to ensure that the partition of the target vehicle is updated in time after the position of the fixed obstacle changes or the fixed obstacle leaves.

[0056] It can be understood that if there is no fixed obstacle in the environmental information flow, the first closed curve and the second closed curve do not need to be updated.

[0057] It can be understood that the environmental information flow in this solution at least includes the information of the first warning area. Based on the information of the first warning area, it can be analyzed whether there are active objects entering the first warning area from the external area, whether there are active objects staying in the first warning area, whether there are active objects entering the second warning area from the first warning area, whether there are active objects entering the first warning area from the second warning area, etc. Therefore, in this embodiment, the environmental information flow must include the information corresponding to the first warning area. In this embodiment, there is no mandatory requirement for whether the environmental information flow includes the information corresponding to the second warning area and the external area. It can be selectively set according to the actual setting of the target vehicle, and is not limited here.

[0058] It can be understood that the environmental sensor includes at least one of an image sensor, an ultrasonic sensor, and a radar sensor, wherein the environmental information stream corresponding to the image sensor is a video stream corresponding to the image, and by analyzing each frame of the video stream, it is possible to determine whether there is an active object in the image and the location information of the active object; the ultrasonic sensor and the radar sensor measure distance and positioning through wave reflection, and if the distance change of an active object measured based on wave reflection is manifested as entering the first warning area, the motion trajectory of the active object can be analyzed.

[0059] It is understandable that in order to improve the reliability and comprehensiveness of the analysis, the environmental information flows of multiple environmental sensors can be fused to obtain more accurate results. The analysis of the environmental information flow includes two stages. The first stage is to monitor whether there are active objects entering the first warning area from the external area, and the second stage is to monitor the movement trajectory of the active objects entering the first warning area from the external area.

[0060] Specifically, the first stage is the triggering basis for the second stage. The first stage is a comprehensive monitoring of the first warning area. The monitoring purpose is: any moving object entering the first warning area from the external area will be identified and marked. The second stage is a specific target monitoring with moving objects as the main body. This stage focuses on the movement trajectory of moving objects. The change of the position of moving objects in different areas in the movement trajectory is the analysis result.

[0061] It can be understood that when the environmental sensor includes an image sensor, the environmental information flow includes a video stream obtained by the image sensor, and the image in the video stream is analyzed. The analysis content includes: identifying active objects in the image and determining in which area the active object is located, and when there is an active object in the image and the active object enters the first warning area from the external area, determining the movement trajectory of the active object.

[0062] Specifically, before identifying moving objects in an image, the image in the video stream may be preprocessed, and the preprocessing includes at least one of noise reduction, contrast enhancement, sharpness enhancement, and exposure adjustment. The preprocessing helps to improve the clarity and detail expression of the image, making the image taken in a dark environment brighter and clearer.

[0063] Specifically, the image recognition algorithm used to identify active objects in an image can select a YOLO (You Only Look Once) network or other networks. Taking the YOLO network as an example, the backbone network backbone in the YOLO network extracts features from the input image to obtain a series of feature maps of different levels and scales; the feature enhancement network neck in the YOLO network effectively integrates these feature maps to enhance the detection capability of the model; the detection head of the YOLO network will use the integrated features to predict the category and position of the objects in each grid, generate a prediction box, and perform classification calculation and confidence calculation, and filter the prediction box through strategies such as non-maximum suppression to obtain the recognition results of each active object. The recognition result selects the position of the active object in the image in the form of a prediction box, and the recognition result can also include the classification recognition result of the active object, and then the specific position of the active object in the ground area can be determined according to the recognition result.

[0064] Specifically, the motion trajectory of moving objects can be determined through the deepsort tracker network: for different prediction boxes in the recognition results, the tracker network will assign them a unique ID and update their tracking status along with the video stream. When the detection result is associated with a known target moving object, the Kalman filter is used to estimate its updated position and speed, and predict the position of the target in the next frame. The Hungarian algorithm is combined with the Mahalanobis distance and cosine distance to match the prediction result with the detection result.

[0065] Therefore, in some specific embodiments, the environmental information stream includes a video stream obtained by an image sensor, and the process of determining the motion trajectory of the active object based on the environmental information stream may include:

[0066] Identify active objects in the video stream based on an image recognition algorithm and obtain identification results of each active object;

[0067] Based on the target tracking algorithm and recognition results, the motion trajectory of each moving object is determined.

[0068] It can be understood that the active object in this embodiment is an object that exhibits a certain degree of activity and moves from the external area to the first warning area. The active object here can be a living thing, such as stray cats, stray dogs and children, or a non-living thing whose position changes, such as a sphere that is easy to roll, a remote-controlled car that is remotely controlled, etc. The specific type is not limited and all are active objects. The preset conditions of the active object can be further set and adjusted according to user settings or actual conditions.

[0069] In the case of multiple moving objects, the above algorithm can effectively distinguish different moving objects and accurately track and obtain accurate motion trajectories. In addition to the above algorithms, other algorithms can also be selected according to the form of environmental information flow and the configuration of specific target vehicles, as long as the motion trajectory of each moving object can be determined based on the environmental information flow.

[0070] It can be understood that based on the partition of the target vehicle, the possible movement trajectories of the active object are as follows Figure 5 As shown, including staying in the external area, entering the first warning area from the external area, entering the external area from the first warning area, staying in the first warning area, entering the second warning area from the first warning area, entering the first warning area from the second warning area, staying in the second warning area, etc., there is a certain jump relationship between these motion trajectories. This embodiment focuses on the entry action of the active object in terms of action, and focuses on the active objects in the first warning area and the second warning area in terms of area. According to the position relationship of each partition, if the active object remains in one area, it stays in the area; when it is concluded from the environmental information flow that the active object leaves the first warning area through the second closed curve, it can be concluded that the active object enters the second warning area from the first warning area. Therefore, as long as the environmental information flow can include the complete information of the first warning area, the judgment of the motion trajectory of the active object required by this embodiment can be realized: entering the first warning area from the external area, entering the external area from the first warning area, staying in the first warning area, entering the second warning area from the first warning area, entering the first warning area from the second warning area, and staying in the second warning area.

[0071] It can be understood that the motion trajectory of the active object implies the positional relationship between the active object and the second warning area, wherein entering the second warning area from the first warning area corresponds to the active object entering the second warning area, and entering the first warning area from the second warning area corresponds to the active object leaving the second warning area. Further, if the motion trajectory of the active object includes entering the second warning area from the first warning area, but does not include entering the first warning area from the second warning area, the current state of the active object is staying in the second warning area. If the motion trajectory of the active object includes entering the second warning area from the first warning area and entering the first warning area from the second warning area, the active object has entered the second warning area.

[0072] Therefore, based on the positional relationship between the active object and the second warning area in the motion trajectory, a foreign object report under the vehicle is determined, and the foreign object report under the vehicle is used to indicate whether there is an active object in the second warning area. The content of the foreign object report under the vehicle may include at least one of the current real-time foreign object count value under the vehicle and the historical record information of the foreign object under the vehicle.

[0073] The real-time underbody foreign object count value is used to indicate whether there is an active object in the current second warning area. When an active object is detected entering the second warning area from the first warning area, the real-time underbody foreign object count value is increased by one. When an active object is detected entering the first warning area from the second warning area, the real-time underbody foreign object count value is reduced by one. Usually, when the target vehicle enters the parking state, the real-time underbody foreign object count value will be initialized to zero, and then change accordingly according to the analysis of the environmental information flow. If time is ignored and only the change of the position of the active object in the motion trajectory is considered, the state of staying in a certain area can be ignored, and only the change of the motion trajectory from one area to another can be paid attention to. At this time, one idea is to count according to the latest state in the current motion trajectory of each active object. Specifically:

[0074] When the latest state of the motion trajectory of an active object is entering the second warning area from the first warning area, the real-time underbody foreign object count value is increased by 1;

[0075] When the latest state of the motion trajectory of an active object is entering the first warning area from the second warning area, the real-time underbody foreign object count value is reduced by 1.

[0076] Another idea is to count the current overall motion trajectory of each active object. Each time it enters the second warning area in the motion trajectory, the real-time undercarriage foreign object count value is increased by one, and each time it leaves the second warning area, that is, enters the first warning area from the second warning area, the real-time undercarriage foreign object count value is reduced by one. When these two ideas are implemented, all the motion trajectories of all active objects before the current moment are used as the counting basis, and a new value is determined based on the counting basis from zero, without considering the real-time undercarriage foreign object count value of the previous calculation cycle, and directly determining the new real-time undercarriage foreign object count value corresponding to the current calculation cycle. In addition to these two ideas, continuous monitoring can also be performed based on the changes in the motion trajectory of each active object over time. As time changes, when an active object is detected entering the second warning area from the first warning area, the real-time foreign object count value under the vehicle is increased by one. When an active object is detected entering the first warning area from the second warning area, the real-time foreign object count value under the vehicle is decreased by one. This type of monitoring continuously updates the real-time foreign object count value under the vehicle during the monitoring period. Each update takes the value of the previous monitoring period as the basis, and adds or subtracts based on the performance of the motion trajectory on the basis of the value of the previous monitoring period.

[0077] It can be understood that the calculation of all real-time under-vehicle foreign object count values ​​in this solution follows the entry and exit counting principle of adding one when entering the second warning area and subtracting one when exiting the second warning area. Therefore, in some specific embodiments, the under-vehicle foreign object report includes the real-time under-vehicle foreign object count value. Based on the positional relationship between the active object in the motion trajectory and the second warning area, the process of determining the under-vehicle foreign object report includes:

[0078] When the movement trajectory of a moving object includes entering the second warning area from the first warning area, the real-time underbody foreign object count value is increased by 1;

[0079] When the movement trajectory of a moving object includes entering the first warning area from the second warning area, the real-time underbody foreign object count value is reduced by 1.

[0080] The historical record information of foreign objects under the vehicle includes the information of the running tracks of all active objects that enter the second warning area from the first warning area or enter the first warning area from the second warning area during the parking process of the target vehicle in the current power-off cycle. The running tracks can be intercepted from the specific environmental information stream, and different forms of historical record information are determined according to the corresponding environmental sensors. For example, the environmental sensor includes an image sensor. In a specific embodiment, the environmental information stream includes a video stream obtained by the image sensor, and the report of foreign objects under the vehicle includes the action video of the active object. Based on the positional relationship between the active object and the second warning area in the motion trajectory, the process of determining the report of foreign objects under the vehicle can include:

[0081] In the case where the motion trajectory of an active object includes entering the second warning area from the first warning area, based on the video stream, determining the video of the active object entering under the vehicle and storing it in the target storage location; the video of entering under the vehicle includes the video of the active object entering the second warning area from the first warning area;

[0082] When the motion trajectory of an active object includes entering the first warning area from the second warning area, based on the video stream, the exit video of the active object under the vehicle is determined and stored in the target storage location; the exit video under the vehicle includes the video of the active object entering the first warning area from the second warning area.

[0083] It can be understood that, in addition to the changes between areas, the movement trajectory of the active object and the length of time it stays in a certain area can also be used as a basis for judging whether the data is valid, so as to reduce the interference of fast-moving objects on this solution. For example, when the active object enters the first warning area from the external area and the length of time it stays in the first warning area reaches the first length of time, the action of the active object entering the first warning area from the external area is considered to be valid, thereby triggering the action of determining the movement trajectory of the active object. If the length of time it stays in does not reach the first length of time and the active object enters the external area again, the entry action is considered to be invalid, and the action of determining the movement trajectory is not triggered. For another example, when the active object enters the second warning area from the first warning area and the length of time it stays in the second warning area reaches the second length of time, the action of the active object entering the second warning area from the first warning area is considered to be valid. Under the premise of this valid action, the movement trajectory of the active object is used as part of the basis for determining the report of foreign objects under the vehicle. If the length of time it stays in the second warning area does not reach the second length of time, the action of entering the second warning area is considered to be invalid, and the action of entering and exiting the second warning area will not be used as the basis for determining the report of foreign objects under the vehicle.

[0084] It can be understood that the determined under-vehicle foreign object report can be used to actively notify the user of the presence of foreign objects under the current vehicle or allow the user to remotely view the current situation and historical parking situation of the target vehicle. It can also be used as the result of the under-vehicle inspection before the user starts the target vehicle, and set the under-vehicle inspection alarm mechanism based on the current under-vehicle foreign object report, specifically including: obtaining the start instruction of the target vehicle; if the under-vehicle foreign object report indicates that there is no active object in the current second warning area, triggering the start instruction; if the under-vehicle foreign object report indicates that there is at least one active object in the current second warning area, triggering the corresponding warning instruction for the presence of foreign objects under the vehicle, and the warning instruction includes at least one of a prompt instruction or a horn-blowing instruction.

[0085] Therefore, in some specific embodiments, the method for monitoring foreign matter under the vehicle may further include:

[0086] Obtaining the start command of the target vehicle;

[0087] If the under-vehicle foreign object report indicates that there is no active object in the current second warning area, a start command is triggered;

[0088] If the under-vehicle foreign object report indicates that there is at least one moving object in the current second warning area, a warning instruction corresponding to the existence of the under-vehicle foreign object is triggered, and the warning instruction includes at least one of a prompt instruction or a horn-blowing instruction.

[0089] The prompt command is specifically used to control the human-machine interaction device to send a prompt message that there is a foreign object under the current target vehicle, so as to prompt the user to get off the vehicle and check under the vehicle. The human-machine interaction device can be a display screen or a speaker, and the specific form of the prompt message can be set according to the actual situation and user needs. The horn drive command is used to control the horn to drive away the foreign object under the vehicle.

[0090] It is understandable that in order to improve the image clarity of the image sensor, in a dark or dim environment, the image can be illuminated by a light source. Specifically, based on the ambient brightness, when the ambient brightness is lower than a specific brightness, the light source corresponding to the position of the moving object can be triggered to supplement the light for the moving object. If the image sensor is a conventional camera, the lighting can be turned on to illuminate the moving object. At the same time, the sudden lighting has a certain effect on driving away the moving object. If the image sensor is an infrared sensor of an infrared night vision system, the infrared light source can be turned on to provide light for the infrared sensor to ensure that the image sensor can capture a clear image. Therefore, in some specific embodiments, when an active object enters the first warning area from an external area in the environmental information flow, it also includes:

[0091] Based on the relative position between the moving object and the target vehicle, a lighting lamp corresponding to the relative position in the target vehicle is controlled to be turned on to illuminate the moving object.

[0092] It is understandable that the lighting lamps on the target vehicle include at least two groups of front lights and rear lights, which are responsible for lighting the area in front of the vehicle and behind the vehicle respectively. According to the location of the active object, the lighting lamps corresponding to the fan-shaped area where the location is located are turned on. The specific control details of the lighting lamps of the target vehicle and the corresponding relationship with the lighting area are determined by the actual configuration of the target vehicle, and are not limited here. The control principle and lighting area of ​​the lighting lamps in the reagent configuration can be used to control the corresponding lighting lamps to illuminate the active object.

[0093] It is understandable that the purpose of turning the light source on or off is to improve the image clarity. Therefore, as long as a moving object appears in the first warning area, the lighting or infrared light source at the corresponding position can be turned on to improve the clarity of the moving object in the image.

[0094] The method of the embodiment of the present application is based on the target vehicle in the parked state to divide the area: including the second warning area projected by the target vehicle, the first warning area outside the second warning area, and the external area outside the first warning area. Then, through the environmental sensor set on the target vehicle, the environmental information flow of the ground area is continuously obtained, and the motion trajectory is determined by analyzing the environmental information flow, and then the foreign object report under the vehicle is determined. The second warning area includes the area corresponding to the projection of the target vehicle. Once an active object enters the second warning area, it is considered to have entered the bottom of the target vehicle. Therefore, by monitoring whether the motion trajectory of the active object enters the second warning area, it is determined whether it has entered the bottom of the vehicle. The generation of the foreign object report under the vehicle can provide users with a reference for the status of foreign objects under the vehicle. Since the movement of the active object has a direction, this solution can complete the analysis of the specific motion trajectory of the active object as long as the environmental information flow obtained by the environmental sensor includes information corresponding to the first warning area. Therefore, the environmental sensor can use a sensor installed on the vehicle body to detect the surrounding environment. There is no need to deliberately set a sensor under the vehicle. It can reliably, stably and efficiently realize the monitoring of foreign objects under the vehicle, and it is convenient for later maintenance.

[0095] All the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here. It should be understood that the size of the sequence number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0096] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0097] Figure 6 Schematic diagram of a vehicle underbody foreign matter monitoring device provided in an embodiment of the present application. Figure 6 As shown, the vehicle bottom foreign body monitoring device comprises:

[0098] The area determination module 601 is used to determine the first closed curve and the second closed curve surrounding the current target vehicle when the target vehicle is in a parked state, and determine the corresponding external area, the first warning area and the second warning area based on the first closed curve and the second closed curve; in any direction of the target vehicle, the first distance between the first closed curve and the target vehicle is greater than the second distance between the second closed curve and the target vehicle, the external area is the ground area outside the first closed curve, the first warning area is the ground area between the first closed curve and the second closed curve, and the second warning area is the ground area within the second closed curve, including the ground area projected by the target vehicle;

[0099] The information flow acquisition module 602 is used to continuously acquire the environmental information flow corresponding to the ground area through the environmental sensor of the target vehicle;

[0100] A first analysis module 603 is used to analyze the environmental information flow, and when an active object enters the first warning area from the external area in the environmental information flow, determine the movement trajectory of the active object based on the environmental information flow;

[0101] The second analysis module 604 is used to determine the under-vehicle foreign object report based on the position relationship between the moving object and the second warning area in the motion trajectory.

[0102] The device of the embodiment of the present application divides the area based on the target vehicle in the parked state: including the second warning area projected by the target vehicle, the first warning area outside the second warning area, and the external area outside the first warning area, and then continuously obtains the environmental information flow of the ground area through the environmental sensor set on the target vehicle, determines the motion trajectory by analyzing the environmental information flow, and then determines the foreign object report under the vehicle. The second warning area includes the area corresponding to the projection of the target vehicle. Once an active object enters the second warning area, it is considered to have entered the bottom of the target vehicle. Therefore, by monitoring whether the motion trajectory of the active object enters the second warning area, it is determined whether it has entered the bottom of the vehicle, and the generation of the foreign object report under the vehicle can provide users with a reference for the status of foreign objects under the vehicle. Since the movement of the active object has a direction, this solution can complete the analysis of the specific motion trajectory of the active object as long as the environmental information flow obtained by the environmental sensor includes information corresponding to the first warning area. Therefore, the environmental sensor can use a sensor installed on the vehicle body to detect the surrounding environment, and there is no need to deliberately set a sensor under the vehicle. It can reliably, stably and efficiently realize the monitoring of foreign objects under the vehicle, and is convenient for later maintenance.

[0103] In an exemplary embodiment, the underbody foreign object report includes a real-time underbody foreign object count value, and the second analysis module is specifically used to:

[0104] When the movement trajectory of a moving object includes entering the second warning area from the first warning area, the real-time underbody foreign object count value is increased by 1;

[0105] When the movement trajectory of a moving object includes entering the first warning area from the second warning area, the real-time underbody foreign object count value is reduced by 1.

[0106] In an exemplary embodiment, the environmental sensor includes an image sensor, the environmental information stream includes a video stream acquired by the image sensor, the underbody foreign object report includes an action video of an active object, and the second analysis module is specifically configured to:

[0107] In the case where the motion trajectory of an active object includes entering the second warning area from the first warning area, based on the video stream, determining the video of the active object entering under the vehicle and storing it in the target storage location; the video of entering under the vehicle includes the video of the active object entering the second warning area from the first warning area;

[0108] When the motion trajectory of an active object includes entering the first warning area from the second warning area, based on the video stream, the exit video of the active object under the vehicle is determined and stored in the target storage location; the exit video under the vehicle includes the video of the active object entering the first warning area from the second warning area.

[0109] In an exemplary embodiment, the device further includes a vehicle bottom inspection warning module, which is used to:

[0110] Obtaining the start command of the target vehicle;

[0111] If the under-vehicle foreign object report indicates that there is no active object in the current second warning area, a start command is triggered;

[0112] If the under-vehicle foreign object report indicates that there is at least one moving object in the current second warning area, a warning instruction corresponding to the existence of the under-vehicle foreign object is triggered, and the warning instruction includes at least one of a prompt instruction or a horn-blowing instruction.

[0113] In an exemplary embodiment, the first analysis module is further configured to:

[0114] Based on the relative position between the moving object and the target vehicle, a lighting lamp corresponding to the relative position in the target vehicle is controlled to be turned on to illuminate the moving object.

[0115] In an exemplary embodiment, the environmental sensor includes an image sensor, the environmental information stream includes a video stream acquired by the image sensor, and the first analysis module is specifically configured to:

[0116] Identify active objects in the video stream based on an image recognition algorithm and obtain identification results of each active object;

[0117] Based on the target tracking algorithm and recognition results, the motion trajectory of each moving object is determined.

[0118] In an exemplary embodiment, the region determination module is specifically configured to:

[0119] When the target vehicle is in a parked state, determining a first closed curve and a second closed curve surrounding the current target vehicle based on a preset distance relationship of the target vehicle;

[0120] The environmental information flow is analyzed, and when a fixed obstacle exists in the environmental information flow, the first closed curve and the second closed curve are updated based on the positions of the fixed obstacle and the target vehicle, and the external area, the first warning area, and the second warning area are updated.

[0121] Figure 7 Schematic diagram of an electronic device 7 provided in an embodiment of the present application. Figure 7 As shown, the electronic device 7 of this embodiment includes: a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the computer program 703, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 701 executes the computer program 703, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0122] The electronic device 7 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 7 may include, but is not limited to, a processor 701 and a memory 702. Those skilled in the art will appreciate that Figure 7 The electronic device 7 is merely an example and does not limit the electronic device 7 , and may include more or less components than those shown in the figure, or different components.

[0123] The processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0124] The memory 702 may be an internal storage unit of the electronic device 7, for example, a hard disk or memory of the electronic device 7. The memory 702 may also be an external storage device of the electronic device 7, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. The memory 702 may also include both an internal storage unit of the electronic device 7 and an external storage device. The memory 702 is used to store computer programs and other programs and data required by the electronic device.

[0125] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0126] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0127] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for monitoring foreign matter under a vehicle, characterized in that: include: When the target vehicle is in a parked state, a first closed curve and a second closed curve surrounding the current target vehicle are determined, and based on the first closed curve and the second closed curve, corresponding external areas, first warning areas and second warning areas are determined; in any direction of the target vehicle, a first distance between the first closed curve and the target vehicle is greater than a second distance between the second closed curve and the target vehicle, the external area is a ground area outside the first closed curve, the first warning area is a ground area between the first closed curve and the second closed curve, the second warning area is a ground area within the second closed curve and the second warning area includes a ground area projected by the target vehicle; Continuously acquiring an environmental information flow corresponding to the ground area through an environmental sensor of the target vehicle; analyzing the environmental information flow, and determining a movement trajectory of an active object entering the first warning area from an external area in the environmental information flow based on the environmental information flow; Based on the positional relationship between the moving object and the second warning area in the motion trajectory, a report of a foreign object under the vehicle is determined.

2. The method according to claim 1, characterized in that The under-vehicle foreign object report includes a real-time under-vehicle foreign object count value. Based on the positional relationship between the active object and the second warning area in the motion trajectory, a process of determining the under-vehicle foreign object report includes: When the movement trajectory of one of the moving objects includes entering the second warning area from the first warning area, adding 1 to the real-time underbody foreign object count value; When the movement trajectory of one of the moving objects includes entering the first warning area from the second warning area, the real-time underbody foreign object count value is reduced by 1.

3. The method according to claim 1, characterized in that The environmental sensor includes an image sensor, the environmental information stream includes a video stream acquired by the image sensor, the under-vehicle foreign object report includes an action video of the active object, and based on the positional relationship between the active object and the second warning area in the motion trajectory, a process of determining the under-vehicle foreign object report includes: In a case where the motion trajectory of an active object includes entering the second warning area from the first warning area, based on the video stream, determining a video of the active object entering under the vehicle and storing it in a target storage location; the video of entering under the vehicle includes a video of the active object entering the second warning area from the first warning area; In a case where the movement trajectory of an active object includes entering the first warning area from the second warning area, based on the video stream, the exit video of the active object under the vehicle is determined and stored in the target storage location; the exit video under the vehicle includes the video of the active object entering the first warning area from the second warning area.

4. The method according to claim 1, characterized in that: Also includes: Obtaining a start instruction of the target vehicle; If the under-vehicle foreign object report indicates that there is no active object in the second warning area, triggering the start instruction; If the under-vehicle foreign object report indicates that at least one of the active objects currently exists in the second warning area, a warning instruction corresponding to the existence of the under-vehicle foreign object is triggered, and the warning instruction includes at least one of a prompt instruction or a horn-blowing instruction.

5. The method according to claim 1, characterized in that In the case where an active object enters the first warning area from an external area in the environmental information flow, the method further includes: Based on the relative position between the moving object and the target vehicle, a lighting lamp in the target vehicle corresponding to the relative position is controlled to be turned on to illuminate the moving object.

6. The method according to claim 1, characterized in that The environmental sensor includes an image sensor, the environmental information stream includes a video stream acquired by the image sensor, and the process of determining the motion trajectory of the moving object based on the environmental information stream includes: Identify the moving objects in the video stream based on an image recognition algorithm, and obtain recognition results of each moving object; Based on the target tracking algorithm and the recognition result, the motion trajectory of each of the moving objects is determined.

7. The method according to any one of claims 1 to 6, characterized in that When the target vehicle is in a parked state, the process of determining a first closed curve and a second closed curve surrounding the target vehicle includes: When the target vehicle is in a parked state, determining a first closed curve and a second closed curve surrounding the target vehicle based on a preset distance relationship of the target vehicle; After continuously acquiring the environmental information flow corresponding to the ground area through the environmental sensor of the target vehicle, the method further includes: The environmental information flow is analyzed, and when a fixed obstacle exists in the environmental information flow, the first closed curve and the second closed curve are updated based on the positions of the fixed obstacle and the target vehicle, and the external area, the first warning area, and the second warning area are updated.

8. A vehicle bottom foreign body monitoring device, characterized in that: include: An area determination module is used to determine a first closed curve and a second closed curve surrounding the current target vehicle when the target vehicle is in a parked state, and determine the corresponding external area, a first warning area and a second warning area based on the first closed curve and the second closed curve; in any direction of the target vehicle, a first distance between the first closed curve and the target vehicle is greater than a second distance between the second closed curve and the target vehicle, the external area is a ground area outside the first closed curve, the first warning area is a ground area between the first closed curve and the second closed curve, the second warning area is a ground area within the second closed curve, and the second warning area includes a ground area projected by the target vehicle; An information flow acquisition module, used to continuously acquire the environmental information flow corresponding to the ground area through the environmental sensor of the target vehicle; A first analysis module is used to analyze the environmental information flow, and when an active object entering the first warning area from an external area appears in the environmental information flow, determine the movement trajectory of the active object based on the environmental information flow; The second analysis module is used to determine the under-vehicle foreign object report based on the positional relationship between the active object and the second warning area in the motion trajectory.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images