A vehicle watching method and device based on 3D space information

Abstract

The application provides a vehicle care method and device based on 3D space information. The method comprises the following steps: S1, acquiring a video stream covering all angles and directions of a target vehicle; S2, determining the position coordinates of the target vehicle and pedestrians appearing near the target vehicle in a three-dimensional space based on 3D target detection or 3D key point detection; S3, identifying the pedestrians and comparing features with a familiar person whitelist in a database to determine whether the pedestrians are in the familiar person whitelist; S4, if the pedestrians are not in the familiar person whitelist, calculating the distance and loitering time of the pedestrians from the target vehicle; S5, according to preset distance and time thresholds, determining whether the pedestrians have suspicious behavior on the vehicle; and S6, if the pedestrians have suspicious behavior on the vehicle, performing an alarm. The application improves the accuracy and sensitivity of the judgment of vehicle dangerous behavior.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular relates to a method and device for vehicle care based on 3D spatial information. Background Art

[0002] With the continuous acceleration of urbanization and the dramatic increase in the number of vehicles, cars have become an indispensable means of transportation in people's daily lives and a form of personal property. However, when parked, vehicles often face various risks such as theft, damage, and scratches, causing anxiety and losses for owners. Therefore, how to effectively monitor vehicles and prevent or promptly detect strangers approaching or engaging in inappropriate behavior is an urgent problem to be solved.

[0003] Currently, there are no mature vehicle monitoring systems on the market. While traditional monitoring systems can record video, they typically only perform two-dimensional predictions and fail to reflect the position and depth of objects in three-dimensional space. Consequently, they cannot accurately identify the relationship between pedestrians and vehicles. This can lead to false or missed alerts. For example, when someone is simply passing by the vehicle, or when someone is engaging in inappropriate behavior behind the vehicle, the system may not detect the anomaly and take timely action. Furthermore, vehicle monitoring systems cannot distinguish between acquaintances and strangers. For example, if the owner's family or friends approach the vehicle, the system may also issue an alert, causing unnecessary inconvenience and embarrassment. Furthermore, they lack the flexibility to set and adjust alert conditions because they typically rely on fixed parameters, such as triggering an alert if someone remains near the vehicle for a certain time or distance. However, these parameters are not applicable in all situations. For example, the normal distance and duration between a person and a vehicle may vary in different scenarios. As a result, the system may be overly sensitive or undersensitive, affecting the effectiveness and efficiency of the alert. Summary of the Invention

[0004] In order to solve the above technical problems, the present application provides a vehicle guarding method and device based on 3D spatial information, which aims to use image recognition technology to determine the target vehicles and acquaintances to be guarded, and flexibly adjust the time threshold and distance threshold to realize vehicle guarding in the target scene and identify strangers approaching or engaging in improper behavior towards the target vehicle.

[0005] The first aspect of the present application is a vehicle care method based on 3D spatial information, which mainly includes:

[0006] Step S1: Obtain video streams covering all angles and directions of the target vehicle;

[0007] Step S2: determining the position coordinates of the target vehicle and pedestrians near the target vehicle in three-dimensional space based on 3D target detection or 3D key point detection;

[0008] Step S3: Identify the pedestrian and compare features with the acquaintance whitelist in the database to determine whether the pedestrian is on the acquaintance whitelist;

[0009] Step S4: If the pedestrian is not in the acquaintance whitelist, calculate the distance and stay time between the pedestrian and the target vehicle;

[0010] Step S5: Determine whether the pedestrian has engaged in inappropriate behavior toward the vehicle based on a preset distance threshold and time threshold;

[0011] Step S6: If the pedestrian behaves inappropriately towards the vehicle, an alarm is issued.

[0012] Preferably, step S3 further comprises:

[0013] Step S31: determining a first characteristic value of the pedestrian based on face recognition, and calculating a first highest probability value of the pedestrian falling into the acquaintance whitelist; and determining a second characteristic value of the pedestrian based on human appearance recognition, and calculating a second highest probability value of the pedestrian falling into the acquaintance whitelist;

[0014] Step S32: When the first highest probability value and the second highest probability value correspond to the same individual in the acquaintance whitelist, the higher of the first highest probability value and the second highest probability value is increased by a set ratio and used as the final feature comparison value; when the first highest probability value and the second highest probability value correspond to different individuals in the acquaintance whitelist, the higher of the first highest probability value and the second highest probability value is used as the final feature comparison value;

[0015] Step S33: When the feature comparison value is greater than the similarity threshold, it is determined that the pedestrian is in the acquaintance whitelist.

[0016] Preferably, step S5 further comprises:

[0017] When the shortest distance between the pedestrian and the target vehicle is less than a distance threshold, and the stay time between the pedestrian and the target vehicle at the shortest distance is greater than a time threshold, it is determined that the pedestrian has misbehaved with the vehicle.

[0018] Preferably, step S5 further comprises:

[0019] Step S51: Discretize the designated distance around the target vehicle into n statistical nodes, and determine the distance L between each statistical node and the vehicle. i ;

[0020] Step S52: Determine the pedestrian's stay time T at each statistical node i ;

[0021] Step S53: Calculate the pedestrian stay index N:

[0022]

[0023] Among them, a is the adjustment factor, T set is the time threshold, L set is the distance threshold. When the stay index N is greater than 0, it is determined that the pedestrian has misbehaved with the vehicle.

[0024] Preferably, in step S6, the alarming method includes but is not limited to:

[0025] Control the horn of the target vehicle to make a sound, control the lights of the target vehicle to flash, or send a video stream to the owner of the vehicle.

[0026] The second aspect of the present application provides a vehicle care device based on 3D spatial information, mainly comprising:

[0027] The monitoring video data acquisition module is used to obtain video streams covering all angles and directions of the target vehicle;

[0028] A coordinate calculation module is used to determine the position coordinates of the target vehicle and pedestrians appearing near the target vehicle in three-dimensional space based on 3D target detection or 3D key point detection;

[0029] A pedestrian comparison module is used to identify the pedestrian and compare the characteristics with the acquaintance whitelist in the database to determine whether the pedestrian is on the acquaintance whitelist;

[0030] A pedestrian-vehicle distance and stay time statistics module is used to calculate the distance and stay time between the pedestrian and the target vehicle if the pedestrian is not in the acquaintance whitelist;

[0031] The misconduct recognition module is used to determine whether the pedestrian has misconducted the vehicle based on the preset distance threshold and time threshold;

[0032] The alarm module is used to sound an alarm if a pedestrian behaves in an unruly manner towards the vehicle.

[0033] Preferably, the pedestrian comparison module includes:

[0034] a dual probability value calculation unit, configured to determine a first characteristic value of the pedestrian based on face recognition and calculate a first highest probability value of the pedestrian falling into the acquaintance whitelist, and simultaneously determine a second characteristic value of the pedestrian based on human appearance recognition and calculate a second highest probability value of the pedestrian falling into the acquaintance whitelist;

[0035] a feature comparison value calculation unit, configured to, when the first highest probability value and the second highest probability value correspond to the same individual in the acquaintance whitelist, increase the higher of the first and second highest probability values ​​by a set percentage and use the higher of the first and second highest probability values ​​as the final feature comparison value; and, when the first and second highest probability values ​​correspond to different individuals in the acquaintance whitelist, use the higher of the first and second highest probability values ​​as the final feature comparison value;

[0036] An acquaintance determination unit is used to determine that the pedestrian is in an acquaintance whitelist when the feature comparison value is greater than a similarity threshold.

[0037] Preferably, the misconduct identification module includes:

[0038] A separate comparison unit is used to determine that the pedestrian has misbehaved with the vehicle when the shortest distance between the pedestrian and the target vehicle is less than a distance threshold and the stay time between the pedestrian and the target vehicle at the shortest distance is greater than a time threshold.

[0039] Preferably, the misconduct identification module includes:

[0040] The node distance statistics unit is used to discretize the specified distance around the target vehicle into n statistical nodes and determine the distance L between each statistical node and the vehicle. i ;

[0041] Node time statistics unit, used to determine the pedestrian's stay time T at each statistical node i ;

[0042] Pedestrian stay index calculation unit, used to calculate the pedestrian stay index N:

[0043]

[0044] Among them, a is the adjustment factor, T set is the time threshold, L set is the distance threshold. When the stay index N is greater than 0, it is determined that the pedestrian has misbehaved with the vehicle.

[0045] Preferably, the alarm method includes but is not limited to:

[0046] Control the horn of the target vehicle to make a sound, control the lights of the target vehicle to flash, or send a video stream to the owner of the vehicle.

[0047] In a third aspect of the present application, a computer device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the vehicle care method based on 3D spatial information as described in any one of the above items.

[0048] In a fourth aspect of the present application, a readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the vehicle care method based on 3D spatial information as described above.

[0049] This application improves the accuracy and sensitivity of judging dangerous vehicle behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flowchart of a preferred embodiment of the vehicle care method based on 3D spatial information of the present application.

[0051] Figure 2 It is a structural diagram of a computer device of a terminal or server suitable for implementing the embodiments of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the implementation of this application clearer, the technical solutions in the implementation of this application will be described in more detail below in conjunction with the drawings in the implementation of this application. In the drawings, the same or similar numbers throughout represent the same or similar elements or elements with the same or similar functions. The described implementation is a part of the implementation of this application, not all of the implementations. The implementation described below with reference to the drawings is exemplary and is intended to be used to explain this application, and should not be understood as a limitation on this application. Based on the implementation in this application, all other implementations obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The implementation of this application is described in detail below in conjunction with the drawings.

[0053] According to the first aspect of the present application, a vehicle care method based on 3D spatial information, such as Figure 1 As shown, it mainly includes:

[0054] Step S1: Obtain a video stream covering all angles and directions of the target vehicle.

[0055] This application is applicable to various vehicle care scenarios, including parking lots, residential parking lots, commercial areas, etc., to ensure the safety of vehicles. Therefore, in step S1, it is necessary to install cameras in these places and obtain real-time video streams. It is understandable that at least one camera needs to be installed around the target vehicle to capture images around the vehicle. The location and number of cameras can be determined based on the size and shape of the vehicle, as well as the environment and lighting of the target scene, to ensure that all angles and directions of the vehicle are covered.

[0056] Step S2: Determine the position coordinates of the target vehicle and pedestrians near the target vehicle in three-dimensional space based on 3D target detection or 3D key point detection.

[0057] In step S2, this application uses 3D target detection or 3D key point detection to detect the position coordinates of vehicles and pedestrians in three-dimensional space, and then in subsequent steps, determines whether there is any abnormal behavior towards the vehicle based on the distance and stay time between the pedestrians and the target vehicle. Here is a brief explanation of 3D target detection or 3D key point detection.

[0058] 3D object detection involves detecting and localizing objects of interest, such as cars, pedestrians, and bicycles, in three-dimensional space, providing a category label and 3D bounding box for each object, including its center coordinates, size, and orientation. The input data for 3D object detection is typically a 3D point cloud captured by a LiDAR or multi-camera system, or a 2D image captured by a monocular or binocular camera. Applications of 3D object detection include autonomous driving, robot navigation, and augmented reality. 3D keypoint detection involves detecting and localizing key points on the human body, such as the head, shoulders, wrists, and knees, in three-dimensional space, providing the three-dimensional coordinates of each keypoint. It can also be a parameterized human body model. The input data for 3D keypoint detection is typically a 2D image or video captured by a monocular or multi-camera system, or a 3D point cloud captured by a depth camera or LiDAR system. Applications of 3D keypoint detection include human-computer interaction, motion analysis, and rehabilitation training. In step S2, 3D object detection or 3D keypoint detection is performed on the image captured by the camera to determine the position coordinates of vehicles and pedestrians in three-dimensional space. 3D object detection or 3D keypoint detection can use existing deep learning or computer vision methods, such as PV-RCNN and MotionBERT, or you can design and train a new model to improve detection performance and efficiency. The results of 3D object detection or 3D keypoint detection can be output as coordinate values ​​for subsequent analysis and calculation.

[0059] It should be noted that the characteristic values ​​of the target vehicle in step S2 are pre-stored in the database and set in advance by the user. For example, you can select a picture of the target vehicle (including license plate number, model, color and other appearance information) through a mobile phone APP or other means, and store this information in the cloud or local database for subsequent identification and comparison.

[0060] Step S3: Identify the pedestrian and compare features with the acquaintance whitelist in the database to determine whether the pedestrian is in the acquaintance whitelist.

[0061] As described in step S2 above, the vehicle information is stored in the database. The acquaintance whitelist is also stored in the database in advance. The user can select the face and body images of acquaintances through a mobile phone app or other means, and then store this information in the cloud or local database for subsequent identification and comparison. Based on the preset acquaintance information, facial recognition and body appearance recognition can be used to comprehensively determine whether the target pedestrian is an acquaintance. For example, in some optional embodiments, step S3 further includes:

[0062] Step S31: determining a first characteristic value of the pedestrian based on face recognition, and calculating a first highest probability value of the pedestrian falling into the acquaintance whitelist; and determining a second characteristic value of the pedestrian based on human appearance recognition, and calculating a second highest probability value of the pedestrian falling into the acquaintance whitelist;

[0063] Step S32: When the first highest probability value and the second highest probability value correspond to the same individual in the acquaintance whitelist, the higher of the first highest probability value and the second highest probability value is increased by a set ratio and used as the final feature comparison value; when the first highest probability value and the second highest probability value correspond to different individuals in the acquaintance whitelist, the higher of the first highest probability value and the second highest probability value is used as the final feature comparison value;

[0064] Step S33: When the feature comparison value is greater than the similarity threshold, it is determined that the pedestrian is in the acquaintance whitelist.

[0065] This embodiment uses two methods to identify whether a pedestrian is an acquaintance. Each method can determine the similarity between the pedestrian and each acquaintance in the database based on step S31, and take the highest value as the first highest probability value and the second highest probability value. Then, in step S32, it is determined whether the two methods correspond to the same acquaintance. If so, the probability of identifying as an acquaintance is considered higher, and the higher value of the first highest probability value and the second highest probability value needs to be increased. After the increase, a new feature comparison value is formed, which is then compared with the similarity threshold in step S33 to finally determine whether the pedestrian is in the acquaintance whitelist. For example, if the pedestrian is determined to be most similar to pedestrian A in the database based on face recognition, the first highest probability value is 70%, and if the pedestrian is determined to be most similar to pedestrian A in the database based on human appearance recognition, the second highest probability value is 80%, then the feature comparison value is increased by 10% based on the second highest probability value, that is, 88%.

[0066] Step S4: If the pedestrian is not in the acquaintance whitelist, the distance and stay time between the pedestrian and the target vehicle are calculated.

[0067] If the pedestrian in the image is a familiar person, no alarm is issued. However, if the pedestrian in the image is a stranger, the distance and duration between the pedestrian and the target vehicle are further calculated. This information can then be compared with distance and time thresholds to determine whether to issue an alarm. These distance and duration can typically be calculated for each frame in the video stream.

[0068] Step S5: Determine whether the pedestrian has engaged in unlawful behavior toward the vehicle based on a preset distance threshold and a preset time threshold.

[0069] The most common way to determine whether a pedestrian has engaged in unruly behavior towards a vehicle is to calculate the minimum value of the distances in step S4, and determine whether the minimum distance, that is, the shortest distance and its duration exceeds a threshold to determine whether there has been unruly behavior towards the vehicle. For example, in some optional embodiments, step S5 further includes: when the shortest distance between the pedestrian and the target vehicle is less than a distance threshold, and the time the pedestrian and the target vehicle stay at the shortest distance is greater than a time threshold, it is determined that the pedestrian has engaged in unruly behavior towards the vehicle.

[0070] In an alternative embodiment, a comprehensive statistical analysis of the time the pedestrian spends at different locations from the target vehicle as reflected in the entire video stream can be performed, and then compared with a distance threshold and a time threshold to determine whether there is any misconduct against the vehicle. For example, in some optional embodiments, step S5 further includes:

[0071] Step S51: Discretize the designated distance around the target vehicle into n statistical nodes, and determine the distance L between each statistical node and the vehicle. i ;

[0072] Step S52: Determine the pedestrian's stay time T at each statistical node i ;

[0073] Step S53: Calculate the pedestrian stay index N:

[0074]

[0075] Among them, a is the adjustment factor, T set is the time threshold, L set is the distance threshold. When the stay index N is greater than 0, it is determined that the pedestrian has misbehaved with the vehicle.

[0076] In this embodiment, the adjustment factor, time threshold and distance threshold can be adjusted flexibly. For example, a=6, T set =5,L set =1, these parameters can be written in the program or specified by the user. The specified distance in step S51 is usually the same as L setFor example, if the distance is 1m, every 20cm is discretized into a statistical node. This design can also be written in the program, and its parameters can also be specified by the user. At 0.2m, the pedestrian's stay time is 6s, at 0.4m, the pedestrian's stay time is 4s, at 0.6m, the pedestrian's stay time is 6s, at 0.8m, the pedestrian's stay time is 2.4s, and at 1m, the pedestrian's stay time is 4s. Then N = 6 / 0.2 + 4 / 0.4 + 6 / 0.6 + 2.4 / 0.8 + 4 / 1 - 6*5*1 = 27. Therefore, it is determined that the pedestrian has engaged in misconduct towards the vehicle.

[0077] In some optional embodiments, the adjustment factor a can be set to a variable value and is proportional to the feature comparison value k formed during the feature comparison in step S3, that is, a=f(k), thereby turning S5 into an adaptive identification step for pedestrians engaging in misconduct towards vehicles. It can be understood that the feature comparison value is used to indicate the probability that the pedestrian is an acquaintance. Since the similarity threshold is set relatively high, when the pedestrian is judged to be not an acquaintance in the database, the conclusion that the pedestrian is not an acquaintance cannot be completely denied. In this case, if the feature comparison value is higher, it means that the pedestrian is still likely to be an acquaintance. At this time, the adjustment factor a is higher, and there is a higher tolerance for the pedestrian approaching the vehicle.

[0078] Step S6: If the pedestrian behaves inappropriately towards the vehicle, an alarm is issued.

[0079] In some optional implementations, in step S6, the manner of issuing an alarm includes but is not limited to:

[0080] Control the horn of the target vehicle to make a sound, control the lights of the target vehicle to flash, or send a video stream to the owner of the vehicle.

[0081] In this embodiment, there can be multiple ways of giving an alarm, such as: issuing a sound or light warning, such as making the horn or lights of the target vehicle sound or flash to attract the attention of people around or scare away strangers; notifying the owner through a mobile phone APP, such as sending a text message or push message to the owner's mobile phone to inform the location and situation of the vehicle, or providing image or video evidence so that the owner can deal with it in time; contacting the police or security, such as calling a nearby police station or security company to report the location and situation of the vehicle, or providing image or video evidence so that the police or security can arrive at the scene in time.

[0082] This application improves the accuracy and sensitivity of judging dangerous vehicle behavior.

[0083] The second aspect of the present application provides a vehicle care device based on 3D spatial information corresponding to the above method, mainly comprising:

[0084] The monitoring video data acquisition module is used to obtain video streams covering all angles and directions of the target vehicle;

[0085] A coordinate calculation module is used to determine the position coordinates of the target vehicle and pedestrians appearing near the target vehicle in three-dimensional space based on 3D target detection or 3D key point detection;

[0086] A pedestrian comparison module is used to identify the pedestrian and compare the characteristics with the acquaintance whitelist in the database to determine whether the pedestrian is on the acquaintance whitelist;

[0087] A pedestrian-vehicle distance and stay time statistics module is used to calculate the distance and stay time between the pedestrian and the target vehicle if the pedestrian is not in the acquaintance whitelist;

[0088] The misconduct recognition module is used to determine whether the pedestrian has misconducted the vehicle based on the preset distance threshold and time threshold;

[0089] The alarm module is used to sound an alarm if a pedestrian behaves in an unruly manner towards the vehicle.

[0090] In some optional implementations, the pedestrian comparison module includes:

[0091] a dual probability value calculation unit, configured to determine a first characteristic value of the pedestrian based on face recognition and calculate a first highest probability value of the pedestrian falling into the acquaintance whitelist, and simultaneously determine a second characteristic value of the pedestrian based on human appearance recognition and calculate a second highest probability value of the pedestrian falling into the acquaintance whitelist;

[0092] a feature comparison value calculation unit, configured to, when the first highest probability value and the second highest probability value correspond to the same individual in the acquaintance whitelist, increase the higher of the first and second highest probability values ​​by a set percentage and use the higher of the first and second highest probability values ​​as the final feature comparison value; and, when the first and second highest probability values ​​correspond to different individuals in the acquaintance whitelist, use the higher of the first and second highest probability values ​​as the final feature comparison value;

[0093] An acquaintance determination unit is used to determine that the pedestrian is in an acquaintance whitelist when the feature comparison value is greater than a similarity threshold.

[0094] In some optional implementations, the misconduct identification module includes:

[0095] A separate comparison unit is used to determine that the pedestrian has misbehaved with the vehicle when the shortest distance between the pedestrian and the target vehicle is less than a distance threshold and the stay time between the pedestrian and the target vehicle at the shortest distance is greater than a time threshold.

[0096] In some optional implementations, the misconduct identification module includes:

[0097] The node distance statistics unit is used to discretize the specified distance around the target vehicle into n statistical nodes and determine the distance L between each statistical node and the vehicle. i ;

[0098] Node time statistics unit, used to determine the pedestrian's stay time T at each statistical node i ;

[0099] Pedestrian stay index calculation unit, used to calculate the pedestrian stay index N:

[0100]

[0101] Among them, a is the adjustment factor, T set is the time threshold, L set is the distance threshold. When the stay index N is greater than 0, it is determined that the pedestrian has misbehaved with the vehicle.

[0102] In some optional implementations, the manner of issuing an alarm includes but is not limited to:

[0103] Control the horn of the target vehicle to make a sound, control the lights of the target vehicle to flash, or send a video stream to the owner of the vehicle.

[0104] In a third aspect of the present application, a computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a vehicle care method based on 3D spatial information.

[0105] In a fourth aspect of the present application, a readable storage medium stores a computer program that, when executed by a processor, implements the vehicle care method based on 3D spatial information described above. The computer-readable storage medium may be included in the apparatus described in the above embodiments, or it may exist independently and not incorporated into the apparatus. The computer-readable storage medium carries one or more programs that, when executed by the apparatus, process data according to the above method.

[0106] Reference below Figure 2 , which shows a structural diagram of a computer device 400 suitable for implementing the embodiments of the present application. Figure 2 The computer device shown is only an example and should not limit the functions and scope of use of the embodiments of the present application.

[0107] like Figure 2As shown, computer device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage portion 408 into a random access memory (RAM) 403. Various programs and data required for the operation of device 400 are also stored in RAM 403. CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to bus 404.

[0108] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 408 including a hard disk; and a communication section 409 including a network interface card such as a LAN card or a modem. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. Removable media 411, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 410 as needed, so that computer programs read therefrom can be installed into the storage section 408 as needed.

[0109] In particular, according to the embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above functions defined in the method of the present application are executed. It should be noted that the computer storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0110] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code includes one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0111] The modules or units described in the embodiments of this application may be implemented in software or hardware. The modules or units described may also be provided in a processor, and the names of these modules or units do not, in certain circumstances, limit the modules or units themselves.

[0112] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A vehicle care method based on 3D spatial information, characterized in that: include: Step S1: Obtain video streams covering all angles and directions of the target vehicle; Step S2: determining the position coordinates of the target vehicle and pedestrians near the target vehicle in three-dimensional space based on 3D target detection or 3D key point detection; Step S3: Identify the pedestrian and compare features with the acquaintance whitelist in the database to determine whether the pedestrian is on the acquaintance whitelist; Step S4: If the pedestrian is not in the acquaintance whitelist, calculate the distance and stay time between the pedestrian and the target vehicle; Step S5: Determine whether the pedestrian has engaged in inappropriate behavior toward the vehicle based on a preset distance threshold and time threshold; Step S6: If the pedestrian behaves inappropriately towards the vehicle, an alarm is issued; Wherein, step S5 further includes: Step S51: Discretize the designated distance around the target vehicle into n statistical nodes, and determine the distance L between each statistical node and the vehicle. i ; Step S52: Determine the pedestrian's stay time T at each statistical node i ; Step S53: Calculate the pedestrian stay index N: Among them, a is the adjustment factor, T set is the time threshold, L set is the distance threshold. When the stay index N is greater than 0, it is determined that the pedestrian has misbehaved with the vehicle; The adjustment factor a is proportional to the feature comparison value k formed when the pedestrian is compared with the acquaintance whitelist in the database.

2. The vehicle care method based on 3D spatial information according to claim 1, characterized in that: Step S3 further comprises: Step S31: determining a first characteristic value of the pedestrian based on face recognition, and calculating a first highest probability value of the pedestrian falling into the acquaintance whitelist; and determining a second characteristic value of the pedestrian based on human appearance recognition, and calculating a second highest probability value of the pedestrian falling into the acquaintance whitelist; Step S32: When the first highest probability value and the second highest probability value correspond to the same individual in the acquaintance whitelist, the higher of the first highest probability value and the second highest probability value is increased by a set ratio and used as the final feature comparison value; when the first highest probability value and the second highest probability value correspond to different individuals in the acquaintance whitelist, the higher of the first highest probability value and the second highest probability value is used as the final feature comparison value; Step S33: When the feature comparison value is greater than the similarity threshold, it is determined that the pedestrian is in the acquaintance whitelist.

3. The vehicle care method based on 3D spatial information according to claim 1, characterized in that: Step S5 further comprises: When the shortest distance between the pedestrian and the target vehicle is less than a distance threshold, and the stay time between the pedestrian and the target vehicle at the shortest distance is greater than a time threshold, it is determined that the pedestrian has misbehaved with the vehicle.

4. The vehicle care method based on 3D spatial information according to claim 1, characterized in that: In step S6, the alarming method includes but is not limited to: Control the horn of the target vehicle to make a sound, control the lights of the target vehicle to flash, or send a video stream to the owner of the vehicle.

5. A vehicle care device based on 3D spatial information, characterized in that: include: The monitoring video data acquisition module is used to obtain video streams covering all angles and directions of the target vehicle; A coordinate calculation module is used to determine the position coordinates of the target vehicle and pedestrians appearing near the target vehicle in three-dimensional space based on 3D target detection or 3D key point detection; A pedestrian comparison module is used to identify the pedestrian and compare the characteristics with the acquaintance whitelist in the database to determine whether the pedestrian is on the acquaintance whitelist; A pedestrian-vehicle distance and stay time statistics module is used to calculate the distance and stay time between the pedestrian and the target vehicle if the pedestrian is not in the acquaintance whitelist; The misconduct recognition module is used to determine whether the pedestrian has misconducted the vehicle based on the preset distance threshold and time threshold; An alarm module is used to sound an alarm if a pedestrian behaves inappropriately towards the vehicle; The misconduct identification module includes: The node distance statistics unit is used to discretize the specified distance around the target vehicle into n statistical nodes and determine the distance L between each statistical node and the vehicle. i ; Node time statistics unit, used to determine the pedestrian's stay time T at each statistical node i ; Pedestrian stay index calculation unit, used to calculate the pedestrian stay index N: Among them, a is the adjustment factor, T set is the time threshold, L set is the distance threshold. When the stay index N is greater than 0, it is determined that the pedestrian has misbehaved with the vehicle; The adjustment factor a is proportional to the feature comparison value k formed when the pedestrian is compared with the acquaintance whitelist in the database.

6. The vehicle care device based on 3D spatial information according to claim 5, characterized in that: The pedestrian comparison module includes: a dual probability value calculation unit, configured to determine a first characteristic value of the pedestrian based on face recognition and calculate a first highest probability value of the pedestrian falling into the acquaintance whitelist, and simultaneously determine a second characteristic value of the pedestrian based on human appearance recognition and calculate a second highest probability value of the pedestrian falling into the acquaintance whitelist; a feature comparison value calculation unit, configured to, when the first highest probability value and the second highest probability value correspond to the same individual in the acquaintance whitelist, increase the higher of the first and second highest probability values ​​by a set percentage and use the higher of the first and second highest probability values ​​as the final feature comparison value; and, when the first and second highest probability values ​​correspond to different individuals in the acquaintance whitelist, use the higher of the first and second highest probability values ​​as the final feature comparison value; An acquaintance determination unit is used to determine that the pedestrian is in an acquaintance whitelist when the feature comparison value is greater than a similarity threshold.

7. The vehicle care device based on 3D spatial information according to claim 5, characterized in that: The misconduct identification module includes: A separate comparison unit is used to determine that the pedestrian has misbehaved with the vehicle when the shortest distance between the pedestrian and the target vehicle is less than a distance threshold and the stay time between the pedestrian and the target vehicle at the shortest distance is greater than a time threshold.

8. The vehicle care device based on 3D spatial information according to claim 5, characterized in that: Ways to report an incident include but are not limited to: Control the horn of the target vehicle to make a sound, control the lights of the target vehicle to flash, or send a video stream to the owner of the vehicle.

Images