Vehicle monitoring method and device and vehicle

Abstract

The invention relates to the technical field of vehicle control, and discloses a vehicle monitoring method and device and a vehicle. The target vehicle is provided with image display equipment, and the method comprises the following steps: acquiring monitoring information around the target vehicle; and if it is identified that the distance between the at least one object and the target vehicle is smaller than a first distance threshold value through the monitoring information and lasts for a first duration, displaying a virtual object on an image display device of the target vehicle. According to the scheme, the object close to the target vehicle can know that the target vehicle monitors the safety state of the object at the moment, and the object close to the target vehicle is warned by displaying the virtual object on the image display equipment, so that the safety risk of the target vehicle is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle control, and in particular to a vehicle monitoring method, device and vehicle. Background Art

[0002] The parking monitoring capabilities currently available on the market (referred to as Sentry Mode) were initially implemented using a single gyroscope, which would sound an alarm upon detecting a collision or vibration.

[0003] With the continuous advancement of automobile intelligence, vehicles can now remain in sentry mode after parking. Usually, in order to save energy, vehicles are usually in standby mode when parked. When there are people or objects approaching the vehicle, the vehicle can automatically enter the monitoring state in sentry mode, and collect and record the surrounding image information through the on-board camera. However, in the above solution, pedestrians around the vehicle cannot perceive whether the vehicle has turned on the sentry mode. Therefore, there is an urgent need for a method that can inform pedestrians around the vehicle that the sentry mode has been turned on and warn pedestrians around. Summary of the invention

[0004] In view of this, the present application provides a vehicle monitoring method, device and vehicle to reduce the safety risk of the target vehicle. The technical solution is as follows.

[0005] In a first aspect, a vehicle monitoring method is provided, the method being applied to a target vehicle; the method comprising:

[0006] Acquire monitoring information around the target vehicle;

[0007] If it is identified through the monitoring information that the distance between at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period, a virtual object is displayed on the image display device of the target vehicle.

[0008] In an optional embodiment, the method further includes:

[0009] If the camera recognizes that the face of at least one object is facing the target vehicle cabin, the image display device and at least one of the virtual objects are controlled to perform a first operation according to the relative position between the at least one object and the vehicle.

[0010] In an optional embodiment, the method further includes:

[0011] If it is recognized that the position information of the at least one object satisfies a specified condition, controlling at least one of the image display device and the virtual object to perform a first operation according to the relative position between the at least one object and the vehicle;

[0012] The specified conditions are that at least one object is in a first position; the distance between the first position and the target vehicle is less than a first distance threshold; and the acute angle formed by a line connecting the first position and the center point of the target vehicle and the central axis of the target vehicle is greater than a first angle threshold.

[0013] In an optional implementation, controlling the virtual object to perform the first operation includes:

[0014] The virtual object is controlled to perform a first virtual action toward the direction of the location of the at least one object.

[0015] In an optional implementation, the image display device is fixed to the vehicle via a rotating component, and the controlling the image display device to perform the first operation includes:

[0016] The image display device is controlled to rotate toward the location of the at least one object through a rotating component of the image display device.

[0017] In an optional implementation, if the at least one object includes a plurality of objects whose distance is greater than a second distance threshold, controlling the image display device to rotate toward the position of the at least one object through a rotating component of the image display device includes:

[0018] respectively obtaining the locations of the plurality of objects;

[0019] The image display device is controlled to swing toward the locations of the multiple objects by a rotating component of the image display device.

[0020] In an optional implementation, the monitoring information of the target vehicle further includes vibration information collected by a vibration sensor provided on the target vehicle;

[0021] The method further comprises:

[0022] When the vibration intensity indicated by the vibration information is greater than an intensity threshold, a virtual object is displayed on the image display device of the target vehicle.

[0023] In an optional embodiment, the method further includes:

[0024] When the vibration intensity indicated by the vibration information is greater than the intensity threshold, obtaining the vibration direction of the vibration information;

[0025] The image display device is controlled to rotate toward the vibration orientation, and / or the virtual object is controlled to perform a second virtual action toward the vibration orientation.

[0026] In a second aspect, a vehicle monitoring device is provided, the device being applied to a target vehicle; the target vehicle is provided with an image display device, the device comprising:

[0027] An acquisition module, used to acquire monitoring information around the target vehicle;

[0028] A display module is used to display a virtual object on an image display device of the target vehicle if it is identified through the monitoring information that the distance between at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period.

[0029] In an optional implementation, the display module is further used for:

[0030] If the camera recognizes that the face of at least one object is facing the target vehicle cabin, the image display device and at least one of the virtual objects are controlled to perform a first operation according to the relative position between the at least one object and the vehicle.

[0031] In an optional implementation, the display module is further used for:

[0032] If it is recognized that the position information of the at least one object satisfies a specified condition, controlling at least one of the image display device and the virtual object to perform a first operation according to the relative position between the at least one object and the vehicle;

[0033] The specified conditions are that at least one object is in a first position; the distance between the first position and the target vehicle is less than a first distance threshold; and the acute angle formed by a line connecting the first position and the center point of the target vehicle and the central axis of the target vehicle is greater than a first angle threshold.

[0034] In an optional implementation, the display module is further used for:

[0035] The virtual object is controlled to perform a first virtual action toward the direction of the location of the at least one object.

[0036] In an optional implementation, the image display device is fixed to the vehicle via a rotating component, and the display module is further used for:

[0037] Controlling the image display device to rotate toward the location of the at least one object by a rotating component of the image display device;

[0038] In an optional implementation, the at least one object includes a plurality of objects whose distance is greater than a second distance threshold, and the display module is further configured to:

[0039] respectively obtaining the locations of the plurality of objects;

[0040] The image display device is controlled to swing toward the locations of the multiple objects by a rotating component of the image display device.

[0041] In an optional implementation, the monitoring information of the target vehicle further includes vibration information collected by a vibration sensor provided on the target vehicle; and the display module is further used for:

[0042] When the vibration intensity indicated by the vibration information is greater than an intensity threshold, a virtual object is displayed on the image display device of the target vehicle.

[0043] In a third aspect, a vehicle is provided, wherein the computer device includes a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the above-mentioned vehicle monitoring method by executing the computer instructions.

[0044] In a fourth aspect, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the above-mentioned vehicle monitoring method.

[0045] In a fifth aspect, a computer program product or a computer program is provided, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the above-mentioned vehicle monitoring method.

[0046] The technical solution provided by this application may have the following beneficial effects:

[0047] The target vehicle in the present application can obtain monitoring information around the target vehicle, and the target vehicle can identify the monitoring information around the target vehicle. Once it is identified through the monitoring information that the distance between at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period, it means that there is at least one object close to the target vehicle. At this time, a virtual object will be displayed on the image display device of the target vehicle to warn the object close to the target vehicle, so that the object close to the target vehicle knows that the target vehicle has monitored its own safety status at this time, thereby reducing the safety risk of the target vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1A schematic diagram of a target vehicle involved in an embodiment of the present application is shown.

[0050] Figure 2 The present invention is a flow chart of a vehicle monitoring method according to an exemplary embodiment.

[0051] Figure 3 The present invention is a flow chart of a vehicle monitoring method according to an exemplary embodiment.

[0052] Figure 4 A schematic diagram showing a judgment of starting an alarm process involved in an embodiment of the present application

[0053] Figure 5 A schematic diagram of the orientation of a target vehicle involved in an embodiment of the present application is shown.

[0054] Figure 6 It is a structural schematic diagram of a vehicle monitoring device provided in an embodiment of the present application.

[0055] Figure 7 It is a structural schematic diagram of a computer device provided by an optional embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0057] In the description of the embodiments of the present application, the term "corresponding" may indicate a direct or indirect correspondence between two items, or an association relationship between the two items, or a relationship between indication and being indicated, configuration and being configured, and the like.

[0058] Please refer to Figure 1 , which shows a schematic diagram of a target vehicle involved in an embodiment of the present application. Figure 1 As shown, the target vehicle is provided with a plurality of sensors on its body. Optionally, the target vehicle may be provided with a first sensor 101 and a second sensor 102; correspondingly, the other side of the target vehicle may also be provided with a third sensor and a fourth sensor. Optionally, the front and rear of the target vehicle may also be provided with sensors to collect information in front of and behind the vehicle.

[0059] Optionally, in an embodiment of the present application, the sensors provided on the body of the target vehicle may include image sensors, millimeter wave sensors, ultrasonic sensors, and vibration sensors.

[0060] Among them, the image sensor can be an image acquisition device such as a camera. When image sensors are installed on both sides, the front and the rear of the vehicle body, the target vehicle can collect image information within 360 degrees around the target vehicle as required.

[0061] Among them, millimeter wave sensors and ultrasonic sensors can be used to detect information about objects close to the target vehicle. When millimeter wave sensors or ultrasonic sensors are installed on both sides, the front and the rear of the vehicle body, when an object approaches the target vehicle, the millimeter wave sensors or ultrasonic sensors can accurately distinguish the object and the direction between the object and the target vehicle.

[0062] The vibration sensor can be set at a specific position of the target vehicle to detect whether the target vehicle receives external vibration. Optionally, the vehicle controller of the target vehicle will obtain the data transmitted by the vibration sensor in real time. If the vibration data generated by the vibration sensor is too large, it means that the target vehicle may be at risk of collision (for example, being scratched by other vehicles in the parking lot, or colliding with other objects during driving). At this time, the vehicle controller of the target vehicle directly triggers the alarm process, performs operations such as honking the horn and turning on the double flash warning lights.

[0063] With the continuous advancement of automobile intelligence, vehicles can now be equipped with a sentry mode, so that when the target vehicle is parked and stopped at a designated location, it can monitor whether there is an object approaching the target vehicle and whether an object has performed certain actions on the target vehicle. However, in order to save energy, the vehicle is usually in a standby state in the sentry mode when it is parked, and the vehicle does not collect and record the surrounding environmental information at this time. When there are people or objects approaching the vehicle, the vehicle can automatically enter the monitoring state in the sentry mode, and collect and record the surrounding image information through the on-board camera. However, pedestrians around the vehicle cannot perceive whether the vehicle has turned on the sentry mode at this time. Although the sentry mode can record the relevant image information when pedestrians cause damage to the target vehicle, the target vehicle has been damaged at this time. The present application proposes a method that can display the corresponding virtual object through the image display device of the target vehicle when the pedestrian approaches the target vehicle, so that the pedestrian approaching the target vehicle knows that the target vehicle has turned on the sentry mode at this time, and is monitoring the surrounding images, thereby reducing the risk of pedestrians damaging the target vehicle.

[0064] Figure 2is a method flow chart of a vehicle monitoring method according to an exemplary embodiment. The method is applied to a target vehicle, and specifically the method can be executed by a vehicle controller in the target vehicle, and the vehicle controller can be a computer device. Figure 2 As shown, the vehicle monitoring method may include the following steps:

[0065] Step 201, obtaining monitoring information around the target vehicle.

[0066] Optionally, the monitoring information around the target vehicle is obtained when the target vehicle is in a parked state. When the target vehicle is in a parked state (for example, parked in a parking lot or road parking space for a long time), it is inevitable that various objects (such as motor vehicles, non-motor vehicles, and pedestrians, etc.) will approach the target vehicle during the long parking process. Therefore, the situation around the target vehicle can be monitored when the target vehicle is in a parked state, thereby obtaining the monitoring information around the target vehicle in the parked state.

[0067] Optionally, in an embodiment of the present application, the monitoring information of the target vehicle may include image information collected by an image sensor, and may also include distance information collected by an ultrasonic sensor or a millimeter wave sensor. Specifically, in an embodiment of the present application, the target vehicle will monitor the surrounding situation in real time, but even the target vehicle will not save all the monitored information. In an optional implementation of an embodiment of the present application, when the target vehicle detects that an object is approaching the target vehicle, the image sensor of the target vehicle can collect image information of the object's location and identify whether there is a face therein. If so, the monitoring information is retained, and the monitoring information will include facial information of at least one object identified by the target vehicle in at least one direction.

[0068] Step 202: If it is identified through monitoring information that the distance between at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period, a virtual object is displayed on an image display device of the target vehicle.

[0069] If the target vehicle obtains monitoring information, it can be judged according to the monitoring information whether the target vehicle has a safety risk at this time, such as whether there is an object close to the target vehicle, or there is a trend of contacting the target vehicle. If the target vehicle identifies the distance between at least one object and the target vehicle by monitoring information is less than the first distance threshold, and continues the first duration (for example, there is an object within two meters of the target vehicle, and it lasts more than 5 seconds), that is, there is an object close to the target vehicle, and it does not leave within a period of time, and it can be considered that the target vehicle has a certain safety risk at this time. Therefore, in order to avoid at least one object (such as a pedestrian, or a non-motor vehicle driven by a pedestrian, etc.) from causing damage to the target vehicle, a virtual object can be displayed on the image display device of the target vehicle, so as to inform the pedestrian that the target vehicle is in a state of "someone monitoring" at this time, and the pedestrian is warned not to cause damage to the target vehicle, thereby improving the safety of the target vehicle during parking.

[0070] In an embodiment of the present application, the virtual object may be a virtual person, animal, robot, or other 2D or 3D animation object that can perform anthropomorphic operations.

[0071] In an embodiment of the present application, the image display device may be a central control display screen inside the target vehicle.

[0072] In an optional embodiment, the image display device further includes a display area disposed on the body of the target vehicle. For example, a corresponding image display screen may be disposed at the front running light or rear running light of the target vehicle, and in this case, the target vehicle may display virtual objects on the image display screen at the front running light or rear running light of the target vehicle in addition to displaying virtual objects on the central control display screen.

[0073] To sum up, the target vehicle in the present application can obtain monitoring information around the target vehicle, and the target vehicle can identify the monitoring information around the target vehicle. Once it is identified through the monitoring information that the distance between at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period, it means that there is at least one object close to the target vehicle. At this time, a virtual object will be displayed on the image display device of the target vehicle to warn the object close to the target vehicle, so that the object close to the target vehicle knows that the target vehicle has monitored its own safety status at this time, thereby reducing the safety risk of the target vehicle.

[0074] Figure 3 is a method flow chart of a vehicle monitoring method according to an exemplary embodiment. The method is applied to a target vehicle, and specifically the method can be executed by a vehicle controller in the target vehicle, and the vehicle controller can be a computer device. Figure 3 As shown, the vehicle monitoring method may include the following steps:

[0075] Step 301, obtaining monitoring information around the target vehicle.

[0076] Optionally, in an embodiment of the present application, when the target vehicle is in a parking state, it can be determined whether to turn on the sentry mode according to the user's settings.

[0077] If the target vehicle turns on the sentry mode, the surrounding environment of the target vehicle can be monitored in real time through the sensors deployed on the target vehicle.

[0078] Optionally, since the power consumption of ultrasonic sensors is relatively low, in an embodiment of the present application, when the target vehicle is in the standby state of sentry mode, an ultrasonic sensor can be used to measure whether there are objects approaching around it at a specified frequency; and when an object is detected within a specified range, the millimeter wave sensor and / or image sensor is started to accurately collect information around the target vehicle, thereby generating monitoring information of the target vehicle.

[0079] Step 302: If it is identified through the monitoring information that the distance between the at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period, a virtual object is displayed on an image display device of the target vehicle.

[0080] In the embodiment of the present application, the image display device is a central control display screen arranged on the center console of the target vehicle.

[0081] In an embodiment of the present application, if the target vehicle identifies that the distance between at least one object and the target vehicle is less than the first distance threshold value through monitoring information, and lasts for the first duration, it means that the target vehicle has a safety hazard at this time. For example, the first duration is 5s, and the first distance threshold value is 2m. At this time, if the target vehicle detects that there is at least one object within 2m of the target vehicle (assuming that at least one object includes the first object, and the first object is a pedestrian), the target vehicle's alarm process can be triggered, that is, a virtual object is displayed on the image display device of the target vehicle. At this time, if the first object is still not far away from the target vehicle, it can be observed from the window of the target vehicle that a virtual object is displayed in the central control display screen of the target vehicle. At this time, the target vehicle can inform the first object by displaying a virtual object, and the target vehicle is in a state of starting and being monitored at this time, so as to warn the first object not to cause damage to the target vehicle, thereby improving the safety of the target vehicle during parking and reducing the risk of damage to the target vehicle during parking.

[0082] Optionally, the first object may be one of the objects with facial information in the monitoring information of the target vehicle; that is, if in the monitoring information of the target vehicle, if there is an object (such as the first object) whose distance from the target vehicle is less than a first distance threshold and lasts for a first time period, a virtual object may be displayed on the image display device of the target vehicle to issue a warning to the first object and other objects that may exist around the target vehicle.

[0083] Optionally, the first object may also be an object that is closest to the target vehicle among at least one object detected by the target vehicle.

[0084] Furthermore, the target vehicle can also be configured to determine the relationship between the facial orientation of at least one object and the target center point after identifying through monitoring information that the distance between at least one object and the target vehicle is less than a first distance threshold and lasts for a first period of time; if the facial orientation of the first object and the center point of the target vehicle meet preset conditions, a virtual object is displayed on the image display device.

[0085] Taking the example of at least one object including a first object, when it is detected that the distance between the first object and the target vehicle is less than a first distance threshold and lasts for a first time period, there is a possibility that the first object is approaching the target vehicle, but the first object may still just pass by the target vehicle at this time. At this time, it is not necessary to display the virtual object on the target display device.

[0086] Therefore, in order to minimize the possibility of the target vehicle's alarm being falsely triggered, it is necessary to further determine whether the target that the first object is approaching is the target vehicle. Figure 4 , which shows a schematic diagram of judging the start of an alarm process involved in an embodiment of the present application. Figure 4 As shown, specifically, the target vehicle can determine the facial orientation of the first object based on the facial information of the first object in the monitoring information of the target vehicle, and generate a first ray with the first object as the starting point and the facial orientation as the direction; at this time, if the distance between the center point of the target vehicle and the first ray is less than a preset distance (that is, the facial orientation of the first object and the center point of the target vehicle meet the preset conditions), it means that the first object is likely to approach or touch the target vehicle. Therefore, it can be considered that the target vehicle needs to start the alarm process to display a virtual object on the image display device of the target vehicle to warn the first object, thereby reducing the possibility of the first object causing damage to the target vehicle and improving the safety of the target vehicle during parking.

[0087] Furthermore, the first object is not a target object pre-stored in the target vehicle.

[0088] Optionally, the target vehicle may be provided with a facial recognition system, for example, the target vehicle may realize operations such as unlocking and controlling the vehicle through facial recognition. In order to determine which objects have the authority to unlock and control the vehicle, the user of the target vehicle may upload the facial information of the target object to the target vehicle in advance, so that the target object can realize unlocking and controlling the target vehicle through the facial information.

[0089] Therefore, when the target vehicle is in sentry mode, if the first object detected in the monitoring information of the target vehicle is the target object, it means that an authorized user (such as a driver) is approaching the target vehicle. At this time, the target vehicle does not need to display a virtual object on the image display device. On the contrary, in another possible implementation method, when the target object is detected approaching the target vehicle, the target vehicle is unlocked and the sentry mode is exited.

[0090] Step 303: If the camera recognizes that the face of at least one object is facing the target vehicle cabin, control the image display device and at least one of the virtual objects to perform a first operation according to the relative position between the at least one object and the vehicle.

[0091] In a possible implementation manner, the target vehicle controls the virtual object to perform a first virtual action toward the direction of the location of the at least one object.

[0092] Optionally, the first virtual action may be turning eyes, tilting the head, or moving to the first position. For example, when the first position where the first object is located is the left side of the target vehicle, the target vehicle may control the virtual object to turn its eyes to look to the left; or the target vehicle may control the virtual object to turn its head to the left to show a state of looking to the left; or the target vehicle may control the virtual object to move to the left to create a scene in which the virtual object approaches the first position to check whether anyone is approaching.

[0093] Taking the example of at least one object including a first object, at this time, since the face of the first object is facing the cockpit of the target vehicle, the first object is more likely to observe that the virtual object displayed on the image display device in the cockpit of the target vehicle performs a first virtual action. Therefore, through the first virtual action of the virtual object, the first object is more clearly warned not to damage the target vehicle, thereby further improving the safety of the target vehicle during parking.

[0094] In a possible implementation manner, the target vehicle controls the image display device to rotate toward the location of the at least one object through a rotating component of the image display device.

[0095] The image display device is fixed to the vehicle via a rotating component.

[0096] Optionally, the rotating component may be a rotating shaft, in which case the image display device is connected to the center console of the target vehicle via the rotating shaft, and the target vehicle controls the rotation of the rotating shaft to achieve the rotation of the target display device in different directions. For example, the target vehicle may control the target display device to rotate toward the main driving direction or the co-driving direction of the target vehicle via the rotating shaft.

[0097] At this time, assuming that the distance between the first object among at least one object and the target vehicle is less than a first distance threshold (less than 2 meters) and lasts for a first time length (more than 5 seconds), and the camera on the main driving side of the target vehicle collects the facial information of the first object, then the target vehicle can judge that the first object is in the main driving position of the target vehicle based on the facial information of the first object. At this time, the target vehicle can control the target display device to rotate toward the main driving direction of the target vehicle through the rotation axis to increase the possibility of the first object observing the virtual object of the target display device, thereby increasing the success rate of warning the first object through the virtual object, and then improving the safety of the target vehicle during parking.

[0098] Furthermore, if at least one object includes a second object and a third object, the target vehicle needs to determine whether the second object and the third object are in a similar position. If the positions are similar, for example, when both are captured by the camera on the main driving side of the target vehicle, the target vehicle can still control the target display device to rotate toward the main driving direction of the target vehicle through the rotation axis. However, if the positions are not similar, for example, the facial information of the second object is captured by the camera on the main driving side of the target vehicle, and the facial information of the third object is captured by the camera on the co-pilot side of the target vehicle, then the target vehicle needs to control the image display device to rotate toward the positions of the second object and the third object in turn through the rotation axis (for example, rotate toward the main driving direction and the co-pilot direction in turn).

[0099] Optionally, the rotation frequency is related to the distances between the second object and the third object and the target vehicle, for example, the smaller the distances between the second object and the third object and the target vehicle, the greater the rotation frequency.

[0100] Optionally, when the target vehicle controls the image acquisition device to rotate toward each direction where at least one object is located in turn, each time it rotates to a direction, the dwell time after the rotation is selected according to the distance between the object in that direction and the target vehicle. Similarly, taking the example that the second object and the third object are located on the main driver's side and the co-driver's side of the target vehicle respectively, if the second object is closer to the target vehicle and the third object is farther from the target vehicle, the image acquisition device will stay for the first time after rotating toward the main driver's direction; and then the image acquisition device will stay for the second time after rotating toward the co-driver's direction. In other words, the warning process of the target vehicle will focus more on objects that are closer to the target vehicle and have greater safety hazards, thereby reducing the safety risks during the parking process of the target vehicle.

[0101] For example, in the embodiment of the present application, the position where the first object (such as a human body) is close can be distinguished according to the situations recognized by different cameras. Specifically, it is recognized that within the range of the left-view camera, there is a human body approaching the target vehicle, and the distance between the human body and the target vehicle is less than the first distance threshold (which can be a preset value or a user-defined value within a certain range, for example, the user can select a value between 1 meter and 5 meters as the first distance threshold), and it lasts for a first duration (which can be a preset value or a user-defined value within a certain range, such as 2s to 10s), the screen deflects to the left by a certain angle (which can be a preset value or the maximum deflection angle of the image display device to the left by the rotation axis), and the pitch angle is 0 degrees. At this time, the central control display screen is horizontally deflected to the left. Similarly, when it is recognized that within the range of the right-view camera, there is a human body approaching and, and the distance between the human body and the target vehicle is less than the first distance threshold, and it lasts for a first duration, the screen deflects to the right by a certain angle (which can be a preset value or the maximum deflection angle of the image display device to the right by the rotation axis), and the pitch angle is 0 degrees (i.e., horizontal deflection). When a human body is detected approaching within the range of the front / rear view camera, no deflection operation is performed and the central control display screen remains facing the center.

[0102] That is, if it is recognized that the distance between people and the target vehicle in multiple perspectives is less than the first distance threshold and lasts for more than the first time period, the direction of the person closest to the target vehicle is used for processing. For example, if during an alarm process in sentry mode, someone approaches the left camera first and meets the conditions for executing the alarm process, the camera will first turn to the left; then someone approaches the right camera even closer and meets the conditions for executing the alarm process, the camera needs to turn to the right.

[0103] Optionally, in an embodiment of the present application, in addition to using a camera to identify the position of at least one object, a millimeter wave sensor or an ultrasonic sensor may be used to identify the position, and because when the image display device is connected to the center console of the target vehicle via a rotating axis, the rotation angle of the image display device is limited, that is, it can only be rotated to a certain angle toward the driver's seat or the co-driver's seat.

[0104] In this case, if there is at least one object close to the target vehicle, it is necessary for the object to rotate on both sides of the target vehicle. If the at least one object is at the front of the vehicle, it is difficult to rotate the image display device at the center console of the target vehicle to the front of the vehicle by rotating the rotation axis. If the at least one object is at the parking space, it is not necessary to rotate the rotation axis at this time.

[0105] Therefore, in a possible implementation, if it is recognized that the position information of the at least one object meets a specified condition, then according to the relative position between the at least one object and the vehicle, at least one of the image display device and the virtual object is controlled to perform a first operation;

[0106] The specified condition is that the at least one object is in a first position; the distance between the first position and the target vehicle is less than a first distance threshold; the acute angle formed by a line connecting the first position and the center point of the target vehicle and the central axis of the target vehicle is greater than a first angle threshold.

[0107] Please refer to Figure 5 , which shows a schematic diagram of the orientation of a target vehicle involved in an embodiment of the present application. Figure 5 As shown, in an embodiment of the present application, taking at least one object including a first object as an example, when the first object is detected by the sensor of the target vehicle, the target vehicle can obtain the specific position of the first object, and determine the first orientation of the first object based on the line connecting the first position of the first object and the center point of the target vehicle. At this time, if the acute angle formed by the line connecting the first position and the center point of the target vehicle and the central axis of the target vehicle is large, it can be explained that the first object is on the side of the target vehicle; and if the acute angle formed by the line connecting the first position and the center point of the target vehicle and the central axis of the target vehicle is small, it can be explained that the first object is in the front direction or the rear direction of the target vehicle.

[0108] Specifically, controlling the virtual object to perform the first operation may be controlling the virtual object to perform a first virtual action toward a first orientation where the first position is located.

[0109] When the first object is in the first position, and because the acute angle formed by the first position and the central axis of the target vehicle is greater than the first angle threshold, the first position can be considered as the side of the target vehicle, that is, the first object approaches the target vehicle from the side. Therefore, in order to further warn the first object, the virtual object displayed on the central control screen can be controlled to perform a first virtual action toward the first position where the first object is located, thereby improving the safety of the target vehicle during parking and avoiding damage to the vehicle by other objects as much as possible.

[0110] The first virtual action may be to turn the eyes, turn the head, or move to the first position. For example, when the first position where the first object is located is the left side of the target vehicle, the target vehicle may control the virtual object to turn the eyes to look to the left; or the target vehicle may control the virtual object to turn the head to the left to show a state of looking to the left; or the target vehicle may control the virtual object to move to the left to create a scene where the virtual object approaches the first position to check whether anyone is approaching.

[0111] Specifically, controlling the image display device to perform the first operation may be controlling the image display device to rotate to the first orientation by a specified angle.

[0112] Similarly, when the first object is in the first position, and because the acute angle formed by the first position and the central axis of the target vehicle is greater than the first angle threshold, the first position can be considered to be the side of the target vehicle, that is, the first object approaches the target vehicle from the side. Therefore, in order to increase the possibility of the first object observing the virtual object in the central control display screen, the angle of the central control display screen can be changed to reduce the risk of the first object causing damage to the target vehicle and improve the safety of the target vehicle during parking.

[0113] In another possible implementation of the embodiment of the present application, the rotating component can also be a robotic arm having multiple degrees of freedom, and the image display device is connected to the center console of the target vehicle via a rotating axis. In this case, the target vehicle can control the robotic arm having multiple degrees of freedom to move or rotate the image display device in various directions.

[0114] Furthermore, if the at least one object includes multiple objects whose distance is greater than a second distance threshold, the target vehicle respectively obtains the orientations of the locations of the multiple objects; and then controls the image display device to swing toward the orientations of the locations of the multiple objects through the rotating component of the image display device.

[0115] In the embodiment of the present application, it is also possible to determine whether the orientations of the objects are close by whether the distance between the objects is greater than a second distance threshold. Taking the example of multiple objects including a second object and a third object, if the distance between the second object and the third object is greater than the second distance threshold (for example, greater than 5m), it means that the second object and the third object are in different orientations, and the target vehicle can obtain the orientations of the second object and the third object through a camera, an ultrasonic sensor, or a millimeter wave sensor.

[0116] Furthermore, if the rotating component is a robotic arm, the image display device has a greater degree of freedom of movement and can move or rotate in various directions. Therefore, the target vehicle can control the robotic arm to make the image display device swing toward the direction where the second object is located and the direction where the third object is located.

[0117] For example, if the second object is located at the main driver's side and the third object is located at the co-driver's side, the target vehicle can use a robotic arm to control the image display device to swing to the windshield facing the main driver's side and the windshield facing the co-driver's side in sequence, thereby maximizing the possibility of the second object and the third object observing the virtual object on the image display device, and then ensuring the safety of the target vehicle during parking by alerting the second object and the third object.

[0118] For another example, if the second object is located at the front of the vehicle and the third object is located at the driving position, since the multi-degree-of-freedom robotic arm can move the image display device of the target vehicle to various positions in the cockpit of the target vehicle, the target vehicle can control the image display device to swing toward the front windshield and rear windshield of the target vehicle in sequence through the robotic arm, so that the second object and the third object located at the front of the vehicle and the parking space respectively also have a greater possibility of observing the virtual object of the image display device, further reducing the risk of objects close to the target vehicle in various positions causing damage to the target vehicle, and improving the safety of the target vehicle during parking.

[0119] Optionally, in an embodiment of the present application, the monitoring information of the target vehicle also includes vibration information collected by a vibration sensor provided on the target vehicle; at this time, the first alarm condition also includes that the vibration intensity indicated by the vibration information is greater than the intensity threshold, that is, when the vibration intensity indicated by the vibration information is greater than the intensity threshold, a virtual object is displayed on the image display device of the target vehicle.

[0120] That is, in the embodiment of the present application, in addition to detecting whether there are objects approaching the target vehicle around the target vehicle based on image sensors, ultrasonic sensors, millimeter wave sensors, etc., thereby determining whether an alarm process needs to be executed, the target vehicle can also determine whether an alarm process needs to be executed based on information collected by vibration sensors.

[0121] At this time, when the vibration intensity indicated by the vibration information collected by the vibration sensor is greater than the intensity threshold, it means that the target vehicle is also at risk of damage. Therefore, the target vehicle can enter the sentinel mode monitoring state and monitor the status around the target vehicle at the same time through image sensors, millimeter wave sensors and ultrasonic sensors.

[0122] Further, when the vibration intensity indicated by the vibration information is greater than the intensity threshold, the target vehicle obtains the vibration direction of the vibration information; controls the image display device to rotate toward the vibration direction, and / or controls the virtual object to perform a second virtual action toward the vibration direction.

[0123] When the vibration intensity indicated by the vibration information is greater than the intensity threshold, it can be indicated that the target vehicle may have collided, for example, the adjacent vehicle parked next to the target vehicle opened the door at a too large angle, resulting in a collision with the body of the target vehicle. At this time, the target vehicle may have suffered certain damage, and in order to prevent the user of the adjacent vehicle (such as the driver or passenger) from opening the door at too large an angle again, the target vehicle can determine the direction of the collision through the vibration direction of the vibration information, thereby controlling the image display device to rotate toward the vibration direction and / or controlling the virtual object to perform a second virtual action (one of turning the eyes, tilting the head, or moving to the first direction) toward the vibration direction to warn other objects, thereby reducing the secondary damage received by the target vehicle and improving the safety of the target vehicle.

[0124] To sum up, the target vehicle in the present application can obtain monitoring information around the target vehicle, and the target vehicle can identify the monitoring information around the target vehicle. Once it is identified through the monitoring information that the distance between at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period, it means that there is at least one object close to the target vehicle. At this time, a virtual object will be displayed on the image display device of the target vehicle to warn the object close to the target vehicle, so that the object close to the target vehicle knows that the target vehicle has monitored its own safety status at this time, thereby reducing the safety risk of the target vehicle.

[0125] In the embodiments of the present application, a vehicle monitoring device is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0126] The present application provides a vehicle monitoring device. Figure 6 : is a structural schematic diagram of a vehicle monitoring device provided in an embodiment of the present application, the device comprising:

[0127] An acquisition module 601 is used to acquire monitoring information around the target vehicle;

[0128] The display module 602 is used to display a virtual object on the image display device of the target vehicle if it is identified through the monitoring information that the distance between the at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period.

[0129] In an optional implementation, the display module is further used for:

[0130] If the camera recognizes that the face of at least one object is facing the target vehicle cabin, the image display device and at least one of the virtual objects are controlled to perform a first operation according to the relative position between the at least one object and the vehicle.

[0131] In an optional implementation, the display module is further used for:

[0132] If it is recognized that the position information of the at least one object satisfies a specified condition, controlling at least one of the image display device and the virtual object to perform a first operation according to the relative position between the at least one object and the vehicle;

[0133] The specified conditions are that at least one object is in a first position; the distance between the first position and the target vehicle is less than a first distance threshold; and the acute angle formed by a line connecting the first position and the center point of the target vehicle and the central axis of the target vehicle is greater than a first angle threshold.

[0134] In an optional implementation, the display module is further used for:

[0135] The virtual object is controlled to perform a first virtual action toward the direction of the location of the at least one object.

[0136] In an optional implementation, the image display device is fixed to the vehicle via a rotating component, and the display module is further used for:

[0137] Controlling the image display device to rotate toward the location of the at least one object by a rotating component of the image display device;

[0138] In an optional implementation, the at least one object includes a plurality of objects whose distance is greater than a second distance threshold, and the display module is further configured to:

[0139] respectively obtaining the locations of the plurality of objects;

[0140] The image display device is controlled to swing toward the locations of the multiple objects by a rotating component of the image display device.

[0141] In an optional implementation, the monitoring information of the target vehicle further includes vibration information collected by a vibration sensor provided on the target vehicle; and the display module is further used for:

[0142] When the vibration intensity indicated by the vibration information is greater than an intensity threshold, a virtual object is displayed on the image display device of the target vehicle.

[0143] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0144] The device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0145] The embodiment of the present invention further provides a computer device, which is arranged in a target vehicle to serve as a vehicle controller of the target vehicle. Figure 6 The device shown.

[0146] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphic information in a graphical user interface on an external input / output device (such as a display device coupled to an interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0147] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0148] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0149] The memory 20 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device and the like. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 may optionally include a memory remotely provided with respect to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0150] The memory 20 may include volatile memory, for example, random access memory; the memory may also include non-volatile memory, for example, flash memory, a hard disk, or a solid-state drive; the memory 20 may further include a combination of the above types of memory.

[0151] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0152] The embodiments of the present invention also provide a computer-readable storage medium. The methods according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code originally stored in a remote storage medium or a non-transitory machine-readable storage medium and downloaded through a network and to be stored in a local storage medium, so that the methods described herein can be stored in such software processes on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium can also include a combination of the above types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.

[0153] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0154] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A vehicle monitoring method, characterized in that: The method is applied to a target vehicle; The target vehicle is provided with an image display device, and the method comprises: Acquire monitoring information around the target vehicle; If it is identified through the monitoring information that the distance between at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period, a virtual object is displayed on the image display device of the target vehicle.

2. The method according to claim 1, characterized in that The method further comprises: If the camera recognizes that the face of at least one object is facing the target vehicle cabin, the image display device and at least one of the virtual objects are controlled to perform a first operation according to the relative position between the at least one object and the vehicle.

3. The method according to claim 1, characterized in that The method further comprises: If it is recognized that the position information of the at least one object satisfies a specified condition, controlling at least one of the image display device and the virtual object to perform a first operation according to the relative position between the at least one object and the vehicle; The specified conditions are that at least one object is in a first position; the distance between the first position and the target vehicle is less than a first distance threshold; and the acute angle formed by a line connecting the first position and the center point of the target vehicle and the central axis of the target vehicle is greater than a first angle threshold.

4. The method according to claim 2 or 3, characterized in that: Controlling the virtual object to perform a first operation includes: The virtual object is controlled to perform a first virtual action toward the direction of the location of the at least one object.

5. The method according to claim 2 or 3, characterized in that: The image display device is fixed to the vehicle via a rotating component, and the controlling the image display device to perform a first operation includes: The image display device is controlled to rotate toward the location of the at least one object through a rotating component of the image display device.

6. The method according to claim 5, characterized in that If the at least one object includes a plurality of objects whose distance is greater than a second distance threshold, controlling the image display device to rotate toward the position of the at least one object by a rotating component of the image display device includes: respectively obtaining the locations of the plurality of objects; The image display device is controlled to swing toward the locations of the multiple objects by a rotating component of the image display device.

7. The method according to any one of claims 1 to 3, characterized in that: The monitoring information of the target vehicle also includes vibration information collected by a vibration sensor provided on the target vehicle; The method further comprises: When the vibration intensity indicated by the vibration information is greater than an intensity threshold, a virtual object is displayed on the image display device of the target vehicle.

8. The method according to claim 7, characterized in that The method further comprises: When the vibration intensity indicated by the vibration information is greater than the intensity threshold, obtaining the vibration direction of the vibration information; The image display device is controlled to rotate toward the vibration orientation, and / or the virtual object is controlled to perform a second virtual action toward the vibration orientation.

9. A vehicle monitoring device, characterized in that: The device is applied to a target vehicle; The target vehicle is provided with an image display device, and the device comprises: An acquisition module, used to acquire monitoring information around the target vehicle; A display module is used to display a virtual object on an image display device of the target vehicle if it is identified through the monitoring information that the distance between at least one object and the target vehicle is less than a first distance threshold and lasts for a first time period.

10. A vehicle, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the vehicle monitoring method according to any one of claims 1 to 8 by executing the computer instructions.

Images