Method, system, device set and storage medium for vehicle monitoring

Abstract

The embodiment of the application discloses a kind of vehicle monitoring method, system, equipment group and storage medium, belong to automobile technical field.The method comprises: when target vehicle is in the state of parking and locking, any sensor detects that there is object activity around target vehicle, and the domain controller of target vehicle wakes up multiple cameras to take image.Target vehicle can identify the image taken by security identification model, if the identification result is unsafe, then the domain controller can save the image taken, upload to server, and notify user to view.In this way, if object activity causes target vehicle to be damaged, then user can retrieve the image taken from domain controller or server, find the person or vehicle involved in the accident, so as to obtain economic compensation.Using the embodiment of the application, the possibility of property loss can be reduced.

Description

Technical Field

[0001] This application relates to the field of automotive technology, and in particular to a method, system, device group, and storage medium for vehicle monitoring. Background Technology

[0002] With social development and the advancement of science and technology, the number of cars on the road nationwide is increasing. As a result, the number of parking spaces is becoming insufficient, and drivers often need to park their vehicles in distant or remote parking spaces.

[0003] Thus, when a vehicle is scratched or stolen by another vehicle, the driver often cannot obtain relevant evidence because there are no cameras installed near the parking space, resulting in significant property loss. Summary of the Invention

[0004] This application provides a method, system, device group, and storage medium for vehicle monitoring, which can solve the problems in related technologies. The technical solution is as follows:

[0005] Firstly, a method for vehicle monitoring is provided, the method being applied to a vehicle monitoring system, the vehicle monitoring system including a mobile terminal, a server, and a target vehicle, the target vehicle having multiple cameras, multiple ultrasonic sensors, a tilt sensor, an anti-theft device, a domain controller, and a wireless gateway, including:

[0006] When the target vehicle is parked and locked, and the number of times the periodic detection data of any ultrasonic sensor changes within a specified time exceeds a preset threshold, the domain controller wakes up the multiple cameras.

[0007] The domain controller controls the plurality of cameras to begin capturing images of the area surrounding the target vehicle;

[0008] The multiple cameras send the captured images to the domain controller;

[0009] When the shooting time of the multiple cameras reaches the preset shooting time, shooting stops and the camera enters a sleep state.

[0010] The domain controller inputs the images captured by the multiple cameras within a preset shooting time into the security identification model. If the output result of the security identification model is secure, the images captured within the preset shooting time are deleted. If the output result of the security identification model is insecure, the images captured within the preset shooting time are stored locally and sent to the wireless gateway. The security identification model is a machine learning model.

[0011] The wireless gateway sends the images captured within the preset shooting time to the server;

[0012] The server stores the images captured within the preset shooting time and sends them to the mobile terminal;

[0013] The mobile terminal displays a prompt message, which prompts the driver to view the images taken within the preset time period.

[0014] In one possible approach, before the domain controller controls the plurality of cameras to begin capturing images of the area around the target vehicle, the method further includes:

[0015] When the target vehicle is parked and locked, and the tilt sensor detects a change in the tilt angle between the target vehicle's body and the designated plane, the domain controller wakes up the multiple cameras.

[0016] In one possible approach, before the domain controller wakes up the plurality of cameras, the method further includes:

[0017] The domain controller receives alarm notifications sent by the anti-theft device of the target vehicle.

[0018] In one possible embodiment, the anti-theft device is a fingerprint reader;

[0019] Before the domain controller receives the alarm notification sent by the anti-theft device of the target vehicle, the method further includes:

[0020] When the fingerprint information recognized by the fingerprint reader does not match the pre-stored fingerprint information, an alarm notification is sent to the domain controller.

[0021] Secondly, a vehicle monitoring system is provided, comprising multiple pressure sensors, cameras, an in-vehicle terminal, and a mobile terminal, a server, and a target vehicle. The target vehicle has multiple cameras, multiple ultrasonic sensors, a tilt sensor, an anti-theft device, a domain controller, and a wireless gateway, wherein:

[0022] The domain controller is configured to wake up the multiple cameras when the target vehicle is parked and locked, and the number of times the periodic detection data of any ultrasonic sensor changes within a specified time exceeds a preset threshold; and control the multiple cameras to start capturing images of the area around the target vehicle.

[0023] The multiple cameras are used to send captured images to the domain controller; when the shooting time of the multiple cameras reaches the preset shooting time, the shooting stops and the camera enters a sleep state.

[0024] The domain controller is further configured to input the images captured by the plurality of cameras within a preset shooting time into a security identification model. If the output result of the security identification model is secure, the images captured within the preset shooting time are deleted. If the output result of the security identification model is insecure, the images captured within the preset shooting time are stored locally and sent to the wireless gateway. The security identification model is a machine learning model.

[0025] The wireless gateway is used to send the images captured within the preset shooting time to the server;

[0026] The server is used to store the images captured within the preset shooting time and send them to the mobile terminal.

[0027] The mobile terminal is used to display prompt information, wherein the prompt information is used to prompt the driver to view the images taken within the preset time period.

[0028] In one possible embodiment, the domain controller is further configured to:

[0029] When the target vehicle is parked and locked, and the tilt sensor detects a change in the tilt angle between the target vehicle's body and the designated plane, the multiple cameras are activated.

[0030] In one possible embodiment, the domain controller is further configured to:

[0031] The domain controller receives alarm notifications sent by the anti-theft device of the target vehicle.

[0032] In one possible embodiment, the anti-theft device is characterized by being a fingerprint reader;

[0033] The fingerprint reader is used to send an alarm notification to the domain controller when the fingerprint information it recognizes does not match the pre-stored fingerprint information.

[0034] Thirdly, a computer device group is provided, including a plurality of computer devices, each computer device including a memory and a processor, the memory for storing computer program instructions; the processor executes the computer program instructions stored in the memory to cause the computer device group to perform the method of the first aspect and its possible implementations.

[0035] Fourthly, a computer-readable storage medium is provided that stores computer program instructions, and in response to the computer program instructions being executed by a group of computer devices, the group of computer devices performs the method of the first aspect and its possible implementations.

[0036] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0037] The method provided in this application allows for the following scenario: when a target vehicle is parked and locked, if any sensor detects movement of an object around the vehicle, the target vehicle's domain controller activates multiple cameras to capture images. The target vehicle can then use a security identification model to identify the captured images. If the identification result is deemed unsafe, the domain controller can save the captured images, upload them to a server, and notify the user to view them. In this way, if the movement of an object causes damage to the target vehicle, the user can retrieve the captured images from the domain controller or server to locate the perpetrator or vehicle and obtain financial compensation. This reduces the possibility of property damage. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the structure of a vehicle monitoring system provided in an embodiment of this application;

[0040] Figure 2 This is a schematic diagram of the structure of a mobile terminal provided in an embodiment of this application;

[0041] Figure 3 This is a schematic diagram of the structure of a server provided in an embodiment of this application;

[0042] Figure 4 This is a flowchart illustrating a vehicle monitoring method provided in an embodiment of this application;

[0043] Figure 5 This is a flowchart illustrating a vehicle monitoring method provided in an embodiment of this application;

[0044] Figure 6 This is a flowchart illustrating a vehicle monitoring method provided in an embodiment of this application;

[0045] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0046] This application provides a method for vehicle monitoring, which is applied to a vehicle monitoring system. The structure of the system can be as follows: Figure 1As shown, the system includes a mobile terminal, a server, and a target vehicle. The target vehicle is equipped with multiple cameras, ultrasonic sensors, tilt sensors, an anti-theft device, a domain controller, and a wireless gateway. This method is used to monitor the environment around the vehicle and can save relevant images. The mobile terminal can be a mobile phone, tablet, laptop, etc. The server can be a backend server for a related application; it can be a single server or a group of multiple devices.

[0047] The target vehicle may have multiple cameras, including panoramic cameras and multi-function cameras. Panoramic cameras can be mounted on the top of the front and rear bumpers, on the left and right doors, and on the left and right side mirrors, or in other locations on the vehicle. Multi-function cameras can be mounted below the windshield or in other locations on the vehicle. The camera mounted above the front bumper and the multi-function camera mounted below the windshield are primarily used to capture images directly in front of the vehicle. The multi-function camera mounted below the windshield has a higher shooting height than the camera mounted above the front bumper, making it more likely to capture facial images. The camera mounted above the rear bumper is primarily used to capture images directly behind the vehicle. The camera mounted on the left door and the camera on the left side mirror are primarily used to capture images to the left of the vehicle. The camera mounted on the right door and the camera on the left side mirror are primarily used to capture images to the right of the vehicle. The field of view of these cameras depends on the type of camera. In practical applications, wide-angle cameras or cameras with a normal viewing angle can be used. Ultrasonic sensors can be installed on the top of the front and rear bumpers of the target vehicle, as well as on the left and right doors, or at other locations on the target vehicle. An ultrasonic sensor installed on the top of the front bumper is primarily used to detect objects in front of the target vehicle. An ultrasonic sensor installed on the top of the rear bumper is primarily used to detect objects behind the target vehicle. An ultrasonic sensor installed on the left door is primarily used to detect objects to the left of the target vehicle. An ultrasonic sensor installed on the right door is primarily used to detect objects to the right of the target vehicle. Tilt sensors can be installed on the chassis of the target vehicle or at other locations on the target vehicle. Anti-theft devices can be installed on the door locks of the target vehicle or at other locations on the target vehicle. Anti-theft devices can be electronic or chip-based. The domain controller connects to multiple cameras and sensors, enabling image recognition, data processing, and other functions. A wireless gateway is used for wireless communication between the domain controller and the server.

[0048] From a hardware perspective, the structure of a mobile terminal can be as follows: Figure 2 As shown, it includes a processor 210, a memory 220, a display unit 230, and a communication unit 240.

[0049] The processor 210 can be a CPU (central processing unit) or a SoC (system on chip), etc. The processor 210 can be used to execute various instructions involved in the method.

[0050] The memory 220 may include various volatile or non-volatile memories, such as SSD (solid state disk) and DRAM (dynamic random access memory). The memory 220 can be used to store pre-stored data, intermediate data, and result data during the display of prompt information, such as images captured within a preset time period.

[0051] The display component 230 can be a standalone screen, or a screen integrated with the terminal body, a projector, etc. The screen can be a touch screen or a non-touch screen. The display component is used to display images captured within a preset time period.

[0052] The communication component 240 can be a wired network connector, a WiFi (wireless fidelity) module, a Bluetooth module, a cellular network communication module, etc. The communication component can be used to transmit data with other devices, such as servers or other terminals.

[0053] In addition to processors and memory, terminals may also include audio acquisition components, audio output components, etc.

[0054] The audio acquisition component can be a microphone, used to capture the user's voice. The audio output component can be a speaker, headphones, etc., used to play audio.

[0055] From a hardware perspective, the server's structure can be as follows: Figure 3 As shown, it includes a processor 310, a memory 320, and a communication component 330.

[0056] The processor 310 can be a CPU or SoC, etc., and the processor 310 can be used to execute various instructions involved in the method.

[0057] The memory 320 may include various volatile or non-volatile memories, such as SSDs and DRAM. The memory 320 can be used to store pre-stored data, intermediate data, and result data of various messages sent by the wireless gateway, such as images captured within a preset time period.

[0058] The communication component 330 can be a wired network connector, a WiFi module, a Bluetooth module, a cellular network communication module, etc. The communication component can be used to transmit data with other devices, such as servers or other terminals. For example, a server can send images to a mobile terminal.

[0059] In the automotive industry, after parking their vehicles, drivers may encounter collisions or minor collisions with other people or vehicles, resulting in scratches or scrapes. In such cases, drivers may need video evidence to identify the responsible party or vehicle in order to obtain subsequent financial compensation.

[0060] This application provides a vehicle monitoring method for the above-mentioned application scenarios. The processing flow of this method is as follows: Figure 4 As shown, the processing steps include the following:

[0061] 401. When the target vehicle is parked and locked, and the number of times the periodic detection data of any ultrasonic sensor changes within a specified time exceeds a preset threshold, the domain controller wakes up multiple cameras.

[0062] In implementation, after the user parks the target vehicle in a parking space, they lock the vehicle. When the target vehicle's domain controller detects that the vehicle is off and locked, it activates multiple ultrasonic sensors. These sensors emit ultrasonic waves periodically, which interact with objects around the vehicle. The objects reflect the ultrasonic waves, which are then received by the sensors. The ultrasonic sensors can receive ultrasonic waves reflected from different locations on the object surfaces, generating a 3D depth map with a certain resolution. The generated 3D depth map is then sent to the domain controller. The domain controller inputs the 3D depth map received in the previous cycle and the 3D depth map received in the current cycle into a similarity model to obtain a similarity score. When the similarity score is lower than a similarity threshold, a change is recorded. This similarity model can be a machine learning model, such as a neural network model. If, after a specified time, the number of recorded changes exceeds a preset threshold, it indicates that there is movement around the target vehicle. This could be due to someone moving around the vehicle, or other vehicles reversing or parking. These activities could potentially damage the target vehicle, in which case the target vehicle's domain controller activates multiple cameras.

[0063] 402. The domain controller controls multiple cameras to begin capturing images of the area around the target vehicle.

[0064] In practice, to capture a comprehensive view of the scene surrounding the target vehicle, all cameras mounted on the exterior of the vehicle can be controlled to begin capturing images of the surrounding area. The multiple cameras on the target vehicle can then stitch together the real-time images to obtain a bird's-eye view of the area around the vehicle.

[0065] 403, multiple cameras send images to the domain controller.

[0066] In practice, multiple cameras can send their own images to the domain controller in real time, or they can send a stitched bird's-eye view image to the domain controller in real time.

[0067] 404. When the shooting time of multiple cameras reaches the preset shooting time, shooting will stop and the camera will enter sleep mode.

[0068] The preset shooting duration can be 5 minutes.

[0069] In practice, when the shooting time of multiple cameras reaches the preset shooting time, in order not to waste the processing and storage resources of the target vehicle, the multiple cameras can stop shooting and enter a sleep state.

[0070] 405. The domain controller inputs images captured by multiple cameras within a preset shooting time into the security identification model. If the output of the security identification model is secure, the images captured within the preset shooting time are deleted. If the output of the security identification model is insecure, the images captured within the preset shooting time are stored locally and sent to the wireless gateway.

[0071] The safety recognition model is a pre-trained machine learning model, such as a neural network model. During training, the target vehicle's domain controller pre-stores safe and dangerous samples. Users can add images of themselves and their families around the target vehicle to the safe samples, resulting in updated safe samples. The safety recognition model is trained to maximize the difference between the updated safe and dangerous samples. The safety recognition model that reaches the training termination condition is then used as the trained safety recognition model.

[0072] In implementation, the domain controller inputs images captured by each camera within a preset shooting time into the security identification model. The model then comprehensively assesses the safety of the target vehicle based on these images. If the model outputs "safe," the target vehicle is considered safe, and the images are deemed useless and can be deleted to save storage space on the domain controller. If the model outputs "unsafe," the target vehicle is considered safe, and the images may be used as evidence; in this case, the images can be stored locally. Alternatively, the images captured within the preset shooting time can be sent to the wireless gateway to alert the user.

[0073] 406. The wireless gateway sends the images captured within the preset shooting time to the server.

[0074] Among them, the wireless gateway is a forwarding station for the domain controller to communicate wirelessly with the outside world.

[0075] 407. The server will store the images captured within the preset shooting time and send them to the mobile terminal.

[0076] In practice, the server can store images captured within a preset shooting time, allowing users to easily view and retrieve them later. Additionally, the server can send images captured within the preset shooting time to mobile devices, prompting users to review the images and confirm the safety of the target vehicle.

[0077] 408, a prompt message is displayed on the mobile terminal.

[0078] The notification message prompts the driver to review images captured by multiple cameras on the target vehicle within a preset time period. This notification can be a text message or a pop-up notification within an application.

[0079] In practice, users can click on links in SMS notifications or pop-up notifications in the application. The mobile terminal recognizes the user's click and displays images. Users can then use images captured by multiple cameras within a preset time period to determine if the target vehicle is safe. If the vehicle is safe, the user can delete the images captured by the cameras within the preset time period. If the vehicle is unsafe, the user can go to the vehicle's parking location to inspect it. Additionally, users can use the images captured by the target vehicle's multiple cameras within the preset time period to identify the vehicle or person responsible for the accident.

[0080] This application embodiment also provides a vehicle monitoring method for the above-mentioned application scenarios. The processing flow of this method can be as follows: Figure 5 As shown, the processing steps include the following:

[0081] 501. When the target vehicle is parked and locked, and the tilt angle of the target vehicle body relative to the specified plane changes as detected by the tilt sensor, the domain controller wakes up multiple cameras.

[0082] The designated plane can be the horizontal ground where the target vehicle is currently located.

[0083] In practice, after the user parks the target vehicle in a parking space, they lock the vehicle. Once the target vehicle's domain controller recognizes that the vehicle is off and locked, it activates the vehicle's tilt sensor. When the target vehicle is collided with by another vehicle or forcefully pushed, the tilt sensor detects a change in the vehicle's tilt angle relative to a designated plane. At this point, the domain controller activates multiple cameras.

[0084] 502, The domain controller controls multiple cameras to begin capturing images of the area around the target vehicle.

[0085] 503, multiple cameras send the images they capture to the domain controller.

[0086] 504. When the shooting time of multiple cameras reaches the preset shooting time, shooting will stop and the camera will enter sleep mode.

[0087] 505. The domain controller inputs images captured by multiple cameras within a preset shooting time into the security identification model. If the output of the security identification model is secure, the images captured within the preset shooting time are deleted. If the output of the security identification model is insecure, the images captured within the preset shooting time are stored locally and sent to the wireless gateway.

[0088] 506. The wireless gateway sends the images captured within the preset shooting time to the server.

[0089] 507. The server will store the images captured within the preset shooting time and send them to the mobile terminal.

[0090] 508, a prompt message is displayed on the mobile terminal.

[0091] The specific processing of steps 502-508 is similar to that of steps 402-408. Please refer to the relevant descriptions of steps 402-408, which will not be repeated here.

[0092] This application embodiment also provides a vehicle monitoring method for the above-mentioned application scenarios. The processing flow of this method can be as follows: Figure 6 As shown, the processing steps include the following:

[0093] 601, The anti-theft device of the target vehicle responds to the trigger condition and sends an alarm notification to the domain controller.

[0094] The anti-theft device can be a fingerprint reader, which can be installed on the door handle of the target vehicle.

[0095] In practice, after parking the target vehicle in a parking space, the user locks the vehicle. Once the target vehicle's domain controller detects that the vehicle is off and locked, it activates the vehicle's anti-theft system. If the anti-theft system is a fingerprint scanner, if someone touches the door handle, the scanner will identify the person's fingerprint. If the fingerprint identified by the scanner does not match pre-stored fingerprint information, or if the number of times the fingerprint identified by the scanner does not match pre-stored fingerprint information reaches a preset threshold, an alarm notification is sent to the domain controller.

[0096] 602, the domain controller receives an alarm notification from the target vehicle's anti-theft device, waking up multiple cameras.

[0097] 603, The domain controller controls multiple cameras to begin capturing images of the area around the target vehicle.

[0098] 604, multiple cameras send images to the domain controller.

[0099] 605. When the shooting time of multiple cameras reaches the preset shooting time, the shooting will stop and the camera will enter sleep mode.

[0100] 606. The domain controller inputs images captured by multiple cameras within a preset shooting time into the security identification model. If the output of the security identification model is secure, the images captured within the preset shooting time are deleted. If the output of the security identification model is insecure, the images captured within the preset shooting time are stored locally and sent to the wireless gateway.

[0101] 607. The wireless gateway sends the images captured within the preset shooting time to the server.

[0102] 608. The server will store the images captured within the preset shooting time and send them to the mobile terminal.

[0103] 609, The mobile terminal displays a prompt message.

[0104] The specific processing of steps 603-609 is similar to that of steps 402-408. Please refer to the relevant explanations for steps 402-408, which will not be repeated here.

[0105] The three trigger domain controllers described above can be used individually, in pairs, or all at once to control multiple cameras to start capturing images around the target vehicle.

[0106] The method provided in this application allows for the following scenario: when a target vehicle is parked and locked, if any sensor detects movement of an object around the vehicle, the target vehicle's domain controller activates multiple cameras to capture images. The target vehicle can then use a security identification model to identify the captured images. If the identification result is deemed unsafe, the domain controller can save the captured images, upload them to a server, and notify the user to view them. In this way, if the movement of an object causes damage to the target vehicle, the user can retrieve the captured images from the domain controller or server to locate the perpetrator or vehicle and obtain financial compensation. This reduces the possibility of property damage.

[0107] Based on the same technical concept, embodiments of this application also provide a vehicle monitoring system, such as... Figure 1 As shown, the vehicle monitoring system includes a mobile terminal, a server, and a target vehicle. The target vehicle has multiple cameras, multiple ultrasonic sensors, a tilt sensor, an anti-theft device, a domain controller, and a wireless gateway, among which:

[0108] The domain controller is used to wake up multiple cameras when the target vehicle is parked and locked, and the number of times the periodic detection data of any ultrasonic sensor changes within a specified time exceeds a preset threshold; and to control the multiple cameras to start capturing images of the area around the target vehicle.

[0109] Multiple cameras are used to send captured images to the domain controller; when the shooting time of multiple cameras reaches the preset shooting time, shooting stops and the system enters sleep mode.

[0110] The domain controller is also used to input images captured by multiple cameras within a preset shooting time into a security identification model. If the output result of the security identification model is secure, the images captured within the preset shooting time are deleted. If the output result of the security identification model is insecure, the images captured within the preset shooting time are stored locally and sent to the wireless gateway. The security identification model is a machine learning model.

[0111] A wireless gateway is used to send images captured within a preset shooting time to a server.

[0112] The server is used to store images captured within a preset shooting time and send them to the mobile terminal.

[0113] The mobile terminal is used to display prompts, which in turn prompt the driver to view the images taken within a preset time period.

[0114] In one possible approach, the domain controller is also used for:

[0115] When the target vehicle is parked and locked, and the tilt sensor detects a change in the tilt angle between the vehicle body and the designated plane, multiple cameras are activated.

[0116] In one possible embodiment, the domain controller is further configured to:

[0117] The domain controller receives alarm notifications sent by the anti-theft device of the target vehicle.

[0118] In one possible embodiment, the anti-theft device is characterized by being a fingerprint reader;

[0119] The fingerprint reader sends an alarm notification to the domain controller when the detected fingerprint information does not match the pre-stored fingerprint information.

[0120] The system provided in this application allows for the following operation: When a target vehicle is parked and locked, if any sensor detects movement of an object around the vehicle, the target vehicle's domain controller activates multiple cameras to capture images. The target vehicle can then use a security identification model to identify the captured images. If the identification result is deemed unsafe, the domain controller can save the captured images, upload them to a server, and notify the user to view them. In this way, if the movement of an object causes damage to the target vehicle, the user can retrieve the captured images from the domain controller or server to locate the perpetrator or vehicle responsible for the incident and obtain financial compensation. This reduces the likelihood of property damage.

[0121] It should be noted that the vehicle monitoring system provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle monitoring system and the vehicle monitoring method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0122] Figure 7A structural block diagram of an electronic device 700 provided in an embodiment of this application is shown. This electronic device can be a mobile terminal as described in the above embodiments. The electronic device 700 can be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The electronic device 700 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0123] Typically, electronic device 700 includes a processor 701 and a memory 702.

[0124] Processor 701 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 701 may be implemented using at least one hardware form selected from DSP (digital signal processing), FPGA (field-programmable gate array), and PLA (programmable logic array). Processor 701 may also include a main processor and a coprocessor. The main processor, also known as a CPU (central processing unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 701 may integrate a GPU (graphics processing unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 701 may also include an AI (artificial intelligence) processor, which is used to handle computational operations related to machine learning.

[0125] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 702 are used to store at least one instruction, which is executed by the processor 701 to implement the method provided in the embodiments of this application.

[0126] In some embodiments, the electronic device 700 may optionally include a peripheral device interface 703 and at least one peripheral device. The processor 701, memory 702, and peripheral device interface 703 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, a positioning assembly 708, and a power supply 707.

[0127] Peripheral device interface 703 can be used to connect at least one I / O (input / output) related peripheral device to processor 701 and memory 702. In some embodiments, processor 701, memory 702 and peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 701, memory 702 and peripheral device interface 703 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0128] The radio frequency (RF) circuit 704 is used to receive and transmit RF (radio frequency) signals, also known as electromagnetic signals. The RF circuit 704 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 704 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (wireless fidelity) networks. In some embodiments, the RF circuit 704 may also include circuitry related to NFC (near field communication), which is not limited in this application.

[0129] Display screen 705 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 705 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 701 for processing. In this case, display screen 705 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 705, disposed on the front panel of electronic device 700; in other embodiments, there may be at least two display screens, disposed on different surfaces of electronic device 700 or in a folded design; in still other embodiments, display screen 705 may be a flexible display screen, disposed on a curved or folded surface of electronic device 700. Furthermore, display screen 705 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 705 may be made of materials such as LCD (liquid crystal display) or OLED (organic light-emitting diode).

[0130] The camera assembly 706 is used to acquire images or videos. Optionally, the camera assembly 706 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (virtual reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0131] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 701 for processing, or input to the radio frequency circuit 704 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located in a different part of the electronic device 700. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0132] The positioning component 708 is used to locate the current geographic location of the electronic device 700 in order to enable navigation or LBS (location-based service). The positioning component 708 can be a positioning component based on GPS (global positioning system), BeiDou system, or Galileo system.

[0133] Power supply 709 is used to supply power to various components in electronic device 700. Power supply 709 can be alternating current, direct current, a disposable battery, or a rechargeable battery. When power supply 709 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0134] In some embodiments, the electronic device 700 further includes one or more sensors 710. The one or more sensors 710 include, but are not limited to: an accelerometer 711, a gyroscope 712, a pressure sensor 713, a fingerprint sensor 714, an optical sensor 715, and a proximity sensor 716.

[0135] Accelerometer 711 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established by electronic device 700. For example, accelerometer 711 can be used to detect the components of gravitational acceleration on the three coordinate axes. Processor 701 can control display screen 705 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 711. Accelerometer 711 can also be used for games or for acquiring user motion data.

[0136] The gyroscope sensor 712 can detect the orientation and rotation angle of the electronic device 700. The gyroscope sensor 712, in conjunction with the accelerometer sensor 711, can collect 3D motion data from the user on the electronic device 700. Based on the data collected by the gyroscope sensor 712, the processor 701 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0137] The pressure sensor 713 can be disposed on the side bezel of the electronic device 700 and / or on the lower layer of the display screen 705. When the pressure sensor 713 is disposed on the side bezel of the electronic device 700, it can detect the user's grip signal on the electronic device 700, and the processor 701 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 713. When the pressure sensor 713 is disposed on the lower layer of the display screen 705, the processor 701 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 705. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0138] The fingerprint sensor 714 is used to collect a user's fingerprint. The processor 701 identifies the user based on the fingerprint collected by the fingerprint sensor 714, or vice versa. When the user's identity is verified as trusted, the processor 701 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 714 can be located on the front, back, or side of the electronic device 700. When the electronic device 700 has a physical button or manufacturer logo, the fingerprint sensor 714 can be integrated with the physical button or manufacturer logo.

[0139] An optical sensor 715 is used to collect ambient light intensity. In one embodiment, the processor 701 can control the display brightness of the display screen 705 based on the ambient light intensity collected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the display screen 705 is increased; when the ambient light intensity is low, the display brightness of the display screen 705 is decreased. In another embodiment, the processor 701 can also dynamically adjust the shooting parameters of the camera assembly 706 based on the ambient light intensity collected by the optical sensor 715.

[0140] 716, also known as a distance sensor, is typically located on the front panel of the electronic device 700. 716 is used to detect the distance between the user and the front of the electronic device 700. In one embodiment, when 716 detects that the distance between the user and the front of the electronic device 700 is gradually decreasing, the processor 701 controls the display screen 705 to switch from a screen-on state to a screen-off state; when 716 detects that the distance between the user and the front of the electronic device 700 is gradually increasing, the processor 701 controls the display screen 705 to switch from a screen-off state to a screen-on state.

[0141] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on the electronic device 700, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0142] In this application embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to complete the method of performing interactive operations in the above embodiments. This computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be ROM (read-only memory), RAM (random access memory), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0143] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0144] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0145] The above description is only some possible embodiments of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for vehicle monitoring, characterized in that, The method is applied to a vehicle monitoring system, which includes a mobile terminal, a server, and a target vehicle. The target vehicle has multiple cameras, multiple ultrasonic sensors, a tilt sensor, an anti-theft device, a domain controller, and a wireless gateway. When the target vehicle is parked and locked, the ultrasonic sensor receives ultrasonic waves reflected from different positions on the surface of the object and generates a three-dimensional depth map. The ultrasonic sensor sends the three-dimensional depth map to the domain controller; The domain controller inputs the 3D depth map received in the previous cycle and the 3D depth map received in the current cycle into the similarity model to obtain the similarity. When the similarity is lower than the similarity threshold, a change is recorded. In response to detecting that someone touches the door handle of the vehicle, the fingerprint of the person touching the door handle of the vehicle is identified to obtain a fingerprint recognition detection result, which is used to indicate whether the fingerprint of the person touching the door handle of the vehicle matches a pre-stored fingerprint. In response to at least one of the following events: the number of times the periodic detection data of any ultrasonic sensor changes within a specified time exceeds a preset threshold, or the detection result that the fingerprint of the person touching the door handle of the vehicle does not match the pre-stored fingerprint is obtained; the domain controller wakes up the plurality of cameras. The domain controller controls the multiple cameras to start capturing images around the target vehicle, and stitches the real-time captured images together to obtain a bird's-eye view of the target vehicle's surroundings. The multiple cameras will send the stitched bird's-eye view image to the domain controller in real time; When the shooting time of the multiple cameras reaches the preset shooting time, shooting stops and the camera enters a sleep state. The domain controller inputs a bird's-eye view image, stitched together from images captured by the multiple cameras within a preset shooting time, into a security recognition model. If the security recognition model outputs a secure result, the images captured within the preset shooting time and the bird's-eye view image stitched together from those images are deleted. If the security recognition model outputs an insecure result, the images captured within the preset shooting time are stored locally and sent to the wireless gateway. The security recognition model is a machine learning model. The wireless gateway sends the images captured within the preset shooting time to the server; The server stores the images captured within the preset shooting time and sends them to the mobile terminal; The mobile terminal displays a prompt message, which prompts the driver to view the images taken within the preset shooting time.

2. The method according to claim 1, characterized in that, Before the domain controller controls the plurality of cameras to begin capturing images of the area around the target vehicle, the method further includes: When the target vehicle is parked and locked, and the tilt sensor detects a change in the tilt angle between the target vehicle's body and the designated plane, the domain controller wakes up the multiple cameras.

3. The method according to claim 1, characterized in that, Before the domain controller wakes up the plurality of cameras, the method further includes: The domain controller receives alarm notifications sent by the anti-theft device of the target vehicle.

4. The method according to claim 3, characterized in that, The anti-theft device is a fingerprint reader; Before the domain controller receives the alarm notification sent by the anti-theft device of the target vehicle, the method further includes: When the fingerprint information recognized by the fingerprint reader does not match the pre-stored fingerprint information, an alarm notification is sent to the domain controller.

5. A vehicle monitoring system, characterized in that, The vehicle monitoring system includes a mobile terminal, a server, and a target vehicle. The target vehicle has multiple cameras, multiple ultrasonic sensors, a tilt sensor, an anti-theft device, a domain controller, and a wireless gateway, wherein: The domain controller is used to receive ultrasonic waves reflected from different positions on the surface of the object and generate a three-dimensional depth map when the target vehicle is parked and locked. The ultrasonic sensor sends the three-dimensional depth map to the domain controller; The domain controller inputs the 3D depth map received in the previous cycle and the 3D depth map received in the current cycle into the similarity model to obtain the similarity. When the similarity is lower than the similarity threshold, a change is recorded. In response to detecting that someone touches the door handle of the vehicle, the fingerprint of the person touching the door handle of the vehicle is identified to obtain a fingerprint recognition detection result, which is used to indicate whether the fingerprint of the person touching the door handle of the vehicle matches a pre-stored fingerprint. In response to at least one of the following: when the number of changes in the periodic detection data of any ultrasonic sensor within a specified time exceeds a preset threshold, or when a detection result is obtained that the fingerprint of the person touching the door handle of the vehicle does not match the pre-stored fingerprint, the multiple cameras are activated; the multiple cameras are controlled to start capturing images around the target vehicle, and the real-time captured images are stitched together to obtain a bird's-eye view image of the target vehicle. The multiple cameras are used to send the stitched bird's-eye view image to the domain controller in real time; when the shooting time of the multiple cameras reaches the preset shooting time, the shooting stops and enters a sleep state. The domain controller is further configured to input a bird's-eye view image stitched together from images captured by the multiple cameras within a preset shooting time into a security recognition model. If the output result of the security recognition model is secure, the images captured within the preset shooting time and the bird's-eye view image stitched together from the images captured within the preset shooting time are deleted. If the output result of the security recognition model is insecure, the images captured within the preset shooting time are stored locally and sent to the wireless gateway. The security recognition model is a machine learning model. The wireless gateway is used to send the images captured within the preset shooting time to the server; The server is used to store the images captured within the preset shooting time and send them to the mobile terminal. The mobile terminal is used to display prompt information, wherein the prompt information is used to prompt the driver to view the images captured within the preset shooting time.

6. The system according to claim 5, characterized in that, The domain controller is also used for: When the target vehicle is parked and locked, and the tilt sensor detects a change in the tilt angle between the target vehicle's body and the designated plane, the multiple cameras are activated.

7. The system according to claim 5, characterized in that, The domain controller is also used for: The domain controller receives alarm notifications sent by the anti-theft device of the target vehicle.

8. The system according to claim 7, characterized in that, The anti-theft device is a fingerprint reader; The fingerprint reader is used to send an alarm notification to the domain controller when the fingerprint information it recognizes does not match the pre-stored fingerprint information.

9. A computer device group, characterized in that, It includes multiple computer devices, each of which includes a processor and memory; The processors of the plurality of computer devices are configured to execute instructions stored in the memory of the plurality of computer devices to cause the group of computer devices to perform the method as described in any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a group of computer devices, enable the group of computer devices to perform the method of any one of claims 1-4.

Images