Vehicle leaving reminding method and device, computing equipment and computer program product

Abstract

The embodiment of the invention provides a vehicle leaving reminding method. The method comprises the following steps: determining a target tracking object in a vehicle; initiating a visual tracking process based on the target tracking object; obtaining a vehicle state corresponding to the vehicle, and determining vehicle leaving intention information corresponding to the target user according to the vehicle state; determining detection information corresponding to the target tracking object through a visual tracking process when the vehicle leaves; and if the detection information indicates that the target tracking object is in the vehicle, initiating a missing prompt. The target tracking object is tracked, and the tracking process is called by judging the vehicle leaving intention of the user, so that the missing reminding aiming at the target tracking object is initiated when the target tracking object is detected after the user leaves the vehicle, the missing of the object is avoided, and the user experience is improved. And the safety of articles in the vehicle is improved.

Description

Technical Field

[0001] The present specification relates to the field of computer application technology, specifically, to vehicle departure reminder technology under the field of computer application technology, and more specifically, to a vehicle departure reminder method, device, computing equipment and computer program product. Background Art

[0002] With the development of automobile technology, vehicles are appearing more and more frequently in people's lives. During use, people are prone to leaving items in vehicles. Generally, the left-behind reminder in vehicles only supports reminders of specific items in a small range, such as radio frequency detection of vehicle keys in a specific location; this method cannot be adapted to other items that users often use, which can easily cause items to be left behind, affecting the safety of items in the car. Summary of the invention

[0003] The embodiments of this specification provide a vehicle leaving reminder method, device, computing device and computer program product to implement a personalized vehicle leaving reminder function.

[0004] To achieve the above technical objectives, the embodiments of this specification provide the following technical solutions:

[0005] In a first aspect, an embodiment of the present specification provides a vehicle leaving reminder method, comprising:

[0006] In response to an interactive operation of a target user in a vehicle, determining a target tracking object inside the vehicle;

[0007] Initiate a visual tracking process based on the target tracking object, the visual tracking process is used to instruct the vehicle to maintain visual tracking of the target tracking object through an in-vehicle camera during driving;

[0008] Acquiring a vehicle state corresponding to the vehicle, so as to determine the vehicle leaving intention information corresponding to the target user according to the vehicle state;

[0009] If the leaving-the-vehicle intention information indicates that the target user leaves the vehicle, determining detection information corresponding to the target tracking object through the visual tracking process;

[0010] If the detection information indicates that the target tracking object is inside the vehicle, a lost reminder for the target tracking object is initiated.

[0011] In a second aspect, an embodiment of the present specification provides a vehicle leaving reminder device, comprising:

[0012] An interaction unit, configured to determine a target tracking object inside the vehicle in response to an interaction operation of a target user in the vehicle;

[0013] a processing unit, configured to initiate a visual tracking process based on the target tracking object, wherein the visual tracking process is configured to instruct the vehicle to maintain visual tracking of the target tracking object through an in-vehicle camera during driving;

[0014] The processing unit is further used to obtain a vehicle state corresponding to the vehicle, so as to determine the vehicle leaving intention information corresponding to the target user according to the vehicle state;

[0015] The processing unit is further configured to determine detection information corresponding to the target tracking object through the visual tracking process if the leaving-the-vehicle intention information indicates that the target user leaves the vehicle;

[0016] The interaction unit is further configured to initiate a lost reminder for the target tracking object if the detection information indicates that the target tracking object is inside the vehicle.

[0017] In a third aspect, an embodiment of the present specification further provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the vehicle leaving reminder method as described above when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present specification further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the vehicle leaving reminder method as described above is implemented.

[0019] In a fifth aspect, an embodiment of the present specification provides a computer program product or a computer program, wherein the computer program product includes a computer program, and the computer program can be stored in a computer-readable storage medium or in the cloud; the processor of the computer device reads the computer program, and the processor implements the steps of the above-mentioned vehicle departure reminder method when executing the computer program.

[0020] It can be seen from the above technical solutions that the vehicle leaving reminder method provided in the embodiment of this specification determines the target tracking object inside the vehicle by responding to the interactive operation of the target user in the vehicle; then initiates a visual tracking process based on the target tracking object, and the visual tracking process is used to instruct the vehicle to maintain visual tracking of the target tracking object through the in-vehicle camera during driving; and obtains the vehicle state corresponding to the vehicle to determine the vehicle leaving intention information corresponding to the target user according to the vehicle state; if the vehicle leaving intention information indicates that the target user leaves the vehicle, the detection information corresponding to the target tracking object is determined through the visual tracking process; if the detection information indicates that the target tracking object is inside the vehicle, a lost reminder for the target tracking object is initiated. Thus, a personalized vehicle object lost reminder process is realized. Since the target tracking object is kept tracked, and the tracking process is called by judging the user's intention to leave the vehicle, a lost reminder for the target tracking object is initiated when the target tracking object is detected after the user leaves the vehicle, thereby avoiding the loss of items and improving the safety of items in the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0022] Figure 1 A schematic diagram of an application environment of a vehicle leaving reminder method provided in one embodiment of this specification;

[0023] Figure 2 A flowchart of a vehicle leaving reminder method provided for one embodiment of this specification;

[0024] Figure 3 A schematic diagram of a scenario of a vehicle leaving reminder method provided for one embodiment of this specification;

[0025] Figure 4 A schematic diagram of another vehicle leaving reminder method provided for one embodiment of this specification;

[0026] Figure 5 A schematic diagram of another vehicle leaving reminder method provided for one embodiment of this specification;

[0027] Figure 6 A schematic diagram of another vehicle leaving reminder method provided for one embodiment of this specification;

[0028] Figure 7A schematic diagram of another vehicle leaving reminder method provided for one embodiment of this specification;

[0029] Figure 8 A schematic diagram of another vehicle leaving reminder method provided for one embodiment of this specification;

[0030] Fig. 9 A schematic diagram of the functional modules of a vehicle leaving reminder device provided in one embodiment of this specification;

[0031] Fig.10 A schematic diagram of the structure of a computing device provided for one embodiment of the present specification. DETAILED DESCRIPTION

[0032] Unless otherwise defined, the technical terms or scientific terms used in the embodiments of this specification shall have the common meanings understood by persons with ordinary skills in the field to which this specification belongs. The words "first", "second" and similar words used in the embodiments of this specification do not indicate any order, quantity or importance, but are only used to avoid confusion of constituent elements.

[0033] Unless the context requires otherwise, throughout the specification, "plurality" means "at least two", and "including" is interpreted as an open, inclusive meaning, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "examples", "specific examples" or "some examples" are intended to indicate that a particular feature, structure, material or characteristic associated with the embodiment or example is included in at least one embodiment or example of the specification. The schematic representation of the above terms does not necessarily refer to the same embodiment or example.

[0034] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0035] With the development of automobile technology, vehicles are appearing more and more frequently in people's lives, and people are prone to leaving items in vehicles. Generally, the left-behind reminder in vehicles only supports reminders of specific items in a small range, such as radio frequency detection of vehicle keys in a specific location; this method cannot be adapted to other items that users often use, which can easily cause items to be left behind, affecting the safety of items in the car.

[0036] To solve the above problems, the embodiments of this specification provide a vehicle leaving reminder system, and the vehicle leaving reminder method provided in the embodiments of this specification is applied to the vehicle leaving reminder system. The reminder system tracks and detects the target tracking object in the vehicle, and determines the object position when the user leaves the vehicle to prevent items from being left behind.

[0037] Specifically, the vehicle leaving reminder system may include: Figure 1 The operating environment formed by the client 110, server 120 and vehicle 130 in the embodiment. The client 110 communicates with the server 120 through the network. The client 110 is wirelessly connected to the vehicle 130. The vehicle 130 communicates with the server 120 through a network connection. Among them, the client 110 can be an electronic device with network access capability. Specifically, for example, the client 110 can be a desktop computer, a tablet computer, a laptop computer, a smart phone, a digital assistant, a smart wearable device, a shopping guide terminal, a television, a smart speaker, a microphone, etc. Among them, smart wearable devices include but are not limited to smart bracelets, smart watches, smart glasses, smart helmets, smart necklaces, etc. Alternatively, the client 110 can also be software that can be run in the electronic device. The server 120 can be an electronic device with certain computing and processing capabilities. It can have a network communication module, a processor and a memory, etc. Of course, the server 120 can also refer to software running in the electronic device. The server 120 can also be a distributed server, which can be a system with multiple processors, memories, network communication modules, etc. operating in collaboration. Alternatively, the server 120 can also be a cluster formed by several servers 120. Alternatively, with the development of science and technology, the server 120 may also be a new technical means capable of realizing the corresponding functions of the embodiments of the specification, for example, a new form of "server" based on quantum computing.

[0038] Specifically, when setting reminders in the vehicle based on the above reminder system, the target tracking object inside the vehicle is determined by responding to the interactive operation of the target user in the vehicle; then a visual tracking process is initiated based on the target tracking object, and the visual tracking process is used to instruct the vehicle to maintain visual tracking of the target tracking object through the in-vehicle camera during driving; and the vehicle state corresponding to the vehicle is obtained to determine the target user's corresponding leaving intention information according to the vehicle state; if the leaving intention information indicates that the target user leaves the vehicle, the detection information corresponding to the target tracking object is determined through the visual tracking process; if the detection information indicates that the target tracking object is inside the vehicle, a lost reminder for the target tracking object is initiated. Thus, a personalized in-vehicle object lost reminder process is realized. Since the target tracking object is kept tracked, and the tracking process is called by judging the user's intention to leave the vehicle, a lost reminder for the target tracking object is initiated when the target tracking object is detected after the user leaves the vehicle, thereby avoiding the loss of items and improving the safety of items in the vehicle.

[0039] Based on the above concept, an embodiment of the present specification provides a vehicle leaving reminder method. The vehicle leaving reminder method provided by the embodiment of the present specification will be exemplarily described below in conjunction with the accompanying drawings.

[0040] To be applied to Figure 1 Taking the vehicle in as an example, some embodiments of this specification exemplarily illustrate the vehicle leaving reminder method, and the vehicle leaving reminder method includes:

[0041] 201. In response to an interactive operation of a target user in a vehicle, determine a target tracking object inside the vehicle.

[0042] In this embodiment, the target tracking object inside the vehicle is the object that the target user needs to pay attention to, which can be a stationary object such as a mobile phone, wallet, keys and other items; it can also be an object that may be in motion, such as an animal, a baby, etc.; the specific target tracking object can be one or more, that is, there is no limit on the number of target tracking objects determined by the target user's interactive operations in the vehicle, and the specific number depends on the actual scenario.

[0043] Specifically, this embodiment performs targeted visual tracking of items that users are interested in in the car to prevent items from being left in the car. Among them, visual tracking is achieved through object tracking, which is an important task in the field of computer vision. It can continuously track the movement and changes of objects in the scene by analyzing image sequences.

[0044] In a possible scenario, the process of determining the target tracking object inside the vehicle can be performed by the target user's selection operation in the interactive interface; and the display content in the interactive interface can be the in-vehicle image or video captured by the in-vehicle camera. Therefore, for the determination process of the target tracking object, the in-vehicle camera can be first called up to collect the in-vehicle image through the in-vehicle camera; wherein the in-vehicle camera can be based on the configuration of the occupant monitoring system (OMS), which is a system in the vehicle that monitors the occupants and objects in the city based on visual perception and analysis technology. Monitor and analyze the items in the vehicle, specifically, a top-mounted, high-field-of-view, high-resolution camera can be used to achieve in-vehicle monitoring. Then, the in-vehicle image is displayed based on the vehicle-related interactive interface, wherein the vehicle-related interactive interface can be the vehicle-related vehicle interface, or it can be the interface of the vehicle-related mobile terminal, such as the vehicle control interface in the vehicle control application; and then in response to the target user's interactive operation on the in-vehicle image, the target tracking object inside the vehicle is determined.

[0045] Specifically, the target user's interactive operation on the in-car image in the above scenario may be a selection operation, such as Figure 3 As shown, Figure 3 A scene diagram of a vehicle leaving reminder method provided for one embodiment of the present specification; the figure shows an in-vehicle image displayed in the vehicle computer interface, and the user determines the target tracking object by selecting the in-vehicle image (sliding box selection).

[0046] Furthermore, considering that the selection process of the in-vehicle image may involve multiple different object elements, the selection can be made after the corresponding elements are identified in the image, so as to improve the accuracy of the corresponding object range of the target tracking object. Specifically, the in-vehicle image can be firstly recognized to obtain the annotation information; then the vehicle image can be annotated based on the annotation information; and then the target tracking object inside the vehicle can be determined in response to the target user's interactive operation on the annotated in-vehicle image. Specifically, Figure 4 As shown, Figure 4 A scene diagram of another vehicle leaving reminder method provided for an embodiment of the present specification; the figure shows that after image analysis and edge extraction based on the imaging of the in-vehicle camera, common target objects such as mobile phones, clothes, footballs, wallets, computers, keys, etc. are identified, so as to facilitate subsequent determination of the target objects that the user needs to track based on the user's operation, and form their image feature information, and add them to the watch list to assist the execution of the visual tracking process.

[0047] In another possible scenario, the target tracking object can also be carried out through voice recognition, that is, after the image is recognized, the description information of the corresponding object can be obtained, so that matching and recognition can be carried out through voice. Specifically, firstly, the voice control command initiated by the target user is obtained; then, in response to the voice control command, the annotated in-vehicle image is matched with the annotations to determine the target tracking object inside the vehicle. For example, Figure 5 In the scenario shown, Figure 5 A schematic diagram of a scenario of another method of leaving the vehicle to remind an embodiment of the present specification; the diagram shows that the user can directly wake up the voice assistant and say "help me pay attention to the phone", thereby triggering visual tracking of the phone (target tracking object); and the car assistant can feedback "OK, paid attention", indicating that the phone has been visually locked.

[0048] In another possible scenario, since the objects that the user is paying attention to have a certain trend, that is, the objects that the user is paying attention to may be the same or multiple, a direct call can be made at this time. Specifically, the process of directly calling the target tracking object can be based on the forgotten reminder mode, that is, first responding to the target user's interactive operation on the in-vehicle image, start the forgotten reminder mode; then responding to the start of the forgotten reminder mode, call the historical tracking information, which is based on the selection record of the target tracking object, that is, the historical record of the target tracking object in the visual tracking process, and specifically can select the most recent one, or can select the one within the tracking period (1 month), and then count the number of tracking of different items, and select the one with the most number of times for recommendation; then determine the target tracking object inside the vehicle based on the historical tracking information. The activation of the forgotten reminder mode can be initiated by the user through the vehicle computer interface, or it can be initiated through the terminal control interface, for example, in Figure 6 In the scenario shown, Figure 6 A scenario diagram of another method for leaving the vehicle reminder provided for an embodiment of the present specification; the figure shows that the left-behind reminder mode is activated through a mobile phone APP before getting on the vehicle, so that a seamless left-behind reminder process is performed after getting on the vehicle. By calling the historical tracking data of the cloud server, the target tracking object can be determined and the corresponding visual tracking process is executed, thereby simplifying the user's driving process.

[0049] It should be noted that the in-vehicle camera used in this embodiment can be one (for example, an in-vehicle high-angle camera) or multiple, and different in-vehicle cameras can use the same angle of view or different in-vehicle angles to facilitate multi-angle tracking of the target tracking object, that is, by comparing images from different angles to calibrate the position of the target tracking object. Figure 7 As shown, Figure 7A scene diagram of another vehicle exit reminder method provided for an embodiment of the present specification; the figure shows a football detected by a side (B-pillar) camera, which can determine that the football is placed on the seat, so as to facilitate a detailed description of the position of the football and improve the accuracy of target tracking.

[0050] 202. Initiate a visual tracking process based on the target tracking object, where the visual tracking process is used to instruct the vehicle to maintain visual tracking of the target tracking object through an in-vehicle camera during driving.

[0051] In this embodiment, the visual tracking process is the execution process corresponding to the occupant monitoring system, which is used to detect and track the occupants and objects in the vehicle; wherein, the visual tracking process is a program running in the background of the vehicle processor, which does not occupy the display resources of the vehicle interface during driving; in addition, the data cache generated by its tracking process may include an image sequence based on time configuration, or a video within a time period, or trajectory information containing image elements, and the specific data form depends on the actual scenario.

[0052] Specifically, the visual tracking process can be based on the attention list, that is, firstly, edge recognition is performed on the target tracking object to obtain the target recognition object; then, feature extraction is performed on the target recognition object to obtain the target tracking feature; and the attention list is configured based on the target tracking feature; then, according to the feature guidance of the attention list, the visual tracking process is initiated. That is, based on the user's operation, the target object that the user needs to track is determined, and its image feature information is formed and added to the attention list. During the user's driving process, the visual tracking of the objects in the car is maintained through the attention list. Accordingly, the position change information of the target tracking object can be recorded through the attention list, so as to facilitate the subsequent determination of the object position.

[0053] It should be noted that the visual tracking process in this embodiment requires the user's authorization before it can be triggered. When it is found that the user has not authorized it, a reminder can be given; the specific reminder method includes a function prompt in the mobile terminal asking whether to authorize in order to obtain the user's authorization, such as the prompt "whether to turn on the item anti-loss mode", and the prompt process can also be displayed through the vehicle interface.

[0054] 203. Obtain a vehicle state corresponding to the vehicle to determine the target user's exit intention information corresponding to the vehicle according to the vehicle state.

[0055] In this embodiment, determining the leaving intention information corresponding to the target user is a process of judging whether the user has a tendency to leave the vehicle in combination with the vehicle status (main driver's seat, door, power system, etc.).

[0056] Specifically, the configuration of the vehicle status can be multi-dimensional, that is, first obtain the driving status corresponding to the vehicle to determine the driving information, such as the vehicle speed and gear position and other information indicating the driving status; and determine the vehicle components associated with the target user for driving control to obtain the component information corresponding to the vehicle components, such as the seat occupancy turntable and the status information of the switch status light component of the main driver's door; then combine the driving information and component information to judge the vehicle status to obtain the vehicle status; and determine the corresponding leaving intention information of the target user according to the vehicle status. Therefore, before each user leaves the car, the user's intention to leave the car can be comprehensively judged through information such as seat occupancy, main driver's door, vehicle speed and gear position. For example, if the main driver's seat is not occupied, the main driver's door is open, the vehicle speed is 0 and the gear is in P gear, then the user's intention to leave the car is judged as the user leaving the car.

[0057] 204. If the leaving-the-vehicle intention information indicates that the target user leaves the vehicle, the detection information corresponding to the target tracking object is determined through a visual tracking process.

[0058] In this embodiment, when it is determined that the user is about to leave the vehicle, the position of the target tracking object is detected, that is, the detection information is information obtained by detecting the position of the target tracking object.

[0059] Specifically, the detection range of detection information can be configured based on a time period. For example, if the image position of the object of interest does not change during the period when the user intends to leave the vehicle (for example, 1 minute), a left-behind item reminder is triggered, and the user is reminded through the in-vehicle screen, voice, etc., thereby avoiding the situation where the user temporarily gets off the vehicle but does not leave the vehicle, and the left-behind reminder is mistakenly triggered.

[0060] 205. If the detection information indicates that the target tracking object is inside the vehicle, a lost reminder for the target tracking object is initiated.

[0061] In this embodiment, when the user leaves the vehicle, the camera inside the vehicle is used to determine whether the target tracking object is inside the vehicle; if the item selected by the user is recognized, it is considered that the left-behind item is detected, and the vehicle computer reminds through sound and light, etc. The specific reminder can be in the form of inside the vehicle, outside the vehicle, or a combination of the two, such as a combination of one or more of the inside-vehicle screen, outside-vehicle voice playback, outside-vehicle light flashing, etc. to remind the user.

[0062] In one possible scenario, the target tracking object may be left in a corner that cannot be detected by the camera inside the vehicle due to vehicle bumps or human movement. At this time, the position of the target tracking object can be determined in combination with the inertial data after leaving the vehicle. Since the inertial data needs to be configured with the corresponding inertial data acquisition module, the corresponding inertial sensor can be configured for the target tracking object, such as an inertial sensor attached to the surface of an object.

[0063] Specifically, the mobile terminal itself has the configuration of inertial sensors, and due to its high frequency of use, special attention needs to be paid. The following takes the mobile terminal as an example. Considering that the mobile phone is easy to slip into the gap between the seats or fall on the floor, the visual tracking process does not detect the mobile phone at this time, which may result in misjudgment. Therefore, the inertial data of the mobile phone can be combined to determine whether the mobile phone is lost, such as Figure 8 As shown, Figure 8 A scenario diagram of another method for reminding a user to leave the vehicle is provided for an embodiment of the present specification; the figure shows that when the user is detected leaving the vehicle, if the mobile phone selected by the user is not detected, the user can be prompted to get off the vehicle, and a secondary detection is performed based on the inertial data of the mobile phone; specifically, the signal is transmitted to the mobile phone through the vehicle-mobile phone channel (Bluetooth, LTE, etc.), and the mobile phone performs inertial navigation recognition of the mobile phone within the time window, determines the displacement of the mobile phone, and distinguishes whether the mobile phone is left in the car.

[0064] Specifically, if the detection information indicates that the mobile terminal is not detected, the inertial data of the mobile terminal in the acquisition window can be obtained, wherein the acquisition window can be within a preset time period after the user leaves the car, such as 1 minute; then the displacement information of the mobile terminal in the acquisition window is determined based on the inertial data; if the displacement information indicates that the mobile terminal has not moved, a lost reminder for the target tracking object is initiated. That is, if the mobile phone is displaced, it means that the mobile phone has been taken away by the person in the car, and there is no need to issue a lost reminder in the car; if the mobile phone is not displaced, it means that the mobile phone may have fallen into a corner in the car, thereby reminding the user.

[0065] Optionally, in the process of reminding the user, data backtracking can be performed based on the camera data to indicate the possible lost location. That is, when the displacement information indicates that the mobile terminal has not moved, the tracking data indicated by the visual tracking process is called, and the tracking data is data containing the object location, such as images and videos arranged based on the acquisition time; then the lost location of the mobile terminal is determined based on the tracking data; Dong Er configures the reminder information based on the lost location to initiate a lost reminder for the target tracking object according to the reminder information, thereby improving the accuracy of the lost reminder.

[0066] In summary, this embodiment determines the target tracking object inside the vehicle by responding to the target user's interactive operation in the vehicle; then initiates a visual tracking process based on the target tracking object, and the visual tracking process is used to instruct the vehicle to maintain visual tracking of the target tracking object through the in-vehicle camera during driving; and obtains the vehicle state corresponding to the vehicle to determine the target user's corresponding leaving intention information according to the vehicle state; if the leaving intention information indicates that the target user leaves the vehicle, the detection information corresponding to the target tracking object is determined through the visual tracking process; if the detection information indicates that the target tracking object is inside the vehicle, a lost reminder for the target tracking object is initiated. Thus, a personalized in-vehicle object lost reminder process is realized. Since the target tracking object is kept tracked, and the tracking process is called by judging the user's intention to leave the vehicle, a lost reminder for the target tracking object is initiated when the target tracking object is detected after the user leaves the vehicle, thereby avoiding the loss of items and improving the safety of items in the vehicle.

[0067] It should be noted that the multiple embodiments in this specification emphasize the parts that are different from other embodiments, and the various embodiments can be explained in comparison with each other. Any combination of the multiple embodiments in this specification by technicians in the relevant field based on general technical knowledge is included in the disclosure scope of this specification.

[0068] In an exemplary embodiment of the present specification, a vehicle leaving reminder device 900 is also provided, comprising:

[0069] An interaction unit 901, configured to determine a target tracking object inside the vehicle in response to an interaction operation of a target user in the vehicle;

[0070] A processing unit 902 is used to initiate a visual tracking process based on the target tracking object, wherein the visual tracking process is used to instruct the vehicle to maintain visual tracking of the target tracking object through an in-vehicle camera during driving;

[0071] The processing unit 902 is further configured to obtain a vehicle state corresponding to the vehicle, so as to determine the vehicle leaving intention information corresponding to the target user according to the vehicle state;

[0072] The processing unit 902 is further configured to determine detection information corresponding to the target tracking object through the visual tracking process if the leaving-the-vehicle intention information indicates that the target user leaves the vehicle;

[0073] The interaction unit 901 is further configured to initiate a lost reminder for the target tracking object if the detection information indicates that the target tracking object is inside the vehicle.

[0074] Optionally, in some possible embodiments, the interaction unit 901 is specifically used to call up the in-vehicle camera to collect the in-vehicle image through the in-vehicle camera;

[0075] The interaction unit 901 is specifically configured to display the in-vehicle image based on the interaction interface associated with the vehicle;

[0076] The interaction unit 901 is specifically configured to determine the target tracking object inside the vehicle in response to the target user's interaction operation on the in-vehicle image.

[0077] Optionally, in some possible embodiments, the interaction unit 901 is specifically configured to perform image recognition on the in-vehicle image to obtain annotation information;

[0078] The interaction unit 901 is specifically configured to annotate the vehicle image based on the annotation information;

[0079] The interaction unit 901 is specifically configured to determine the target tracking object inside the vehicle in response to the target user's interaction operation on the labeled in-vehicle image.

[0080] Optionally, in some possible embodiments, the interaction unit 901 is specifically configured to obtain the voice control instruction initiated by the target user;

[0081] The interaction unit 901 is specifically configured to perform annotation matching on the annotated in-vehicle image in response to the voice control instruction to determine the target tracking object inside the vehicle.

[0082] Optionally, in some possible embodiments, the interaction unit 901 is specifically configured to start a forgotten reminder mode in response to an interaction operation of the target user on the in-vehicle image;

[0083] The interaction unit 901 is specifically configured to call historical tracking information in response to the start of the lost reminder mode, where the historical tracking information is determined based on the selection record of the target tracking object;

[0084] The interaction unit 901 is specifically configured to determine the target tracking object inside the vehicle based on the historical tracking information.

[0085] Optionally, in some possible embodiments, the processing unit 902 is specifically configured to perform edge recognition on the target tracking object to obtain a target recognition object;

[0086] The processing unit 902 is specifically used to extract features of the target recognition object to obtain target tracking features;

[0087] The processing unit 902 is specifically configured to configure a watch list based on the target tracking feature;

[0088] The processing unit 902 is specifically configured to initiate the visual tracking process according to the feature guidance of the attention list.

[0089] Optionally, in some possible embodiments, the processing unit 902 is specifically configured to obtain a driving state corresponding to the vehicle to determine driving information;

[0090] The processing unit 902 is specifically configured to determine a vehicle component of the vehicle associated with the target user for driving control, so as to obtain component information corresponding to the vehicle component;

[0091] The processing unit 902 is specifically used to determine the vehicle state by combining the driving information and the component information to obtain the vehicle state;

[0092] The processing unit 902 is specifically configured to determine the vehicle leaving intention information corresponding to the target user according to the vehicle status.

[0093] Optionally, in some possible embodiments, the processing unit 902 is specifically configured to acquire inertial data of the mobile terminal within a collection window if the detection information indicates that the mobile terminal is not detected;

[0094] The processing unit 902 is specifically configured to determine the displacement information of the mobile terminal within the acquisition window according to the inertial data;

[0095] The processing unit 902 is specifically configured to initiate a lost reminder for the target tracking object if the displacement information indicates that the mobile terminal has not moved.

[0096] Optionally, in some possible embodiments, the processing unit 902 is specifically configured to call the tracking data indicated by the visual tracking process if the displacement information indicates that the mobile terminal has not moved;

[0097] The processing unit 902 is specifically configured to determine a lost location of the mobile terminal according to the tracking data;

[0098] The processing unit 902 is specifically configured to configure reminder information based on the lost location, so as to initiate the lost reminder according to the reminder information.

[0099] Specifically, the processing unit and the interaction unit in this embodiment may correspond to physical components. For example, the processing unit may be a processing module such as a CPU, a GPU, or a FPGA, and the interaction unit may be an interaction module such as a screen, a speaker, or a projector. The specific physical components may be any components or combinations of components having the above functions. The specific method depends on the actual scenario and is not limited here.

[0100] The above-mentioned reminder device determines the target tracking object inside the vehicle by responding to the interactive operation of the target user in the vehicle; then initiates a visual tracking process based on the target tracking object, and the visual tracking process is used to instruct the vehicle to maintain visual tracking of the target tracking object through the in-vehicle camera during driving; and obtains the vehicle state corresponding to the vehicle to determine the target user's corresponding leaving intention information according to the vehicle state; if the leaving intention information indicates that the target user leaves the vehicle, the detection information corresponding to the target tracking object is determined through the visual tracking process; if the detection information indicates that the target tracking object is inside the vehicle, a lost reminder for the target tracking object is initiated. Thus, a personalized in-vehicle object lost reminder process is realized. Since the target tracking object is kept tracked, and the tracking process is called by judging the user's intention to leave the vehicle, a lost reminder for the target tracking object is initiated when the target tracking object is detected after the user leaves the vehicle, thereby avoiding the loss of items and improving the safety of items in the vehicle.

[0101] For the specific limitations of the vehicle departure reminder device, please refer to the limitations of the vehicle departure reminder method mentioned above, which will not be repeated here. The various unit modules in the above-mentioned vehicle departure reminder device can be implemented in whole or in part through software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0102] Another embodiment of the present application further provides a computing device, see Fig.10 As shown, an exemplary embodiment of the present specification also provides a computing device, including: a memory and a processor, the memory stores a computer program, and the processor executes the steps of the vehicle departure reminder method described in the above embodiments of the present specification according to various embodiments of the present specification when executing the computer program.

[0103] The internal structure of the computing device can be as follows Fig.10As shown, the computing device includes a processor, a memory, a network interface and an input device connected through a system bus. Among them, the processor of the computing device is used to provide computing and control capabilities. The memory of the computing device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computing device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps of the vehicle leaving reminder method according to various embodiments of the present specification are described in the above embodiments of the present specification.

[0104] The processor may include a main processor and may also include a baseband chip, a modem, etc.

[0105] The memory stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes computer operation instructions. More specifically, the memory may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0106] The processor may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0107] The input device may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0108] Output devices may include means that allow information to be output to a user, such as display screens, printers, speakers, etc.

[0109] The communication interface may include using any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0110] The processor executes the program stored in the memory and calls other devices, which can be used to implement each step of any one of the vehicle departure reminder methods provided in the above embodiments of the present application.

[0111] The computing device may also include a display component and a voice component. The display component may be a liquid crystal display or an electronic ink display. The input device of the computing device may be a touch layer covered on the display component, or a button, trackball or touchpad provided on the housing of the computing device, or an external keyboard, touchpad or mouse.

[0112] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of this specification, and does not constitute a limitation on the computing device to which the scheme of this specification is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0113] In addition to the above-mentioned methods and devices, the vehicle departure reminder method provided in the embodiments of this specification may also be a computer program product, which includes a computer program, which, when executed by a processor, enables the processor to execute the steps of the vehicle departure reminder method according to various embodiments of this specification described in the above "Exemplary Method" section of this specification.

[0114] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present specification, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0115] In addition, an embodiment of the present specification also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the vehicle departure reminder method according to various embodiments of the present specification described in the above "Exemplary Method" section of the present specification.

[0116] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this specification can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0117] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above-mentioned embodiments only express several implementation methods of this specification, and the descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the solutions provided by the embodiments of this specification. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of this specification, which all belong to the protection scope of this specification. Therefore, the protection scope of the patent of this specification shall be based on the attached claims.

Claims

1. A vehicle leaving reminder method, characterized in that: include: In response to an interactive operation of a target user in a vehicle, determining a target tracking object inside the vehicle; Initiate a visual tracking process based on the target tracking object, the visual tracking process is used to instruct the vehicle to maintain visual tracking of the target tracking object through an in-vehicle camera during driving; Acquiring a vehicle state corresponding to the vehicle, so as to determine the vehicle leaving intention information corresponding to the target user according to the vehicle state; If the leaving-the-vehicle intention information indicates that the target user leaves the vehicle, determining detection information corresponding to the target tracking object through the visual tracking process; If the detection information indicates that the target tracking object is inside the vehicle, a lost reminder for the target tracking object is initiated.

2. The method according to claim 1, characterized in that The step of determining a target tracking object inside the vehicle in response to an interactive operation of a target user in the vehicle includes: Invoking the in-vehicle camera to collect an in-vehicle image through the in-vehicle camera; Displaying the in-vehicle image based on an interactive interface associated with the vehicle; In response to the target user's interactive operation on the in-vehicle image, the target tracking object inside the vehicle is determined.

3. The method according to claim 2, characterized in that The step of determining the target tracking object inside the vehicle in response to the target user's interactive operation on the in-vehicle image includes: Performing image recognition on the in-vehicle image to obtain annotation information; annotating the vehicle image based on the annotation information; In response to the target user's interactive operation on the labeled in-vehicle image, the target tracking object inside the vehicle is determined.

4. The method according to claim 3, characterized in that The interactive operation is a voice control instruction, and the determining the target tracking object inside the vehicle in response to the interactive operation of the target user on the labeled in-vehicle image includes: Acquire the voice control instruction initiated by the target user; In response to the voice control instruction, the annotated in-vehicle image is annotated and matched to determine the target tracking object inside the vehicle.

5. The method according to claim 2, characterized in that: The step of determining the target tracking object inside the vehicle in response to the target user's interactive operation on the in-vehicle image includes: In response to the target user's interactive operation on the in-vehicle image, starting a forgotten reminder mode; In response to the activation of the lost reminder mode, calling historical tracking information, the historical tracking information being determined based on a selection record of a target tracking object; The target tracking object inside the vehicle is determined based on the historical tracking information.

6. The method according to claim 1, characterized in that The initiating a visual tracking process based on the target tracking object includes: Performing edge recognition on the target tracking object to obtain a target recognition object; Extracting features from the target recognition object to obtain target tracking features; configuring a watchlist based on the target tracking feature; The visual tracking process is initiated according to the feature guidance of the focus list.

7. The method according to claim 1, characterized in that The acquiring a vehicle state corresponding to the vehicle to determine the vehicle leaving intention information corresponding to the target user according to the vehicle state includes: Acquire the driving state corresponding to the vehicle to determine the driving information; Determine a vehicle component of the vehicle associated with the target user for driving control, so as to obtain component information corresponding to the vehicle component; Combining the driving information and the component information to determine the vehicle state, so as to obtain the vehicle state; Determine the vehicle leaving intention information corresponding to the target user according to the vehicle status.

8. The method according to claim 1, characterized in that The target tracking object is a mobile terminal, and the method further includes: If the detection information indicates that the mobile terminal is not detected, acquiring inertial data of the mobile terminal within a collection window; Determining displacement information of the mobile terminal within the acquisition window according to the inertial data; If the displacement information indicates that the mobile terminal has not moved, a lost reminder for the target tracking object is initiated.

9. The method according to claim 8, characterized in that If the displacement information indicates that the mobile terminal has not moved, initiating a lost reminder for the target tracking object, including: If the displacement information indicates that the mobile terminal has not moved, calling the tracking data indicated by the visual tracking process; Determine the lost location of the mobile terminal according to the tracking data; Reminder information is configured based on the lost location, so as to initiate the lost reminder according to the reminder information.

10. A vehicle, characterized in that: The vehicle executes the vehicle departure reminder method as described in any one of claims 1-9.

11. A computing device, characterized in that: It comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the vehicle leaving reminder method as described in any one of claims 1 to 9 when executing the computer program.

12. A computer program product, characterized in that include: A computer program, which, when executed by a processor, implements the vehicle leaving reminder method described in any one of claims 1 to 9.

Images